Solutions Open Source

Référence de la classe tx_indexedsearch_indexer

Liste de tous les membres

Fonctions membres publiques

 hook_indexContent (&$pObj)
 backend_initIndexer ($id, $type, $sys_language_uid, $MP, $uidRL, $cHash_array=array(), $createCHash=FALSE)
 backend_setFreeIndexUid ($freeIndexUid, $freeIndexSetId=0)
 backend_indexAsTYPO3Page ($title, $keywords, $description, $content, $charset, $mtime, $crdate=0, $recordUid=0)
 init ()
 initializeExternalParsers ()
 indexTypo3PageContent ()
 splitHTMLContent ($content)
 getHTMLcharset ($content)
 convertHTMLToUtf8 ($content, $charset='')
 embracingTags ($string, $tagName, &$tagContent, &$stringAfter, &$paramList)
 typoSearchTags (&$body)
 extractLinks ($content)
 extractHyperLinks ($string)
 indexExternalUrl ($externalUrl)
 getUrlHeaders ($url)
 indexRegularDocument ($file, $force=FALSE, $contentTmpFile='', $altExtension='')
 readFileContent ($ext, $absFile, $cPKey)
 fileContentParts ($ext, $absFile)
 splitRegularContent ($content)
 charsetEntity2utf8 (&$contentArr, $charset)
 processWordsInArrays ($contentArr)
 procesWordsInArrays ($contentArr)
 bodyDescription ($contentArr)
 indexAnalyze ($content)
 analyzeHeaderinfo (&$retArr, $content, $key, $offset)
 analyzeBody (&$retArr, $content)
 metaphone ($word, $retRaw=FALSE)
 submitPage ()
 submit_grlist ($hash, $phash_x)
 submit_section ($hash, $hash_t3)
 removeOldIndexedPages ($phash)
 submitFilePage ($hash, $file, $subinfo, $ext, $mtime, $ctime, $size, $content_md5h, $contentParts)
 submitFile_grlist ($hash)
 submitFile_section ($hash)
 removeOldIndexedFiles ($phash)
 checkMtimeTstamp ($mtime, $phash)
 checkContentHash ()
 checkExternalDocContentHash ($hashGr, $content_md5h)
 is_grlist_set ($phash_x)
 update_grlist ($phash, $phash_x)
 updateTstamp ($phash, $mtime=0)
 updateSetId ($phash)
 updateParsetime ($phash, $parsetime)
 updateRootline ()
 getRootLineFields (&$fieldArr)
 removeLoginpagesWithContentHash ()
 includeCrawlerClass ()
 checkWordList ($wl)
 submitWords ($wl, $phash)
 freqMap ($freq)
 setT3Hashes ()
 setExtHashes ($file, $subinfo=array())
 md5inthash ($str)
 makeCHash ($paramArray)
 log_push ($msg, $key)
 log_pull ()
 log_setTSlogMessage ($msg, $errorNum=0)
 fe_headerNoCache (&$params, $ref)

Attributs publics

 $reasons
 $excludeSections = 'script,style'
 $external_parsers = array()
 $defaultGrList = '0,-1'
 $tstamp_maxAge = 0
 $tstamp_minAge = 0
 $maxExternalFiles = 0
 $forceIndexing = FALSE
 $crawlerActive = FALSE
 $defaultContentArray
 $wordcount = 0
 $externalFileCounter = 0
 $conf = array()
 $indexerConfig = array()
 $hash = array()
 $file_phash_arr = array()
 $contentParts = array()
 $content_md5h = ''
 $internal_log = array()
 $indexExternalUrl_content = ''
 $cHashParams = array()
 $freqRange = 32000
 $freqMax = 0.1
 $csObj
 $metaphoneObj
 $lexerObj

Documentation des fonctions membres

tx_indexedsearch_indexer::hook_indexContent ( &$  pObj  ) 

Parent Object (TSFE) Initialization

Paramètres:
object Parent Object (frontend TSFE object), passed by reference
Renvoie:
void

tx_indexedsearch_indexer::backend_initIndexer ( $  id,
$  type,
$  sys_language_uid,
$  MP,
$  uidRL,
$  cHash_array = array(),
$  createCHash = FALSE 
)

Initializing the "combined ID" of the page (phash) being indexed (or for which external media is attached)

Paramètres:
integer The page uid, &id=
integer The page type, &type=
integer sys_language uid, typically &L=
string The MP variable (Mount Points), &MP=
array Rootline array of only UIDs.
array Array of GET variables to register with this indexing
boolean If set, calculates a cHash value from the $cHash_array. Probably you will not do that since such cases are indexed through the frontend and the idea of this interface is to index non-cachable pages from the backend!
Renvoie:
void

tx_indexedsearch_indexer::backend_setFreeIndexUid ( $  freeIndexUid,
$  freeIndexSetId = 0 
)

Sets the free-index uid. Can be called right after backend_initIndexer()

Paramètres:
integer Free index UID
integer Set id - an integer identifying the "set" of indexing operations.
Renvoie:
void

tx_indexedsearch_indexer::backend_indexAsTYPO3Page ( $  title,
$  keywords,
$  description,
$  content,
$  charset,
$  mtime,
$  crdate = 0,
$  recordUid = 0 
)

Indexing records as the content of a TYPO3 page.

Paramètres:
string Title equivalent
string Keywords equivalent
string Description equivalent
string The main content to index
string The charset of the title, keyword, description and body-content. MUST BE VALID, otherwise nothing is indexed!
integer Last modification time, in seconds
integer The creation date of the content, in seconds
integer The record UID that the content comes from (for registration with the indexed rows)
Renvoie:
void

tx_indexedsearch_indexer::init (  ) 

Initializes the object. $this->conf MUST be set with proper values prior to this call!!!

Renvoie:
void

tx_indexedsearch_indexer::initializeExternalParsers (  ) 

Initialize external parsers

Renvoie:
void private
Voir également:
init()

tx_indexedsearch_indexer::indexTypo3PageContent (  ) 

Start indexing of the TYPO3 page

Renvoie:
void

tx_indexedsearch_indexer::splitHTMLContent ( $  content  ) 

Splits HTML content and returns an associative array, with title, a list of metatags, and a list of words in the body.

Paramètres:
string HTML content to index. To some degree expected to be made by TYPO3 (ei. splitting the header by ":")
Renvoie:
array Array of content, having keys "title", "body", "keywords" and "description" set.
Voir également:
splitRegularContent()

tx_indexedsearch_indexer::getHTMLcharset ( $  content  ) 

Extract the charset value from HTML meta tag.

Paramètres:
string HTML content
Renvoie:
string The charset value if found.

tx_indexedsearch_indexer::convertHTMLToUtf8 ( $  content,
$  charset = '' 
)

Converts a HTML document to utf-8

Paramètres:
string HTML content, any charset
string Optional charset (otherwise extracted from HTML)
Renvoie:
string Converted HTML

tx_indexedsearch_indexer::embracingTags ( $  string,
$  tagName,
&$  tagContent,
&$  stringAfter,
&$  paramList 
)

Finds first occurence of embracing tags and returns the embraced content and the original string with the tag removed in the two passed variables. Returns false if no match found. ie. useful for finding <title> of document or removing <script>-sections

Paramètres:
string String to search in
string Tag name, eg. "script"
string Passed by reference: Content inside found tag
string Passed by reference: Content after found tag
string Passed by reference: Attributes of the found tag.
Renvoie:
boolean Returns false if tag was not found, otherwise true.

tx_indexedsearch_indexer::typoSearchTags ( &$  body  ) 

Removes content that shouldn't be indexed according to TYPO3SEARCH-tags.

Paramètres:
string HTML Content, passed by reference
Renvoie:
boolean Returns true if a TYPOSEARCH_ tag was found, otherwise false.

tx_indexedsearch_indexer::extractLinks ( $  content  ) 

Extract links (hrefs) from HTML content and if indexable media is found, it is indexed.

Paramètres:
string HTML content
Renvoie:
void

tx_indexedsearch_indexer::extractHyperLinks ( $  string  ) 

Extracts all links to external documents from content string.

Paramètres:
string Content to analyse
Renvoie:
array Array of hyperlinks
Voir également:
extractLinks()

tx_indexedsearch_indexer::indexExternalUrl ( $  externalUrl  ) 

Index External URLs HTML content

Paramètres:
string URL, eg. "http://typo3.org/"
Renvoie:
void
Voir également:
indexRegularDocument()

tx_indexedsearch_indexer::getUrlHeaders ( $  url  ) 

Getting HTTP request headers of URL

Paramètres:
string The URL
integer Timeout (seconds?)
Renvoie:
mixed If no answer, returns false. Otherwise an array where HTTP headers are keys

tx_indexedsearch_indexer::indexRegularDocument ( $  file,
$  force = FALSE,
$  contentTmpFile = '',
$  altExtension = '' 
)

Indexing a regular document given as $file (relative to PATH_site, local file)

Paramètres:
string Relative Filename, relative to PATH_site. It can also be an absolute path as long as it is inside the lockRootPath (validated with t3lib_div::isAbsPath()). Finally, if $contentTmpFile is set, this value can be anything, most likely a URL
boolean If set, indexing is forced (despite content hashes, mtime etc).
string Temporary file with the content to read it from (instead of $file). Used when the $file is a URL.
string File extension for temporary file.
Renvoie:
void

tx_indexedsearch_indexer::readFileContent ( $  ext,
$  absFile,
$  cPKey 
)

Reads the content of an external file being indexed. The content from the external parser MUST be returned in utf-8!

Paramètres:
string File extension, eg. "pdf", "doc" etc.
string Absolute filename of file (must exist and be validated OK before calling function)
string Pointer to section (zero for all other than PDF which will have an indication of pages into which the document should be splitted.)
Renvoie:
array Standard content array (title, description, keywords, body keys)

tx_indexedsearch_indexer::fileContentParts ( $  ext,
$  absFile 
)

Creates an array with pointers to divisions of document.

Paramètres:
string File extension
string Absolute filename (must exist and be validated OK before calling function)
Renvoie:
array Array of pointers to sections that the document should be divided into

tx_indexedsearch_indexer::splitRegularContent ( $  content  ) 

Splits non-HTML content (from external files for instance)

Paramètres:
string Input content (non-HTML) to index.
Renvoie:
array Array of content, having the key "body" set (plus "title", "description" and "keywords", but empty)
Voir également:
splitHTMLContent()

tx_indexedsearch_indexer::charsetEntity2utf8 ( &$  contentArr,
$  charset 
)

Convert character set and HTML entities in the value of input content array keys

Paramètres:
array Standard content array
string Charset of the input content (converted to utf-8)
Renvoie:
void

tx_indexedsearch_indexer::processWordsInArrays ( $  contentArr  ) 

Processing words in the array from split*Content -functions

Paramètres:
array Array of content to index, see splitHTMLContent() and splitRegularContent()
Renvoie:
array Content input array modified so each key is not a unique array of words

tx_indexedsearch_indexer::procesWordsInArrays ( $  contentArr  ) 

Processing words in the array from split*Content -functions This function is only a wrapper because the function has been removed (see above).

Paramètres:
array Array of content to index, see splitHTMLContent() and splitRegularContent()
Renvoie:
array Content input array modified so each key is not a unique array of words
Obsolète:

tx_indexedsearch_indexer::bodyDescription ( $  contentArr  ) 

Extracts the sample description text from the content array.

Paramètres:
array Content array
Renvoie:
string Description string

tx_indexedsearch_indexer::indexAnalyze ( $  content  ) 

Analyzes content to use for indexing,

Paramètres:
array Standard content array: an array with the keys title,keywords,description and body, which all contain an array of words.
Renvoie:
array Index Array (whatever that is...)

tx_indexedsearch_indexer::analyzeHeaderinfo ( &$  retArr,
$  content,
$  key,
$  offset 
)

Calculates relevant information for headercontent

Paramètres:
array Index array, passed by reference
array Standard content array
string Key from standard content array
integer Bit-wise priority to type
Renvoie:
void

tx_indexedsearch_indexer::analyzeBody ( &$  retArr,
$  content 
)

Calculates relevant information for bodycontent

Paramètres:
array Index array, passed by reference
array Standard content array
Renvoie:
void

tx_indexedsearch_indexer::metaphone ( $  word,
$  retRaw = FALSE 
)

Creating metaphone based hash from input word

Paramètres:
string Word to convert
boolean If set, returns the raw metaphone value (not hashed)
Renvoie:
mixed Metaphone hash integer (or raw value, string)

tx_indexedsearch_indexer::submitPage (  ) 

Updates db with information about the page (TYPO3 page, not external media)

Renvoie:
void

tx_indexedsearch_indexer::submit_grlist ( $  hash,
$  phash_x 
)

Stores gr_list in the database.

Paramètres:
integer Search result record phash
integer Actual phash of current content
Renvoie:
void
Voir également:
update_grlist()

tx_indexedsearch_indexer::submit_section ( $  hash,
$  hash_t3 
)

Stores section $hash and $hash_t3 are the same for TYPO3 pages, but different when it is external files.

Paramètres:
integer phash of TYPO3 parent search result record
integer phash of the file indexation search record
Renvoie:
void

tx_indexedsearch_indexer::removeOldIndexedPages ( $  phash  ) 

Removes records for the indexed page, $phash

Paramètres:
integer phash value to flush
Renvoie:
void

tx_indexedsearch_indexer::submitFilePage ( $  hash,
$  file,
$  subinfo,
$  ext,
$  mtime,
$  ctime,
$  size,
$  content_md5h,
$  contentParts 
)

Updates db with information about the file

Paramètres:
array Array with phash and phash_grouping keys for file
string File name
array Array of "cHashParams" for files: This is for instance the page index for a PDF file (other document types it will be a zero)
string File extension determining the type of media.
integer Modification time of file.
integer Creation time of file.
integer Size of file in bytes
integer Content HASH value.
array Standard content array (using only title and body for a file)
Renvoie:
void

tx_indexedsearch_indexer::submitFile_grlist ( $  hash  ) 

Stores file gr_list for a file IF it does not exist already

Paramètres:
integer phash value of file
Renvoie:
void

tx_indexedsearch_indexer::submitFile_section ( $  hash  ) 

Stores file section for a file IF it does not exist

Paramètres:
integer phash value of file
Renvoie:
void

tx_indexedsearch_indexer::removeOldIndexedFiles ( $  phash  ) 

Removes records for the indexed page, $phash

Paramètres:
integer phash value to flush
Renvoie:
void

tx_indexedsearch_indexer::checkMtimeTstamp ( $  mtime,
$  phash 
)

Check the mtime / tstamp of the currently indexed page/file (based on phash) Return positive integer if the page needs to be indexed

Paramètres:
integer mtime value to test against limits and indexed page (usually this is the mtime of the cached document)
integer "phash" used to select any already indexed page to see what its mtime is.
Renvoie:
integer Result integer: Generally: <0 = No indexing, >0 = Do indexing (see $this->reasons): -2) Min age was NOT exceeded and so indexing cannot occur. -1) mtime matched so no need to reindex page. 0) N/A 1) Max age exceeded, page must be indexed again. 2) mtime of indexed page doesn't match mtime given for current content and we must index page. 3) No mtime was set, so we will index... 4) No indexed page found, so of course we will index.

tx_indexedsearch_indexer::checkContentHash (  ) 

Check content hash in phash table

Renvoie:
mixed Returns true if the page needs to be indexed (that is, there was no result), otherwise the phash value (in an array) of the phash record to which the grlist_record should be related!

tx_indexedsearch_indexer::checkExternalDocContentHash ( $  hashGr,
$  content_md5h 
)

Check content hash for external documents Returns true if the document needs to be indexed (that is, there was no result)

Paramètres:
integer phash value to check (phash_grouping)
integer Content hash to check
Renvoie:
boolean Returns true if the document needs to be indexed (that is, there was no result)

tx_indexedsearch_indexer::is_grlist_set ( $  phash_x  ) 

Checks if a grlist record has been set for the phash value input (looking at the "real" phash of the current content, not the linked-to phash of the common search result page)

Paramètres:
integer Phash integer to test.
Renvoie:
void

tx_indexedsearch_indexer::update_grlist ( $  phash,
$  phash_x 
)

Check if an grlist-entry for this hash exists and if not so, write one.

Paramètres:
integer phash of the search result that should be found
integer The real phash of the current content. The two values are different when a page with userlogin turns out to contain the exact same content as another already indexed version of the page; This is the whole reason for the grlist table in fact...
Renvoie:
void
Voir également:
submit_grlist()

tx_indexedsearch_indexer::updateTstamp ( $  phash,
$  mtime = 0 
)

Update tstamp for a phash row.

Paramètres:
integer phash value
integer If set, update the mtime field to this value.
Renvoie:
void

tx_indexedsearch_indexer::updateSetId ( $  phash  ) 

Update SetID of the index_phash record.

Paramètres:
integer phash value
Renvoie:
void

tx_indexedsearch_indexer::updateParsetime ( $  phash,
$  parsetime 
)

Update parsetime for phash row.

Paramètres:
integer phash value.
integer Parsetime value to set.
Renvoie:
void

tx_indexedsearch_indexer::updateRootline (  ) 

Update section rootline for the page

Renvoie:
void

tx_indexedsearch_indexer::getRootLineFields ( &$  fieldArr  ) 

Adding values for root-line fields. rl0, rl1 and rl2 are standard. A hook might add more.

Paramètres:
array Field array, passed by reference
Renvoie:
void

tx_indexedsearch_indexer::removeLoginpagesWithContentHash (  ) 

Removes any indexed pages with userlogins which has the same contentHash NOT USED anywhere inside this class!

Renvoie:
void

tx_indexedsearch_indexer::includeCrawlerClass (  ) 

Includes the crawler class

Renvoie:
void

tx_indexedsearch_indexer::checkWordList ( $  wl  ) 

Adds new words to db

Paramètres:
array Word List array (where each word has information about position etc).
Renvoie:
void

tx_indexedsearch_indexer::submitWords ( $  wl,
$  phash 
)

Submits RELATIONS between words and phash

Paramètres:
array Word list array
integer phash value
Renvoie:
void

tx_indexedsearch_indexer::freqMap ( $  freq  ) 

maps frequency from a real number in [0;1] to an integer in [0;$this->freqRange] with anything above $this->freqMax as 1 and back.

Paramètres:
double Frequency
Renvoie:
integer Frequency in range.

tx_indexedsearch_indexer::setT3Hashes (  ) 

Get search hash, T3 pages

Renvoie:
void

tx_indexedsearch_indexer::setExtHashes ( $  file,
$  subinfo = array() 
)

Get search hash, external files

Paramètres:
string File name / path which identifies it on the server
array Additional content identifying the (subpart of) content. For instance; PDF files are divided into groups of pages for indexing.
Renvoie:
array Array with "phash_grouping" and "phash" inside.

tx_indexedsearch_indexer::md5inthash ( $  str  ) 

md5 integer hash Using 7 instead of 8 just because that makes the integers lower than 32 bit (28 bit) and so they do not interfere with UNSIGNED integers or PHP-versions which has varying output from the hexdec function.

Paramètres:
string String to hash
Renvoie:
integer Integer intepretation of the md5 hash of input string.

tx_indexedsearch_indexer::makeCHash ( $  paramArray  ) 

Calculates the cHash value of input GET array (for constructing cHash values if needed)

Paramètres:
array Array of GET parameters to encode
Renvoie:
void

tx_indexedsearch_indexer::log_push ( $  msg,
$  key 
)

Push function wrapper for TT logging

Paramètres:
string Title to set
string Key (?)
Renvoie:
void

tx_indexedsearch_indexer::log_pull (  ) 

Pull function wrapper for TT logging

Renvoie:
void

tx_indexedsearch_indexer::log_setTSlogMessage ( $  msg,
$  errorNum = 0 
)

Set log message function wrapper for TT logging

Paramètres:
string Message to set
integer Error number
Renvoie:
void

tx_indexedsearch_indexer::fe_headerNoCache ( &$  params,
$  ref 
)

Frontend hook: If the page is not being re-generated this is our chance to force it to be (because re-generation of the page is required in order to have the indexer called!)

Paramètres:
array Parameters from frontend
object TSFE object (reference under PHP5)
Renvoie:
void

Documentation des données membres

tx_indexedsearch_indexer::$reasons

Valeur initiale :

 array(
                -1 => 'mtime matched the document, so no changes detected and no content updated',
                -2 => 'The minimum age was not exceeded',
                1 => "The configured max-age was exceeded for the document and thus it's indexed.",
                2 => 'The minimum age was exceed and mtime was set and the mtime was different, so the page was indexed.',
                3 => 'The minimum age was exceed, but mtime was not set, so the page was indexed.',
                4 => 'Page has never been indexed (is not represented in the index_phash table).'
        )

tx_indexedsearch_indexer::$excludeSections = 'script,style'

tx_indexedsearch_indexer::$external_parsers = array()

tx_indexedsearch_indexer::$defaultGrList = '0,-1'

tx_indexedsearch_indexer::$tstamp_maxAge = 0

tx_indexedsearch_indexer::$tstamp_minAge = 0

tx_indexedsearch_indexer::$maxExternalFiles = 0

tx_indexedsearch_indexer::$forceIndexing = FALSE

tx_indexedsearch_indexer::$crawlerActive = FALSE

tx_indexedsearch_indexer::$defaultContentArray

Valeur initiale :

array(
                'title' => '',
                'description' => '',
                'keywords' => '',
                'body' => '',
        )

tx_indexedsearch_indexer::$wordcount = 0

tx_indexedsearch_indexer::$externalFileCounter = 0

tx_indexedsearch_indexer::$conf = array()

tx_indexedsearch_indexer::$indexerConfig = array()

tx_indexedsearch_indexer::$hash = array()

tx_indexedsearch_indexer::$file_phash_arr = array()

tx_indexedsearch_indexer::$contentParts = array()

tx_indexedsearch_indexer::$content_md5h = ''

tx_indexedsearch_indexer::$internal_log = array()

tx_indexedsearch_indexer::$indexExternalUrl_content = ''

tx_indexedsearch_indexer::$cHashParams = array()

tx_indexedsearch_indexer::$freqRange = 32000

tx_indexedsearch_indexer::$freqMax = 0.1

tx_indexedsearch_indexer::$csObj

tx_indexedsearch_indexer::$metaphoneObj

tx_indexedsearch_indexer::$lexerObj

La documentation de cette classe a été générée à partir du fichier suivant :
Généré le Fri Aug 31 11:43:51 2007 pour OBLADY - Typo3 API v4.1.2 par  doxygen 1.5.3