|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.terrier.indexing.Indexer org.terrier.indexing.BasicIndexer org.terrier.indexing.BasicSinglePassIndexer org.terrier.indexing.BlockSinglePassIndexer
public class BlockSinglePassIndexer
Indexes a document collection saving block information for the indexed terms.
It performs a single pass inversion (see BasicSinglePassIndexer
).
All normal block properties are supported. For more information, see BlockIndexer
.
Nested Class Summary | |
---|---|
protected class |
BlockSinglePassIndexer.BasicTermProcessor
This class implements an end of a TermPipeline that adds the term to the DocumentTree. |
protected class |
BlockSinglePassIndexer.DelimFieldTermProcessor
This class behaves in a similar fashion to FieldTermProcessor except that this one treats blocks bounded by delimiters instead of fixed-sized blocks. |
protected class |
BlockSinglePassIndexer.DelimTermProcessor
This class behaves in a similar fashion to BasicTermProcessor except that this one treats blocks bounded by delimiters instead of fixed-sized blocks. |
protected class |
BlockSinglePassIndexer.FieldTermProcessor
This class implements an end of a TermPipeline that adds the term to the DocumentTree. |
Field Summary | |
---|---|
protected int |
BLOCK_SIZE
The maximum number of terms allowed in a block |
protected int |
blockId
The block number in the current document. |
protected int |
MAX_BLOCKS
The maximum number allowed number of blocks in a document. |
protected int |
numOfTokensInBlock
The number of tokens in the current block of the current document. |
Fields inherited from class org.terrier.indexing.BasicIndexer |
---|
numOfTokensInDocument, termFields, termsInDocument |
Constructor Summary | |
---|---|
BlockSinglePassIndexer(java.lang.String pathname,
java.lang.String prefix)
Constructs an instance of this block indexer which uses the single-pass strategy |
Method Summary | |
---|---|
protected void |
createDocumentPostings()
Hook method that creates the right type of DocumentTree class. |
protected void |
createFieldRunMerger(java.lang.String[][] files)
Hook method that creates a FieldRunMerger instance |
protected void |
createMemoryPostings()
Hook method that creates the right type of MemoryPostings class. |
protected void |
createRunMerger(java.lang.String[][] files)
Hook method that creates a RunsMerger instance |
protected TermPipeline |
getEndOfPipeline()
Returns the object that is to be the end of the TermPipeline. |
Methods inherited from class org.terrier.indexing.BasicSinglePassIndexer |
---|
checkFlush, createDirectIndex, createInvertedIndex, createInvertedIndex, finishMemoryPosting, forceFlush, getFileNames, indexDocument, load_indexer_properties, performMultiWayMerge |
Methods inherited from class org.terrier.indexing.BasicIndexer |
---|
finishedInvertedIndexBuild |
Methods inherited from class org.terrier.indexing.Indexer |
---|
createMetaIndexBuilder, finishedDirectIndexBuild, index, indexEmpty, init, load_builder_boundary_documents, load_field_ids, load_pipeline, main, merge, merge, mergeTwoIndices, parseInts, useFieldInformation |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected int numOfTokensInBlock
protected int blockId
protected int BLOCK_SIZE
protected int MAX_BLOCKS
Constructor Detail |
---|
public BlockSinglePassIndexer(java.lang.String pathname, java.lang.String prefix)
pathname
- String location of the indexprefix
- String prefix to file of the indexMethod Detail |
---|
protected TermPipeline getEndOfPipeline()
getEndOfPipeline
in class BasicIndexer
protected void createFieldRunMerger(java.lang.String[][] files) throws java.io.IOException
BasicSinglePassIndexer
createFieldRunMerger
in class BasicSinglePassIndexer
java.io.IOException
- if an I/O error occurs.protected void createRunMerger(java.lang.String[][] files) throws java.lang.Exception
BasicSinglePassIndexer
createRunMerger
in class BasicSinglePassIndexer
java.io.IOException
- if an I/O error occurs.
java.lang.Exception
protected void createMemoryPostings()
BasicSinglePassIndexer
createMemoryPostings
in class BasicSinglePassIndexer
protected void createDocumentPostings()
BasicIndexer
createDocumentPostings
in class BasicIndexer
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |