StructureMerger (Terrier 4.0 API)

Field Summary

Fields
Modifier and Type	Field and Description
`protected String`	`basicDirectIndexPostingIteratorClass`
`protected String`	`basicInvertedIndexPostingIteratorClass`
`protected CompressionFactory.CompressionConfiguration`	`compressionDirectConfig`
`protected CompressionFactory.CompressionConfiguration`	`compressionInvertedConfig`
`protected IndexOnDisk`	`destIndex` destination index
`protected Class<? extends DirectInvertedOutputStream>`	`directFileOutputStreamClass` class to use to write direct file
`protected int`	`fieldCount`
`protected Class<? extends DirectInvertedOutputStream>`	`fieldDirectFileOutputStreamClass`
`protected String`	`fieldDirectIndexPostingIteratorClass`
`protected String`	`fieldInvertedIndexPostingIteratorClass`
`protected boolean`	`keepTermCodeMap`
`protected static org.apache.log4j.Logger`	`logger` the logger used
`protected boolean`	`MetaReverse`
`protected int`	`numberOfDocuments` The number of documents in the merged structures.
`protected long`	`numberOfPointers` The number of pointers in the merged structures.
`protected int`	`numberOfTerms` The number of terms in the collection.
`protected IndexOnDisk`	`srcIndex1` source index 1
`protected IndexOnDisk`	`srcIndex2` source index 2
`protected gnu.trove.TIntIntHashMap`	`termcodeHashmap` A hashmap for converting the codes of terms appearing only in the vocabulary of the second set of data structures into a new set of term codes for the merged set of data structures.

Constructor Summary

Constructors
Constructor and Description

StructureMerger(IndexOnDisk _srcIndex1, IndexOnDisk _srcIndex2, IndexOnDisk _destIndex)
constructor

Constructors
Constructor and Description
`StructureMerger(IndexOnDisk _srcIndex1, IndexOnDisk _srcIndex2, IndexOnDisk _destIndex)` constructor

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`createLexidFile()` creates the final term code to offset file, and the lexicon hash if enabled.
`protected static Class<?>[]`	`getInterfaces(Object o)`
`static void`	`main(String[] args)` Usage: java org.terrier.structures.merging.StructureMerger [binary bits] [inverted file 1] [inverted file 2] [output inverted file]
`protected void`	`mergeDirectFiles()` Merges the two direct files and the corresponding document id files.
`protected void`	`mergeDocumentIndexFiles()` Merges the two document index files, and the meta files.
`protected void`	`mergeInvertedFiles()` Merges the two lexicons into one.
`void`	`mergeStructures()` Merges the structures created by terrier.
`void`	`setOutputIndex(IndexOnDisk _outputIndex)` Sets the output index.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - logger
```
protected static final org.apache.log4j.Logger logger
```
    the logger used
  - termcodeHashmap
```
protected gnu.trove.TIntIntHashMap termcodeHashmap
```
    A hashmap for converting the codes of terms appearing only in the vocabulary of the second set of data structures into a new set of term codes for the merged set of data structures.
  - keepTermCodeMap
```
protected boolean keepTermCodeMap
```
  - numberOfDocuments
```
protected int numberOfDocuments
```
    The number of documents in the merged structures.
  - numberOfPointers
```
protected long numberOfPointers
```
    The number of pointers in the merged structures.
  - numberOfTerms
```
protected int numberOfTerms
```
    The number of terms in the collection.
  - compressionDirectConfig
```
protected CompressionFactory.CompressionConfiguration compressionDirectConfig
```
  - compressionInvertedConfig
```
protected CompressionFactory.CompressionConfiguration compressionInvertedConfig
```
  - MetaReverse
```
protected boolean MetaReverse
```
  - srcIndex1
```
protected IndexOnDisk srcIndex1
```
    source index 1
  - srcIndex2
```
protected IndexOnDisk srcIndex2
```
    source index 2
  - destIndex
```
protected IndexOnDisk destIndex
```
    destination index
  - directFileOutputStreamClass
```
protected Class<? extends DirectInvertedOutputStream> directFileOutputStreamClass
```
    class to use to write direct file
  - fieldDirectFileOutputStreamClass
```
protected Class<? extends DirectInvertedOutputStream> fieldDirectFileOutputStreamClass
```
  - fieldCount
```
protected final int fieldCount
```
  - basicInvertedIndexPostingIteratorClass
```
protected String basicInvertedIndexPostingIteratorClass
```
  - fieldInvertedIndexPostingIteratorClass
```
protected String fieldInvertedIndexPostingIteratorClass
```
  - basicDirectIndexPostingIteratorClass
```
protected String basicDirectIndexPostingIteratorClass
```
  - fieldDirectIndexPostingIteratorClass
```
protected String fieldDirectIndexPostingIteratorClass
```
- Constructor Detail
  - StructureMerger
```
public StructureMerger(IndexOnDisk _srcIndex1,
               IndexOnDisk _srcIndex2,
               IndexOnDisk _destIndex)
```
    constructor
    
    Parameters:
    _srcIndex1 -
    _srcIndex2 -
    _destIndex -
- Method Detail
  - setOutputIndex
```
public void setOutputIndex(IndexOnDisk _outputIndex)
```
    Sets the output index. This index should have no documents
    
    Parameters:
    _outputIndex - the index to be merged to
  - mergeInvertedFiles
```
protected void mergeInvertedFiles()
```
    Merges the two lexicons into one. After this stage, the offsets in the lexicon are ot correct. They will be updated only after creating the inverted file.
  - mergeDirectFiles
```
protected void mergeDirectFiles()
```
    Merges the two direct files and the corresponding document id files.
  - getInterfaces
```
protected static Class<?>[] getInterfaces(Object o)
```
  - mergeDocumentIndexFiles
```
protected void mergeDocumentIndexFiles()
```
    Merges the two document index files, and the meta files.
  - createLexidFile
```
protected void createLexidFile()
```
    creates the final term code to offset file, and the lexicon hash if enabled.
  - mergeStructures
```
public void mergeStructures()
```
    Merges the structures created by terrier.
  - main
```
public static void main(String[] args)
                 throws Exception
```
    Usage: java org.terrier.structures.merging.StructureMerger [binary bits] [inverted file 1] [inverted file 2] [output inverted file]
    Binary bits concerns the number of fields in use in the index.
    
    Throws:
    
    Exception

Class StructureMerger

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

logger

termcodeHashmap

keepTermCodeMap

numberOfDocuments

numberOfPointers

numberOfTerms

compressionDirectConfig

compressionInvertedConfig

MetaReverse

srcIndex1

srcIndex2

destIndex

directFileOutputStreamClass

fieldDirectFileOutputStreamClass

fieldCount

basicInvertedIndexPostingIteratorClass

fieldInvertedIndexPostingIteratorClass

basicDirectIndexPostingIteratorClass

fieldDirectIndexPostingIteratorClass

Constructor Detail

StructureMerger

Method Detail

setOutputIndex

mergeInvertedFiles

mergeDirectFiles

getInterfaces

mergeDocumentIndexFiles

createLexidFile

mergeStructures

main