uk.ac.gla.terrier.structures.merging
Class StructureMerger

java.lang.Object
  uk.ac.gla.terrier.structures.merging.StructureMerger

Direct Known Subclasses:

BlockStructureMerger

public class StructureMerger
extends java.lang.Object

This class merges the structures created by Terrier, so that we use fewer and larger inverted and direct files.

Properties:

• string.use_utf - use UTF support in index. Set to false by default.
• lexicon.use.hash - build a lexicon hash file for new index. Set to true by default.
• merge.direct - merge the direct indices if both indices have them. Set to true by default.

Version:

$Revision: 1.28 $

Author:

Vassilis Plachouras and Craig Macdonald

Constructor Summary

StructureMerger(Index _srcIndex1, Index _srcIndex2, Index _destIndex)


StructureMerger(java.lang.String _srcfilename1, java.lang.String _srcfilename2)
Deprecated.

Method Summary

static void	main(java.lang.String[] args) Usage: java uk.ac.gla.terrier.structures.merging.StructureMerger [binary bits] [inverted file 1] [inverted file 2] [output inverted file]
void	mergeStructures() Merges the structures created by terrier.
void	setNumberOfBits(int bits) Sets the number of bits to write or read for binary encoded numbers
void	setOutputFilename(java.lang.String _outputName) Deprecated.
void	setOutputIndex(Index _outputIndex) Sets the output index.
static void	writeFieldPostings(int[][] postings, int firstId, BitOut output, int binaryBits) Deprecated. use DirectInvertedIndexOutputStream
static void	writeNoFieldPostings(int[][] postings, int firstId, BitOut output) Deprecated. use DirectInvertedIndexOutputStream
static void	writePostings(int[][] postings, int firstId, BitOut output, int binaryBits) Deprecated. Please use DirectInvertedOutputStream instead

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StructureMerger

public StructureMerger(Index _srcIndex1,
                       Index _srcIndex2,
                       Index _destIndex)

StructureMerger

public StructureMerger(java.lang.String _srcfilename1,
                       java.lang.String _srcfilename2)

Deprecated.

A constructor that sets the filenames of the inverted files to merge

Parameters:

_srcfilename1 - the first inverted file to merge

_srcfilename2 - the second inverted file to merge

Method Detail

setNumberOfBits

public void setNumberOfBits(int bits)

Sets the number of bits to write or read for binary encoded numbers

Parameters:

bits - the number of bits to write or read

setOutputFilename

public void setOutputFilename(java.lang.String _outputName)

Deprecated.

Sets the output filename of the merged inverted file

Parameters:

_outputName - the filename of the merged inverted file

setOutputIndex

public void setOutputIndex(Index _outputIndex)

Sets the output index. This index should have no documents

Parameters:

_outputIndex - the index to be merged to

mergeStructures

public void mergeStructures()

Merges the structures created by terrier.

main

public static void main(java.lang.String[] args)

Usage: java uk.ac.gla.terrier.structures.merging.StructureMerger [binary bits] [inverted file 1] [inverted file 2] [output inverted file]

Binary bits concerns the number of fields in use in the index.

writePostings

public static void writePostings(int[][] postings,
                                 int firstId,
                                 BitOut output,
                                 int binaryBits)
                          throws java.io.IOException

Deprecated. Please use DirectInvertedOutputStream instead

Writes the given postings to a bit file. Depending on the value of the field binaryBits, this method will call the appropriate method writeToInvertedFileFields, or writeToInvertedFileNoFields.

Parameters:

postings - the postings list to write.

firstId - the first identifier to write. This can be an id plus one, or the gap of the current id and the previous one.

output - the output bit file.

Throws:

java.io.IOException

writeFieldPostings

public static void writeFieldPostings(int[][] postings,
                                      int firstId,
                                      BitOut output,
                                      int binaryBits)
                               throws java.io.IOException

Deprecated. use DirectInvertedIndexOutputStream

Writes the given postings to a bit file. This method assumes that field information is available as well.

Parameters:

postings - the postings list to write.

firstId - the first identifier to write. This can be an id plus one, or the gap of the current id and the previous one.

output - the output bit file.

Throws:

java.io.IOException

writeNoFieldPostings

public static void writeNoFieldPostings(int[][] postings,
                                        int firstId,
                                        BitOut output)
                                 throws java.io.IOException

Deprecated. use DirectInvertedIndexOutputStream

Writes the given postings to a bit file. This method assumes that field information is not available.

Parameters:

postings - the postings list to write.

firstId - the first identifier to write. This can be an id plus one, or the gap of the current id and the previous one.

output - the output bit file.

Throws:

java.io.IOException - if an error occurs during writing to a file.