org.terrier.structures

Class LexiconOutputStream<KEY>

java.lang.Object

org.terrier.structures.LexiconOutputStream<KEY>

Type Parameters:

KEY -

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

FSOMapFileLexiconOutputStream

public abstract class LexiconOutputStream<KEY>
extends Object
implements Closeable

This class implements an output stream for the lexicon structure.

Author:

Vassilis Plachouras & Craig Macdonald

Field Summary

Fields

Modifier and Type	Field and Description
protected DataOutput	lexiconStream A data input stream to read from the bufferInput.
protected long	numPointersWritten Pointer written - the sum of the Nts
protected int	numTermsWritten
protected long	numTokensWritten collection length - the sum of the TFs

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	LexiconOutputStream()

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Closes the lexicon stream.
long	getNumberOfPointersWritten() Returns the number of pointers there would be in an inverted index built using this lexicon (thus far).
int	getNumberOfTermsWritten() Returns the number of terms written so far by this LexiconOutputStream
long	getNumberOfTokensWritten() Returns the number of tokens there are in the entire collection represented by this lexicon (thus far).
protected void	incrementCounters(EntryStatistics t)
abstract int	writeNextEntry(KEY _key, LexiconEntry _value) Writes a lexicon entry.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

lexiconStream

protected DataOutput lexiconStream

A data input stream to read from the bufferInput.

numPointersWritten

protected long numPointersWritten

Pointer written - the sum of the Nts

numTokensWritten

protected long numTokensWritten

collection length - the sum of the TFs

numTermsWritten

protected int numTermsWritten

Constructor Detail

LexiconOutputStream

protected LexiconOutputStream()

Method Detail

close

public void close()

Closes the lexicon stream. IOException if an I/O error occurs while closing the stream.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

writeNextEntry

public abstract int writeNextEntry(KEY _key,
                 LexiconEntry _value)
                            throws IOException

Writes a lexicon entry.

Parameters:

_key - the key - usually the term

_value - the lexicon entry value

Returns:

the number of bytes written to the file.

Throws:

IOException - if an I/O error occurs

incrementCounters

protected void incrementCounters(EntryStatistics t)

getNumberOfPointersWritten

public long getNumberOfPointersWritten()

Returns the number of pointers there would be in an inverted index built using this lexicon (thus far). This is equal to the sum of the Nts written to this lexicon output stream.

getNumberOfTokensWritten

public long getNumberOfTokensWritten()

Returns the number of tokens there are in the entire collection represented by this lexicon (thus far). This is equal to the sum of the TFs written to this lexicon output stream.

getNumberOfTermsWritten

public int getNumberOfTermsWritten()

Returns the number of terms written so far by this LexiconOutputStream