org.terrier.structures

Class FSOMapFileLexiconOutputStream

java.lang.Object

org.terrier.structures.LexiconOutputStream<String>

org.terrier.structures.FSOMapFileLexiconOutputStream

All Implemented Interfaces:

Closeable, AutoCloseable

public class FSOMapFileLexiconOutputStream
extends LexiconOutputStream<String>

A LexiconOutputStream for FSOMapFileLexicon. Writes to a FSOrderedMapFile.

Since:

3.0

Author:

Craig Macdonald

Field Summary

Fields

Modifier and Type	Field and Description
protected IndexOnDisk	index
protected FixedSizeWriteableFactory<org.apache.hadoop.io.Text>	keyFactory
protected String	leValueClassname
protected FSOrderedMapFile.MapFileWriter	mapFileWriter
protected String	structureName
protected org.apache.hadoop.io.Text	tempKey

Fields inherited from class org.terrier.structures.LexiconOutputStream

lexiconStream, numPointersWritten, numTermsWritten, numTokensWritten

Constructor Summary

Constructors

Constructor and Description
FSOMapFileLexiconOutputStream(IndexOnDisk _index, String _structureName, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass) Construct an instance of the class with
FSOMapFileLexiconOutputStream(IndexOnDisk _index, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass) Construct an instance of the class with
FSOMapFileLexiconOutputStream(IndexOnDisk _index, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory, String valueFactoryClassName) Construct an instance of the class with
FSOMapFileLexiconOutputStream(String filename, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory) Construct an instance of the class with
FSOMapFileLexiconOutputStream(String path, String prefix, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory) Construct an instance of the class with

Method Summary

Methods

Modifier and Type	Method and Description
static void	addLexiconToIndex(IndexOnDisk index, String structureName, String leValueClassname) Adds Lexicon to index
void	close() Closes the lexicon stream.
int	writeNextEntry(String _key, LexiconEntry _value) Writes a lexicon entry.

Methods inherited from class org.terrier.structures.LexiconOutputStream

getNumberOfPointersWritten, getNumberOfTermsWritten, getNumberOfTokensWritten, incrementCounters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keyFactory

protected FixedSizeWriteableFactory<org.apache.hadoop.io.Text> keyFactory

tempKey

protected org.apache.hadoop.io.Text tempKey

mapFileWriter

protected final FSOrderedMapFile.MapFileWriter mapFileWriter

index

protected IndexOnDisk index

leValueClassname

protected String leValueClassname

structureName

protected final String structureName

Constructor Detail

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(String filename,
                             FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
                              throws IOException

Construct an instance of the class with

Parameters:

filename -

_keyFactory -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(String path,
                             String prefix,
                             String _structureName,
                             FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
                              throws IOException

Construct an instance of the class with

Parameters:

path -

prefix -

_structureName -

_keyFactory -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(IndexOnDisk _index,
                             String _structureName,
                             Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

valueFactoryClass -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(IndexOnDisk _index,
                             String _structureName,
                             FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory,
                             Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

_keyFactory -

valueFactoryClass -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(IndexOnDisk _index,
                             String _structureName,
                             FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory,
                             String valueFactoryClassName)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

_keyFactory -

valueFactoryClassName -

Throws:

IOException

Method Detail

writeNextEntry

public int writeNextEntry(String _key,
                 LexiconEntry _value)
                   throws IOException

Description copied from class: LexiconOutputStream

Writes a lexicon entry.

Specified by:

writeNextEntry in class LexiconOutputStream<String>

Parameters:

_key - the key - usually the term

_value - the lexicon entry value

Returns:

the number of bytes written to the file.

Throws:

IOException - if an I/O error occurs

close

public void close()

Description copied from class: LexiconOutputStream

Closes the lexicon stream. IOException if an I/O error occurs while closing the stream.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class LexiconOutputStream<String>

addLexiconToIndex

public static void addLexiconToIndex(IndexOnDisk index,
                     String structureName,
                     String leValueClassname)

Adds Lexicon to index

Parameters:

index -

structureName -

leValueClassname -