org.terrier.structures
Class FSOMapFileLexiconOutputStream

java.lang.Object
  org.terrier.structures.LexiconOutputStream<String>
      org.terrier.structures.FSOMapFileLexiconOutputStream

All Implemented Interfaces:

Closeable

public class FSOMapFileLexiconOutputStream
extends LexiconOutputStream<String>

A LexiconOutputStream for FSOMapFileLexicon. Writes to a FSOrderedMapFile.

Since:

3.0

Author:

craigm

Field Summary

protected Index	index
protected FixedSizeWriteableFactory<org.apache.hadoop.io.Text>	keyFactory
protected String	leValueClassname
protected FSOrderedMapFile.MapFileWriter	mapFileWriter
protected String	structureName
protected org.apache.hadoop.io.Text	tempKey

Fields inherited from class org.terrier.structures.LexiconOutputStream

lexiconStream, numPointersWritten, numTermsWritten, numTokensWritten

Constructor Summary

FSOMapFileLexiconOutputStream(Index _index, String _structureName, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
Construct an instance of the class with

FSOMapFileLexiconOutputStream(Index _index, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
Construct an instance of the class with

FSOMapFileLexiconOutputStream(Index _index, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory, String valueFactoryClassName)
Construct an instance of the class with

FSOMapFileLexiconOutputStream(String filename, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
Construct an instance of the class with

FSOMapFileLexiconOutputStream(String path, String prefix, String _structureName, FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
Construct an instance of the class with

Method Summary

static void	addLexiconToIndex(Index index, String structureName, String leValueClassname) Adds Lexicon to index
void	close() Closes the lexicon stream.
int	writeNextEntry(String _key, LexiconEntry _value) Writes a lexicon entry.

Methods inherited from class org.terrier.structures.LexiconOutputStream

getNumberOfPointersWritten, getNumberOfTermsWritten, getNumberOfTokensWritten, incrementCounters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keyFactory

protected FixedSizeWriteableFactory<org.apache.hadoop.io.Text> keyFactory

tempKey

protected org.apache.hadoop.io.Text tempKey

mapFileWriter

protected final FSOrderedMapFile.MapFileWriter mapFileWriter

index

protected Index index

leValueClassname

protected String leValueClassname

structureName

protected final String structureName

Constructor Detail

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(String filename,
                                     FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
                              throws IOException

Construct an instance of the class with

Parameters:

filename -

_keyFactory -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(String path,
                                     String prefix,
                                     String _structureName,
                                     FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory)
                              throws IOException

Construct an instance of the class with

Parameters:

path -

prefix -

_structureName -

_keyFactory -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(Index _index,
                                     String _structureName,
                                     Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

valueFactoryClass -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(Index _index,
                                     String _structureName,
                                     FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory,
                                     Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

_keyFactory -

valueFactoryClass -

Throws:

IOException

FSOMapFileLexiconOutputStream

public FSOMapFileLexiconOutputStream(Index _index,
                                     String _structureName,
                                     FixedSizeWriteableFactory<org.apache.hadoop.io.Text> _keyFactory,
                                     String valueFactoryClassName)
                              throws IOException

Construct an instance of the class with

Parameters:

_index -

_structureName -

_keyFactory -

valueFactoryClassName -

Throws:

IOException

Method Detail

writeNextEntry

public int writeNextEntry(String _key,
                          LexiconEntry _value)
                   throws IOException

Description copied from class: LexiconOutputStream

Writes a lexicon entry.

Specified by:

writeNextEntry in class LexiconOutputStream<String>

Parameters:

_key - the key - usually the term

_value - the lexicon entry value

Returns:

the number of bytes written to the file.

Throws:

IOException - if an I/O error occurs

close

public void close()

Description copied from class: LexiconOutputStream

Closes the lexicon stream. IOException if an I/O error occurs while closing the stream.

Specified by:

close in interface Closeable

Overrides:

close in class LexiconOutputStream<String>

addLexiconToIndex

public static void addLexiconToIndex(Index index,
                                     String structureName,
                                     String leValueClassname)

Adds Lexicon to index

Parameters:

index -

structureName -

leValueClassname -