org.terrier.structures

Class FSOMapFileLexiconOutputStreamGeneric<T1,T2 extends org.apache.hadoop.io.WritableComparable<?>>

java.lang.Object

org.terrier.structures.LexiconOutputStream<T1>

org.terrier.structures.FSOMapFileLexiconOutputStreamGeneric<T1,T2>

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

FSOMapFileLexiconOutputStream

public abstract class FSOMapFileLexiconOutputStreamGeneric<T1,T2 extends org.apache.hadoop.io.WritableComparable<?>>
extends LexiconOutputStream<T1>

Field Summary

Fields

Modifier and Type	Field and Description
protected IndexOnDisk	index
protected FixedSizeWriteableFactory<T2>	keyFactory
protected String	leValueClassname
protected FSOrderedMapFile.MapFileWriter	mapFileWriter
protected String	structureName
protected T2	tempKey

Fields inherited from class org.terrier.structures.LexiconOutputStream

lexiconStream, numPointersWritten, numTermsWritten, numTokensWritten

Constructor Summary

Constructors

Constructor and Description
FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index, String _structureName, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index, String _structureName, FixedSizeWriteableFactory<T2> _keyFactory, Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index, String _structureName, FixedSizeWriteableFactory<T2> _keyFactory, String valueFactoryClassName)
FSOMapFileLexiconOutputStreamGeneric(String filename, FixedSizeWriteableFactory<T2> _keyFactory)
FSOMapFileLexiconOutputStreamGeneric(String path, String prefix, String _structureName, FixedSizeWriteableFactory<T2> _keyFactory)

Method Summary

Modifier and Type	Method and Description
static void	addLexiconToIndex(IndexOnDisk index, String structureName, Class<? extends Lexicon<?>> lexClass, Class<? extends Iterator<?>> lexInputStreamClass, Class<? extends Iterator<LexiconEntry>> lexEntryInputStreamClass, String leValueClassname) Adds Lexicon to index
void	close() Closes the lexicon stream.
protected abstract void	setKey(T1 k)
int	writeNextEntry(T1 _key, LexiconEntry _value) Writes a lexicon entry.

Methods inherited from class org.terrier.structures.LexiconOutputStream

getNumberOfPointersWritten, getNumberOfTermsWritten, getNumberOfTokensWritten, incrementCounters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keyFactory

protected FixedSizeWriteableFactory<T2 extends org.apache.hadoop.io.WritableComparable<?>> keyFactory

tempKey

protected final T2 extends org.apache.hadoop.io.WritableComparable<?> tempKey

mapFileWriter

protected final FSOrderedMapFile.MapFileWriter mapFileWriter

index

protected IndexOnDisk index

leValueClassname

protected String leValueClassname

structureName

protected final String structureName

Constructor Detail

FSOMapFileLexiconOutputStreamGeneric

public FSOMapFileLexiconOutputStreamGeneric(String path,
                                            String prefix,
                                            String _structureName,
                                            FixedSizeWriteableFactory<T2> _keyFactory)
                                     throws IOException

Throws:

IOException

FSOMapFileLexiconOutputStreamGeneric

public FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index,
                                            String _structureName,
                                            Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                                     throws IOException

Throws:

IOException

FSOMapFileLexiconOutputStreamGeneric

public FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index,
                                            String _structureName,
                                            FixedSizeWriteableFactory<T2> _keyFactory,
                                            Class<? extends FixedSizeWriteableFactory<LexiconEntry>> valueFactoryClass)
                                     throws IOException

Throws:

IOException

FSOMapFileLexiconOutputStreamGeneric

public FSOMapFileLexiconOutputStreamGeneric(IndexOnDisk _index,
                                            String _structureName,
                                            FixedSizeWriteableFactory<T2> _keyFactory,
                                            String valueFactoryClassName)
                                     throws IOException

Throws:

IOException

FSOMapFileLexiconOutputStreamGeneric

public FSOMapFileLexiconOutputStreamGeneric(String filename,
                                            FixedSizeWriteableFactory<T2> _keyFactory)
                                     throws IOException

Throws:

IOException

Method Detail

writeNextEntry

public int writeNextEntry(T1 _key,
                          LexiconEntry _value)
                   throws IOException

Description copied from class: LexiconOutputStream

Writes a lexicon entry.

Specified by:

writeNextEntry in class LexiconOutputStream<T1>

Parameters:

_key - the key - usually the term

_value - the lexicon entry value

Returns:

the number of bytes written to the file.

Throws:

IOException - if an I/O error occurs

setKey

protected abstract void setKey(T1 k)

close

public void close()

Description copied from class: LexiconOutputStream

Closes the lexicon stream. IOException if an I/O error occurs while closing the stream.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class LexiconOutputStream<T1>

addLexiconToIndex

public static void addLexiconToIndex(IndexOnDisk index,
                                     String structureName,
                                     Class<? extends Lexicon<?>> lexClass,
                                     Class<? extends Iterator<?>> lexInputStreamClass,
                                     Class<? extends Iterator<LexiconEntry>> lexEntryInputStreamClass,
                                     String leValueClassname)

Adds Lexicon to index

Parameters:

index -

structureName -

leValueClassname -