Package org.terrier.realtime.memory

Class MemoryLexicon

java.lang.Object

org.terrier.structures.Lexicon<K1>

org.terrier.structures.MapLexicon<java.lang.String,org.apache.hadoop.io.Text>

org.terrier.realtime.memory.MemoryLexicon

All Implemented Interfaces:

java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, java.lang.Iterable<java.util.Map.Entry<java.lang.String,LexiconEntry>>

public class MemoryLexicon
extends MapLexicon<java.lang.String,org.apache.hadoop.io.Text>
implements java.io.Serializable

The lexicon structure for a MemoryIndex. Since this is a memory structure, the lexicon entries are of type MemoryPointers rather than BitIndexPointer.

Since:

4.0

Author:

Richard McCreadie, Stuart Mackie

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.terrier.structures.MapLexicon

MapLexicon.Id2EntryIndexLookup, MapLexicon.IdIsIndex

Nested classes/interfaces inherited from class org.terrier.structures.Lexicon

Lexicon.LexiconFileEntry<KEY2>

Field Summary

Fields inherited from class org.terrier.structures.MapLexicon

keyFactory, map, modificationLock

Constructor Summary

Constructors

Constructor	Description
MemoryLexicon()	Constructor.

Method Summary

Modifier and Type	Method	Description
gnu.trove.TIntObjectHashMap<java.util.ArrayList<java.util.Map.Entry<java.lang.String,LexiconEntry>>>	getTopTermBins(int binsize, int minDf)
java.util.Iterator<java.util.Map.Entry<java.lang.String,LexiconEntry>>	iterator()	Lexicon iterator.
protected void	setK2(java.lang.String key, org.apache.hadoop.io.Text instance)
int	term(java.lang.String term, EntryStatistics es)	Add new term, or update existing term.
int	term(java.lang.String term, EntryStatistics es, int termid)
protected java.lang.String	toK1(org.apache.hadoop.io.Text key)
int	trimLexicon(int cutoff)

Methods inherited from class org.terrier.structures.MapLexicon

close, getIthLexiconEntry, getLexiconEntry, getLexiconEntry, getLexiconEntryRange, numberOfEntries, setTermIdLookup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

MemoryLexicon

public MemoryLexicon()

Constructor.

Method Detail

term

public int term(java.lang.String term,
                EntryStatistics es)

Add new term, or update existing term.

Returns:

The termid of the term.

term

public int term(java.lang.String term,
                EntryStatistics es,
                int termid)

trimLexicon

public int trimLexicon(int cutoff)

getTopTermBins

public gnu.trove.TIntObjectHashMap<java.util.ArrayList<java.util.Map.Entry<java.lang.String,LexiconEntry>>> getTopTermBins(int binsize,
                                                                                                                                 int minDf)

iterator

public java.util.Iterator<java.util.Map.Entry<java.lang.String,LexiconEntry>> iterator()

Lexicon iterator.

Specified by:

iterator in interface java.lang.Iterable<java.util.Map.Entry<java.lang.String,LexiconEntry>>

toK1

protected java.lang.String toK1(org.apache.hadoop.io.Text key)

Specified by:

toK1 in class MapLexicon<java.lang.String,org.apache.hadoop.io.Text>

setK2

protected void setK2(java.lang.String key,
                     org.apache.hadoop.io.Text instance)

Specified by:

setK2 in class MapLexicon<java.lang.String,org.apache.hadoop.io.Text>