org.terrier.structures.indexing.singlepass.hadoop

Class SplitEmittedTerm

java.lang.Object

org.terrier.structures.indexing.singlepass.hadoop.SplitEmittedTerm

All Implemented Interfaces:

Comparable<SplitEmittedTerm>, org.apache.hadoop.io.Writable, org.apache.hadoop.io.WritableComparable<SplitEmittedTerm>

public class SplitEmittedTerm
extends Object
implements org.apache.hadoop.io.WritableComparable<SplitEmittedTerm>

Represents a Term key used during MapReduce Indexing. Term keys are emitted from each map task, and are used for sorting and partitioning the output. Paritioning is done by splitno. Two options for sorting (a) term only, (b) term, split, flush

Since:

3.0

Author:

Richard McCreadie

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SplitEmittedTerm.SETPartitioner Partitions SplitEmittedTerms by split that they came from.
static class	SplitEmittedTerm.SETPartitionerLowercaseAlphaTerm Partitions SplitEmittedTerms by term.
static class	SplitEmittedTerm.SETRawComparatorTerm Sorter by term only
static class	SplitEmittedTerm.SETRawComparatorTermSplitFlush A comparator for comparing different split emitted terms.

Constructor Summary

Constructors

Constructor and Description
SplitEmittedTerm() Empty Constructor
SplitEmittedTerm(String _term, int _splitno, int _flushno) Constructor for a Term key.

Method Summary

Methods

Modifier and Type	Method and Description
int	compareTo(SplitEmittedTerm term2) Compares this Term key to another term key.
static SplitEmittedTerm	createNewTerm(String term, int splitno, int flushno) Factory method for creating a new Term key object
boolean	equals(Object _o)
int	getFlushno()
int	getSplitno()
String	getTerm()
int	hashCode()
void	readFields(DataInput in) Read in a Term key object from the input stream 'in'
void	setFlushno(int _flushno)
void	setSplitno(int _splitno)
void	setTerm(String _term)
String	toString()
void	write(DataOutput out) Write out this Term key to output stream 'out'

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

SplitEmittedTerm

public SplitEmittedTerm()

Empty Constructor

SplitEmittedTerm

public SplitEmittedTerm(String _term,
                int _splitno,
                int _flushno)

Constructor for a Term key. Is used for sorting map output and partitioning posting lists between reducers. Each term is only unique in conjunction with the split and flush that it was emitted from.

Parameters:

_term -

_splitno -

_flushno -

Method Detail

createNewTerm

public static SplitEmittedTerm createNewTerm(String term,
                             int splitno,
                             int flushno)

Factory method for creating a new Term key object

Parameters:

term -

splitno -

flushno -

Returns:

a new split emitted term.

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object _o)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object

readFields

public void readFields(DataInput in)
                throws IOException

Read in a Term key object from the input stream 'in'

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Write out this Term key to output stream 'out'

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

compareTo

public int compareTo(SplitEmittedTerm term2)

Compares this Term key to another term key. Note that terms are unique only in conjunction with their associated split and flush.

Specified by:

compareTo in interface Comparable<SplitEmittedTerm>

getTerm

public String getTerm()

Returns:

the term

setTerm

public void setTerm(String _term)

Parameters:

_term - the term to set

getSplitno

public int getSplitno()

Returns:

the splitno

setSplitno

public void setSplitno(int _splitno)

Parameters:

_splitno - the splitno to set

getFlushno

public int getFlushno()

Returns:

the flushno

setFlushno

public void setFlushno(int _flushno)

Parameters:

_flushno - the flushno to set