org.terrier.structures.indexing.singlepass.hadoop

Class SplitEmittedTerm.SETPartitioner

java.lang.Object

org.terrier.structures.indexing.singlepass.hadoop.SplitEmittedTerm.SETPartitioner

All Implemented Interfaces:

org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>

Enclosing class:

SplitEmittedTerm

public static class SplitEmittedTerm.SETPartitioner
extends Object
implements org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>, org.apache.hadoop.mapred.JobConfigurable

Partitions SplitEmittedTerms by split that they came from.

Constructor Summary

Constructors

Constructor and Description
SplitEmittedTerm.SETPartitioner()

Method Summary

Methods

Modifier and Type	Method and Description
int	calculatePartition(int splitno, int numPartitions) Calculates the partitions for a given split number.
void	configure(org.apache.hadoop.mapred.JobConf conf) Configure the partitioner functionality, i.e.
int	getPartition(SplitEmittedTerm term, MapEmittedPostingList posting, int numPartitions) Retuns the partition for the specified term and posting list, given the specified number of partitions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitEmittedTerm.SETPartitioner

public SplitEmittedTerm.SETPartitioner()

Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf conf)

Configure the partitioner functionality, i.e. calculate the number of splits there were.

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable

getPartition

public int getPartition(SplitEmittedTerm term,
               MapEmittedPostingList posting,
               int numPartitions)

Retuns the partition for the specified term and posting list, given the specified number of partitions.

Specified by:

getPartition in interface org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>

calculatePartition

public int calculatePartition(int splitno,
                     int numPartitions)

Calculates the partitions for a given split number.

Parameters:

splitno - - which split index, starting at 0

numPartitions - - number of partitions (reducers) configured

Returns:

the reduce partition number to allocate the split to.