org.terrier.structures.indexing.singlepass.hadoop
Class SplitEmittedTerm.SETPartitioner

java.lang.Object
  extended by org.terrier.structures.indexing.singlepass.hadoop.SplitEmittedTerm.SETPartitioner
All Implemented Interfaces:
org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>
Enclosing class:
SplitEmittedTerm

public static class SplitEmittedTerm.SETPartitioner
extends java.lang.Object
implements org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>, org.apache.hadoop.mapred.JobConfigurable

Partitions SplitEmittedTerms by split that they came from.


Constructor Summary
SplitEmittedTerm.SETPartitioner()
           
 
Method Summary
 int calculatePartition(int splitno, int numPartitions)
          Calculates the partitions for a given split number.
 void configure(org.apache.hadoop.mapred.JobConf conf)
          Configure the partitioner functionality, i.e.
 int getPartition(SplitEmittedTerm term, MapEmittedPostingList posting, int numPartitions)
          Retuns the partition for the specified term and posting list, given the specified number of partitions.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SplitEmittedTerm.SETPartitioner

public SplitEmittedTerm.SETPartitioner()
Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf conf)
Configure the partitioner functionality, i.e. calculate the number of splits there were.

Specified by:
configure in interface org.apache.hadoop.mapred.JobConfigurable

getPartition

public int getPartition(SplitEmittedTerm term,
                        MapEmittedPostingList posting,
                        int numPartitions)
Retuns the partition for the specified term and posting list, given the specified number of partitions.

Specified by:
getPartition in interface org.apache.hadoop.mapred.Partitioner<SplitEmittedTerm,MapEmittedPostingList>

calculatePartition

public int calculatePartition(int splitno,
                              int numPartitions)
Calculates the partitions for a given split number.

Parameters:
splitno - - which split index, starting at 0
numPartitions - - number of partitions (reducers) configured
Returns:
the reduce partition number to allocate the split to.


Terrier 3.5. Copyright © 2004-2011 University of Glasgow