org.terrier.structures
Class CompressingMetaIndex.CompressingMetaIndexInputFormat

java.lang.Object
  org.terrier.structures.CompressingMetaIndex.CompressingMetaIndexInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>>

Enclosing class:

CompressingMetaIndex

public static class CompressingMetaIndex.CompressingMetaIndexInputFormat
extends java.lang.Object
implements org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>>

A Hadoop input format for a compressing meta index (allows the reading of a meta index as input to a MapReduce job.

Constructor Summary

CompressingMetaIndex.CompressingMetaIndexInputFormat()

Method Summary

org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>>	getRecordReader(org.apache.hadoop.mapred.InputSplit _split, org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf jc, int advisedNumberOfSplits)
void	overrideDataFileBlockSize(long blocksize) Permit the blocksize to be overridden, useful for testing different code paths
static void	setStructure(org.apache.hadoop.mapred.JobConf jc, java.lang.String metaStructureName) Set structure
void	validateInput(org.apache.hadoop.mapred.JobConf jc) Validates the structure based on the job configuration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompressingMetaIndex.CompressingMetaIndexInputFormat

public CompressingMetaIndex.CompressingMetaIndexInputFormat()

Method Detail

setStructure

public static void setStructure(org.apache.hadoop.mapred.JobConf jc,
                                java.lang.String metaStructureName)

Set structure

Parameters:

jc -

metaStructureName -

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>> getRecordReader(org.apache.hadoop.mapred.InputSplit _split,
                                                                                                                           org.apache.hadoop.mapred.JobConf jc,
                                                                                                                           org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                    throws java.io.IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>>

Throws:

java.io.IOException

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf jc,
                                                       int advisedNumberOfSplits)
                                                throws java.io.IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<java.lang.String[]>>

Throws:

java.io.IOException

overrideDataFileBlockSize

public void overrideDataFileBlockSize(long blocksize)

Permit the blocksize to be overridden, useful for testing different code paths

validateInput

public void validateInput(org.apache.hadoop.mapred.JobConf jc)
                   throws java.io.IOException

Validates the structure based on the job configuration

Throws:

java.io.IOException