org.terrier.structures

Class CompressingMetaIndex.CompressingMetaIndexInputFormat

java.lang.Object

org.terrier.structures.CompressingMetaIndex.CompressingMetaIndexInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<String[]>>

Enclosing class:

CompressingMetaIndex

public static class CompressingMetaIndex.CompressingMetaIndexInputFormat
extends Object
implements org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<String[]>>

A Hadoop input format for a compressing meta index (allows the reading of a meta index as input to a MapReduce job.

Constructor Summary

Constructors

Constructor and Description
CompressingMetaIndex.CompressingMetaIndexInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.IntWritable,Wrapper<String[]>>	getRecordReader(org.apache.hadoop.mapred.InputSplit _split, org.apache.hadoop.mapred.JobConf jc, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf jc, int advisedNumberOfSplits)
void	overrideDataFileBlockSize(long blocksize) Permit the blocksize to be overridden, useful for testing different code paths
static void	setStructure(org.apache.hadoop.mapred.JobConf jc, String metaStructureName) Set structure
void	validateInput(org.apache.hadoop.mapred.JobConf jc) Validates the structure based on the job configuration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompressingMetaIndex.CompressingMetaIndexInputFormat

public CompressingMetaIndex.CompressingMetaIndexInputFormat()

Method Detail

setStructure

public static void setStructure(org.apache.hadoop.mapred.JobConf jc,
                String metaStructureName)

Set structure

Parameters:

jc -

metaStructureName -

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.IntWritable,Wrapper<String[]>> getRecordReader(org.apache.hadoop.mapred.InputSplit _split,
                                                                                                        org.apache.hadoop.mapred.JobConf jc,
                                                                                                        org.apache.hadoop.mapred.Reporter reporter)
                                                                                                          throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<String[]>>

Throws:

IOException

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf jc,
                                              int advisedNumberOfSplits)
                                                throws IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.IntWritable,Wrapper<String[]>>

Throws:

IOException

overrideDataFileBlockSize

public void overrideDataFileBlockSize(long blocksize)

Permit the blocksize to be overridden, useful for testing different code paths

validateInput

public void validateInput(org.apache.hadoop.mapred.JobConf jc)
                   throws IOException

Validates the structure based on the job configuration

Throws:

IOException