org.terrier.structures.integer

Class IntegerCodingPostingOutputStream

java.lang.Object

org.terrier.structures.AbstractPostingOutputStream

org.terrier.structures.integer.IntegerCodingPostingOutputStream

public class IntegerCodingPostingOutputStream
extends AbstractPostingOutputStream

This class is used to store a posting list to the disk, compressing it using some integer coding technique. It can be used to save IterablePosting with (optionally) fields and/or blocks. Note: this class writes the posting list in chunks, whose size is specified by chunkSize.

Since:

4.0

Author:

Matteo Catena

Field Summary

Fields

Modifier and Type	Field and Description
protected int[]	ids
protected static org.apache.log4j.Logger	logger The logger used
protected ByteOut	output

Constructor Summary

Constructors

Constructor and Description
IntegerCodingPostingOutputStream(ByteOut output, int chunkSize, int fieldsCount, int hasBlocks, IntegerCodec idsCodec, IntegerCodec tfsCodec, IntegerCodec fieldsCodec, IntegerCodec blocksCodec)
IntegerCodingPostingOutputStream(String filename, int chunkSize, int fieldsCount, int hasBlocks, IntegerCodec idsCodec, IntegerCodec tfsCodec, IntegerCodec fieldsCodec, IntegerCodec blocksCodec)

Method Summary

Methods

Modifier and Type	Method and Description
void	close() close this object.
BitFilePosition	getOffset() What is current offset?
Class<? extends IterablePosting>	getPostingIteratorClass() Returns the IterablePosting class to use for reading structure written by this class
protected void	write(int i, int cnt) compress and write down the current chunk of postings
BitIndexPointer	writePostings(int[][] postings, int startOffset, int Length, int firstId)
BitIndexPointer	writePostings(IterablePosting postings) Write out the specified postings.
BitIndexPointer	writePostings(IterablePosting postings, int previousId)
BitIndexPointer	writePostings(Iterator<Posting> iterator)
BitIndexPointer	writePostings(Iterator<Posting> iterator, int previousId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

The logger used

output

protected ByteOut output

ids

protected int[] ids

Constructor Detail

IntegerCodingPostingOutputStream

public IntegerCodingPostingOutputStream(String filename,
                                int chunkSize,
                                int fieldsCount,
                                int hasBlocks,
                                IntegerCodec idsCodec,
                                IntegerCodec tfsCodec,
                                IntegerCodec fieldsCodec,
                                IntegerCodec blocksCodec)
                                 throws IOException

Parameters:

filename - the file where to write

chunkSize - the chunk size (in term of number of posting)

fieldsCount - the number of fields, if any. 0 otherwise

hasBlocks - 0:no blocks, 1:positions, 2:blocks of any size!=1

idsCodec - the IntegerCodec to use to compress ids

tfsCodec - the IntegerCodec to use to compress tfs

fieldsCodec - the IntegerCodec to use to compress fields (null if there are no fields)

blocksCodec - the IntegerCodec to use to compress blocks (null if there are no blocks)

Throws:

IOException

IntegerCodingPostingOutputStream

public IntegerCodingPostingOutputStream(ByteOut output,
                                int chunkSize,
                                int fieldsCount,
                                int hasBlocks,
                                IntegerCodec idsCodec,
                                IntegerCodec tfsCodec,
                                IntegerCodec fieldsCodec,
                                IntegerCodec blocksCodec)
                                 throws IOException

Parameters:

output - the output channel

chunkSize - the chunk size (in term of number of posting)

fieldsCount - the number of fields, if any. 0 otherwise

hasBlocks - 0:no blocks, 1:positions, 2:blocks of any size!=1

idsCodec - the IntegerCodec to use to compress ids

tfsCodec - the IntegerCodec to use to compress tfs

fieldsCodec - the IntegerCodec to use to compress fields (null if there are no fields)

blocksCodec - the IntegerCodec to use to compress blocks (null if there are no blocks)

Throws:

IOException

Method Detail

getPostingIteratorClass

public Class<? extends IterablePosting> getPostingIteratorClass()

Returns the IterablePosting class to use for reading structure written by this class

Specified by:

getPostingIteratorClass in class AbstractPostingOutputStream

writePostings

public BitIndexPointer writePostings(IterablePosting postings)
                              throws IOException

Write out the specified postings.

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

postings - IterablePosting postings accessed through an IterablePosting object

Throws:

IOException

write

protected void write(int i,
         int cnt)
              throws IOException

compress and write down the current chunk of postings

Parameters:

i - number of postings

cnt - number of blocks

Throws:

IOException

close

public void close()

close this object. suppresses any exception

Specified by:

close in class AbstractPostingOutputStream

getOffset

public BitFilePosition getOffset()

What is current offset?

Specified by:

getOffset in class AbstractPostingOutputStream

writePostings

public BitIndexPointer writePostings(int[][] postings,
                            int startOffset,
                            int Length,
                            int firstId)
                              throws IOException

Specified by:

writePostings in class AbstractPostingOutputStream

Throws:

IOException

writePostings

public BitIndexPointer writePostings(IterablePosting postings,
                            int previousId)
                              throws IOException

Specified by:

writePostings in class AbstractPostingOutputStream

Throws:

IOException

writePostings

public BitIndexPointer writePostings(Iterator<Posting> iterator,
                            int previousId)
                              throws IOException

Specified by:

writePostings in class AbstractPostingOutputStream

Throws:

IOException

writePostings

public BitIndexPointer writePostings(Iterator<Posting> iterator)
                              throws IOException

Specified by:

writePostings in class AbstractPostingOutputStream

Throws:

IOException