org.terrier.structures.bit

Class DirectInvertedOutputStream

java.lang.Object

org.terrier.structures.AbstractPostingOutputStream

org.terrier.structures.bit.DirectInvertedOutputStream

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

BlockDirectInvertedOutputStream, DirectInvertedDocidOnlyOuptutStream, FieldDirectInvertedOutputStream

public class DirectInvertedOutputStream
extends AbstractPostingOutputStream
implements Closeable

Writes a block direct or block inverted index, when passed appropriate posting lists.

Since:

2.0

Author:

Craig Macdonald

Field Summary

Fields

Modifier and Type	Field and Description
protected int	lastDocid
protected static org.slf4j.Logger	logger The logger used
protected BitOut	output what to write to

Constructor Summary

Constructors

Constructor and Description
DirectInvertedOutputStream(BitOut out) Creates a new output stream, writing to the specified BitOut implementation.
DirectInvertedOutputStream(OutputStream os) Creates a new output stream, writing a BitOutputStream to the specified file.
DirectInvertedOutputStream(String filename) Creates a new output stream, writing a BitOutputStream to the specified file.

Method Summary

Modifier and Type	Method and Description
void	close() close this object.
byte	getBitOffset() Deprecated.
BitOut	getBitOut() Return the underlying BitOut implementation being used by the class
long	getByteOffset() Deprecated.
int	getLastDocidWritten()
BitFilePosition	getOffset() What is current offset?
Class<? extends IterablePosting>	getPostingIteratorClass() Returns the IterablePosting class to use for reading structure written by this class
protected BitIndexPointer	writeNoFieldPostings(int[][] postings, int offset, int length, int firstId) Writes the given postings to the bit file.
protected void	writePostingNotDocid(Posting p) Hook method for writing out the remainder of the posting
BitIndexPointer	writePostings(int[][] postings, int firstId) Write out the specified postings.
BitIndexPointer	writePostings(int[][] postings, int startOffset, int Length, int firstId) Write out a range of the specified postings.
BitIndexPointer	writePostings(IterablePosting postings) Write out the specified postings.
BitIndexPointer	writePostings(IterablePosting postings, int previousId) Write out the specified postings, but allowing the delta for the first document to be adjusted
BitIndexPointer	writePostings(Iterator<Posting> iterator) Write out the specified postings.
BitIndexPointer	writePostings(Iterator<Posting> iterator, int previousId) Write out the specified postings, but allowing the delta for the first document to be adjusted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

output

protected BitOut output

what to write to

lastDocid

protected int lastDocid

logger

protected static final org.slf4j.Logger logger

The logger used

Constructor Detail

DirectInvertedOutputStream

public DirectInvertedOutputStream(OutputStream os)
                           throws IOException

Creates a new output stream, writing a BitOutputStream to the specified file. The number of binary bits for fields must also be specified.

Parameters:

os - stream to write to

Throws:

IOException

DirectInvertedOutputStream

public DirectInvertedOutputStream(String filename)
                           throws IOException

Creates a new output stream, writing a BitOutputStream to the specified file. The number of binary bits for fields must also be specified.

Parameters:

filename - Location of the file to write to

Throws:

IOException

DirectInvertedOutputStream

public DirectInvertedOutputStream(BitOut out)

Creates a new output stream, writing to the specified BitOut implementation. The number of binary bits for fields must also be specified.

Parameters:

out - BitOut implementation to write the file to

Method Detail

getPostingIteratorClass

public Class<? extends IterablePosting> getPostingIteratorClass()

Returns the IterablePosting class to use for reading structure written by this class

Specified by:

getPostingIteratorClass in class AbstractPostingOutputStream

writePostings

public BitIndexPointer writePostings(int[][] postings,
                                     int firstId)
                              throws IOException

Write out the specified postings. The delta for the first id must be specified.

Parameters:

postings - The postings to write out

firstId - the (delta) value of the first docid to write out.

Throws:

IOException

writePostings

public BitIndexPointer writePostings(Iterator<Posting> iterator)
                              throws IOException

Write out the specified postings.

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

iterator - an Iterator of Posting objects

Throws:

IOException

writePostings

public BitIndexPointer writePostings(Iterator<Posting> iterator,
                                     int previousId)
                              throws IOException

Write out the specified postings, but allowing the delta for the first document to be adjusted

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

iterator - an Iterator of Posting objects

previousId - id of the previous posting in this stream

Throws:

IOException

writePostings

public BitIndexPointer writePostings(IterablePosting postings,
                                     int previousId)
                              throws IOException

Write out the specified postings, but allowing the delta for the first document to be adjusted

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

postings - IterablePosting postings accessed through an IterablePosting object

previousId - id of the previous posting in this stream

Throws:

IOException

writePostings

public BitIndexPointer writePostings(IterablePosting postings)
                              throws IOException

Write out the specified postings.

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

postings - IterablePosting postings accessed through an IterablePosting object

Throws:

IOException

writePostingNotDocid

protected void writePostingNotDocid(Posting p)
                             throws IOException

Hook method for writing out the remainder of the posting

Throws:

IOException

writePostings

public BitIndexPointer writePostings(int[][] postings,
                                     int startOffset,
                                     int Length,
                                     int firstId)
                              throws IOException

Write out a range of the specified postings. The delta for the first id must be specified.

Specified by:

writePostings in class AbstractPostingOutputStream

Parameters:

postings - The postings to write out

startOffset - The location of the first posting to write out.

Length - The number of postings to be written out.

firstId - the (delta) value of the first docid to write out.

Throws:

IOException

writeNoFieldPostings

protected BitIndexPointer writeNoFieldPostings(int[][] postings,
                                               int offset,
                                               int length,
                                               int firstId)
                                        throws IOException

Writes the given postings to the bit file. This method assumes that field information is not provided.

Parameters:

postings - the postings list to write.

firstId - the first identifier to write. This can be an id plus one, or the gap of the current id and the previous one.

offset - The location of the first posting to write out.

length - The number of postings to be written out.

Throws:

IOException - if an error occurs during writing to a file.

close

public void close()

close this object. suppresses any exception

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class AbstractPostingOutputStream

getOffset

public BitFilePosition getOffset()

What is current offset?

Specified by:

getOffset in class AbstractPostingOutputStream

getByteOffset

public long getByteOffset()

Deprecated.

Return the current offset in bytes in the written file

getBitOffset

public byte getBitOffset()

Deprecated.

Return the current offset in bits in the written file

getBitOut

public BitOut getBitOut()

Return the underlying BitOut implementation being used by the class

getLastDocidWritten

public int getLastDocidWritten()

Specified by:

getLastDocidWritten in class AbstractPostingOutputStream