uk.ac.gla.terrier.indexing
Class TRECCollection

java.lang.Object
  uk.ac.gla.terrier.indexing.TRECCollection

All Implemented Interfaces:

Collection, DocumentExtractor

Direct Known Subclasses:

TRECUTFCollection

public class TRECCollection
extends java.lang.Object
implements Collection, DocumentExtractor

Models a TREC test collection by implementing the interfaces Collection and DocumentExtractor. It provides sequential access to the documents in the collection and also it can return the text of a document as a String. TREC format files are opened using the default encoding unless the trec.encoding has been set to a valid supported encoding.

Properties:

• trec.encoding - encoding to use to open all files. Leave unset for System default encoding.

Version:

$Revision: 1.42 $

Author:

Craig Macdonald & Vassilis Plachouras

Constructor Summary

TRECCollection()
A default constructor that reads the collection specification file, as configured by the property collection.spec, reads a list of blacklisted document numbers, specified by the property trec.blacklist.docids and opens the first collection file to process.

TRECCollection(java.io.InputStream input)
A constructor that reads only the document in the specificed InputStream.

TRECCollection(java.lang.String CollectionSpecFilename, java.lang.String TagSet, java.lang.String BlacklistSpecFilename, java.lang.String docPointersFilename)
Specific constructor: reads the files listed in CollectionSpecFilename, the Blacklist of Document IDs in BlacklistSpecFilename, and stores document offsets and lengths in the document pointers file docPointersFilename.

Method Summary

void	close() Closes the files and streams used by the collection object.
boolean	endOfCollection() Indicates whether the end of the collection has been reached.
java.lang.String	getDocid() Returns the document number of the current document.
Document	getDocument() Returns the current document to process.
Document	getDocument(TagSet _tags, TagSet _exact, TagSet _fields) A TREC-specific getDocument method, that allows the tags to be specified for each document.
java.lang.String	getDocumentString(int docid) Returns the text of a document with the given identifier.
boolean	nextDocument() Moves to the next document to process from the collection.
void	reset() Resets the collection object back to the beginning of the collection.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TRECCollection

public TRECCollection(java.lang.String CollectionSpecFilename,
                      java.lang.String TagSet,
                      java.lang.String BlacklistSpecFilename,
                      java.lang.String docPointersFilename)

Specific constructor: reads the files listed in CollectionSpecFilename, the Blacklist of Document IDs in BlacklistSpecFilename, and stores document offsets and lengths in the document pointers file docPointersFilename. The collection will be parsed according to the TagSet specified by TagSet string

Parameters:

CollectionSpecFilename - The collections specification filename. The file contains a list of filenames to read. Must be specified, fatal error otherwise.

TagSet - the TagSet constructor string to use to obtain the tags to parse for.

BlacklistSpecFilename - A filename to a file containing a list of document identifiers thay have NOT to be processed. Not loaded if null or length 0

docPointersFilename - Where to store document offsets and lengths to. Not used if null.

TRECCollection

public TRECCollection()

A default constructor that reads the collection specification file, as configured by the property collection.spec, reads a list of blacklisted document numbers, specified by the property trec.blacklist.docids and opens the first collection file to process. The default constructor also uses the property trec.collection.pointers, in order to setup the name of the file in which pointers to the text of the documents are saved. TagSet TagSet.TREC_DOC_TAGS is used to tokenize the collection.

TRECCollection

public TRECCollection(java.io.InputStream input)

A constructor that reads only the document in the specificed InputStream. Also reads a list of blacklisted document numbers, specified by the property trec.blacklist.docids and opens the first collection file to process.

Method Detail

nextDocument

public boolean nextDocument()

Moves to the next document to process from the collection.

Specified by:

nextDocument in interface Collection

Returns:

boolean true if there are more documents to process in the collection, otherwise it returns false.

getDocument

public Document getDocument()

Returns the current document to process.

Specified by:

getDocument in interface Collection

Returns:

Document the object of the current document to process.

getDocument

public Document getDocument(TagSet _tags,
                            TagSet _exact,
                            TagSet _fields)

A TREC-specific getDocument method, that allows the tags to be specified for each document.

Returns:

Document the object of the current document to process.

getDocid

public java.lang.String getDocid()

Returns the document number of the current document.

Specified by:

getDocid in interface Collection

Returns:

String the document number of the current document.

endOfCollection

public boolean endOfCollection()

Indicates whether the end of the collection has been reached.

Specified by:

endOfCollection in interface Collection

Returns:

boolean true if there are no more documents to process in the collection, otherwise it returns false.

reset

public void reset()

Resets the collection object back to the beginning of the collection.

Specified by:

reset in interface Collection

getDocumentString

public java.lang.String getDocumentString(int docid)

Returns the text of a document with the given identifier.

Specified by:

getDocumentString in interface DocumentExtractor

Parameters:

docid - the internal identifier of a document.

Returns:

String the text of the document as a string.

close

public void close()

Closes the files and streams used by the collection object.

Specified by:

close in interface Collection