org.terrier.indexing

Class FileDocument

java.lang.Object

org.terrier.indexing.FileDocument

All Implemented Interfaces:

Document

Direct Known Subclasses:

PDFDocument, POIDocument

public class FileDocument
extends Object
implements Document

Models a document which corresponds to one file. The first FileDocument.abstract.length characters can be saved as an abstract.

Author:

Craig Macdonald, Vassilis Plachouras, Richard McCreadie, Rodrygo Santos

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	FileDocument.ReaderWrapper A wrapper around the token stream used to lift the terms from the stream for storage in the abstract

Field Summary

Fields

Modifier and Type	Field and Description
protected int	abstractlength The maximum length of each named abstract (comma separated list)
protected String	abstractname The names of the abstracts to be saved (comma separated list)
protected int	abstractwritten The number of characters currently written
protected Reader	br The input reader.
protected boolean	EOD End of Document.
protected String	filename The name of the file represented by this document.
protected Map<String,String>	fileProperties The number of bytes read from the input.
protected static org.slf4j.Logger	logger
protected TokenStream	tokenStream

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FileDocument()
	FileDocument(InputStream docStream, Map<String,String> docProperties, Tokeniser tok) Constructs an instance of the FileDocument from the given input stream.
	FileDocument(Reader docReader, Map<String,String> docProperties, Tokeniser tok) create a document for a file
	FileDocument(String _filename, InputStream docStream, Tokeniser tok) create a document for a file
	FileDocument(String _filename, Reader docReader, Tokeniser tok) create a document for a file

Method Summary

Methods

Modifier and Type	Method and Description
boolean	endOfDocument() Indicates whether the end of a document has been reached.
Map<String,String>	getAllProperties() Returns the underlying map of all the properties defined by this Document.
Set<String>	getFields() Returns null because there is no support for fields with file documents.
String	getNextTerm() Gets the next term from the Document
String	getProperty(String name) Get a document property
Reader	getReader() Returns the underlying buffered reader, so that client code can tokenise the document itself, and deal with it how it likes.
protected Reader	getReader(InputStream docStream) Returns a buffered reader that encapsulates the given input stream.
protected static Map<String,String>	makeFilenameProperties(String filename)
void	setProperty(String name, String value) Set a document property

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.slf4j.Logger logger

br

protected Reader br

The input reader.

EOD

protected boolean EOD

End of Document. Set by the last couple of lines in getNextTerm()

fileProperties

protected Map<String,String> fileProperties

The number of bytes read from the input.

filename

protected String filename

The name of the file represented by this document.

tokenStream

protected TokenStream tokenStream

abstractname

protected final String abstractname

The names of the abstracts to be saved (comma separated list)

abstractlength

protected final int abstractlength

The maximum length of each named abstract (comma separated list)

abstractwritten

protected int abstractwritten

The number of characters currently written

Constructor Detail

FileDocument

protected FileDocument()

FileDocument

public FileDocument(String _filename,
            Reader docReader,
            Tokeniser tok)

create a document for a file

Parameters:

_filename -

docReader -

tok -

FileDocument

public FileDocument(String _filename,
            InputStream docStream,
            Tokeniser tok)

create a document for a file

Parameters:

_filename -

docStream -

tok -

FileDocument

public FileDocument(Reader docReader,
            Map<String,String> docProperties,
            Tokeniser tok)

create a document for a file

Parameters:

docReader -

docProperties -

tok -

FileDocument

public FileDocument(InputStream docStream,
            Map<String,String> docProperties,
            Tokeniser tok)

Constructs an instance of the FileDocument from the given input stream.

Parameters:

docStream - the input stream that reads the file.

Method Detail

makeFilenameProperties

protected static Map<String,String> makeFilenameProperties(String filename)

getReader

public Reader getReader()

Returns the underlying buffered reader, so that client code can tokenise the document itself, and deal with it how it likes.

Specified by:

getReader in interface Document

getReader

protected Reader getReader(InputStream docStream)

Returns a buffered reader that encapsulates the given input stream.

Parameters:

docStream - an input stream that we want to access as a buffered reader.

Returns:

the buffered reader that encapsulates the given input stream.

getNextTerm

public String getNextTerm()

Gets the next term from the Document

Specified by:

getNextTerm in interface Document

Returns:

String the next term of the document. Null returns should be ignored.

getFields

public Set<String> getFields()

Returns null because there is no support for fields with file documents.

Specified by:

getFields in interface Document

Returns:

null.

endOfDocument

public boolean endOfDocument()

Indicates whether the end of a document has been reached.

Specified by:

endOfDocument in interface Document

Returns:

boolean true if the end of a document has been reached, otherwise, it returns false.

getProperty

public String getProperty(String name)

Get a document property

Specified by:

getProperty in interface Document

Parameters:

name - Name of the property. It is suggested, but not required that this name should not be case insensitive.

setProperty

public void setProperty(String name,
               String value)

Set a document property

getAllProperties

public Map<String,String> getAllProperties()

Returns the underlying map of all the properties defined by this Document.

Specified by:

getAllProperties in interface Document