org.terrier.indexing

Class SimpleMedlineXMLCollection

java.lang.Object

org.terrier.indexing.SimpleXMLCollection

org.terrier.indexing.SimpleMedlineXMLCollection

All Implemented Interfaces:

Closeable, AutoCloseable, Collection

public class SimpleMedlineXMLCollection
extends SimpleXMLCollection

Initial implementation of a class that generates a Collection with Documents from a series of XML files in the Medline format. It process a limited number of documents in an XML file to avoid OutOfMemory problem in case the XML file is too large.

Properties:

• lowercase - lower case all terms obtained. Highly recommended.
• indexing.simplexmlcollection.reformxml - will try to reform broken & entities.
• xml.doc.buffer.size - The maximum number of documents to process per interation.

Author:

Ben He

Field Summary

Fields

Modifier and Type	Field and Description
protected int	currentFileDocCounter The number of documents processed in the current XML file.
String	docEndTag The end tag of documents in the XML files.
String	docTag The tag of documents in the XML files.
String	EOL The end of line string.
String	fileEndTag The tag indicating the end of an XML file.
String	fileTag The tag indicating the start of an XML file.
protected int	NUMBER_OF_DOCS_IN_BUFFER The number of documents to process per iteration.

Fields inherited from class org.terrier.indexing.SimpleXMLCollection

bReformXML, dbFactory, dBuilder, DocIDBlacklist, DocIdIsAttribute, DocIdLocation, DocumentElements, Documents, DocumentTags, ELEMENT_ATTR_SEPARATOR, EOC, FilesToProcess, logger, PropertiesInAttibutes, PropertyElements, TermElements, TermsInAttributes, thisDoc, xmlDoc

Constructor Summary

Constructors

Constructor and Description
SimpleMedlineXMLCollection() The default constructor.
SimpleMedlineXMLCollection(List<String> files, String ignored1, String BlacklistSpecFilename, String ignored2) Constructor required by TRECIndexing
SimpleMedlineXMLCollection(String CollectionSpecFilename, String BlacklistSpecFilename) An alternative constructor.
SimpleMedlineXMLCollection(String CollectionSpecFilename, String ignored1, String BlacklistSpecFilename, String ignored2) Constructor required by TRECIndexing

Method Summary

Modifier and Type	Method and Description
protected boolean	openNextFile() Parse through up to a limited number of documents in the XML file.

Methods inherited from class org.terrier.indexing.SimpleXMLCollection

close, endOfCollection, findDocumentElement, getDocument, hasNext, initialiseParser, initialiseTags, loadBlacklist, main, next, nextDocument, remove, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

currentFileDocCounter

protected int currentFileDocCounter

The number of documents processed in the current XML file.

docTag

public final String docTag

The tag of documents in the XML files.

See Also:

Constant Field Values

docEndTag

public final String docEndTag

The end tag of documents in the XML files.

See Also:

Constant Field Values

fileTag

public final String fileTag

The tag indicating the start of an XML file.

See Also:

Constant Field Values

fileEndTag

public final String fileEndTag

The tag indicating the end of an XML file.

See Also:

Constant Field Values

EOL

public final String EOL

The end of line string.

NUMBER_OF_DOCS_IN_BUFFER

protected final int NUMBER_OF_DOCS_IN_BUFFER

The number of documents to process per iteration.

Constructor Detail

SimpleMedlineXMLCollection

public SimpleMedlineXMLCollection()

The default constructor.

SimpleMedlineXMLCollection

public SimpleMedlineXMLCollection(String CollectionSpecFilename,
                                  String BlacklistSpecFilename)

An alternative constructor.

Parameters:

CollectionSpecFilename - The name of the file containing the location of XML files in the collection.

BlacklistSpecFilename - The name of the file containing the location of the blacklisted XML files in the collection.

SimpleMedlineXMLCollection

public SimpleMedlineXMLCollection(String CollectionSpecFilename,
                                  String ignored1,
                                  String BlacklistSpecFilename,
                                  String ignored2)

Constructor required by TRECIndexing

SimpleMedlineXMLCollection

public SimpleMedlineXMLCollection(List<String> files,
                                  String ignored1,
                                  String BlacklistSpecFilename,
                                  String ignored2)

Constructor required by TRECIndexing

Method Detail

openNextFile

protected boolean openNextFile()

Parse through up to a limited number of documents in the XML file. The limit is specified by property xml.doc.buffer.size.

Overrides:

openNextFile in class SimpleXMLCollection