uk.ac.gla.terrier.indexing
Class Indexer

java.lang.Object
  uk.ac.gla.terrier.indexing.Indexer

Direct Known Subclasses:

BasicIndexer, BlockIndexer

public abstract class Indexer
extends java.lang.Object

Properties:

• indexing.max.tokens - The maximum number of tokens the indexer will attempt to index in a document. If 0, then all tokens will be indexed (default).
• ignore.empty.documents - Assign empty documents documnent Ids. Default true
• indexing.max.docs.per.builder - Maximum number of documents in an index before a new index is created, and merged later.
• indexing.builder.boundary.docnos - Docnos of documents that force the index being created to be completed, and a new index to be commenced. An alternative to indexing.max.docs.per.builder

Version:

$Revision: 1.31 $

Author:

Craig Macdonald

Constructor Summary

Indexer()

Indexer(java.lang.String path)
Creates an instance of the class.

Indexer(java.lang.String path, java.lang.String prefix)
Creates an instance of the class.

Method Summary

abstract void	createDirectIndex(Collection[] collections) An abstract method for creating the direct index, the document index and the lexicon for the given collections.
abstract void	createInvertedIndex() An abstract method for creating the inverted index, given that the the direct index, the document index and the lexicon have already been created.
void	index(Collection[] collections) Creates the data structures for a set of collections.
static void	main(java.lang.String[] args)
static void	merge(java.lang.String mpath, java.lang.String mprefix, int lowest, int highest) Merge a series of indices
static void	merge(java.lang.String mpath, java.lang.String mprefix, java.util.LinkedList llist, int counterMerged) Merge a series of indices

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Indexer

public Indexer()

Indexer

public Indexer(java.lang.String path)

Creates an instance of the class. The generated data structures will be saved in the given path. The default name of the data structures is read from the property terrier.index.prefix (ApplicationSetup.TERRIER_INDEX_PREFIX).

Parameters:

path - String the path where the generated data structures will be saved.

Indexer

public Indexer(java.lang.String path,
               java.lang.String prefix)

Creates an instance of the class. The generated data structures will be saved in the given path. The of the data is given by the prefix parameter.

Parameters:

path - String the path where the generated data structures will be saved.

prefix - String the filename that the data structures will have.

Method Detail

createDirectIndex

public abstract void createDirectIndex(Collection[] collections)

An abstract method for creating the direct index, the document index and the lexicon for the given collections.

Parameters:

collections - Collection[] An array of collections to index

createInvertedIndex

public abstract void createInvertedIndex()

An abstract method for creating the inverted index, given that the the direct index, the document index and the lexicon have already been created.

index

public void index(Collection[] collections)

Creates the data structures for a set of collections. It creates a set of data structures for every indexing.max.docs.per.builder, if the value of this property is greater than zero, and then it mertges the generated data structures.

Parameters:

collections - The document collection objects to index.

merge

public static void merge(java.lang.String mpath,
                         java.lang.String mprefix,
                         int lowest,
                         int highest)

Merge a series of indices

merge

public static void merge(java.lang.String mpath,
                         java.lang.String mprefix,
                         java.util.LinkedList llist,
                         int counterMerged)

Merge a series of indices

main

public static void main(java.lang.String[] args)