org.terrier.structures

Class Index

java.lang.Object

org.terrier.structures.Index

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

Direct Known Subclasses:

IndexOnDisk, MemoryIndex, MultiIndex

public abstract class Index
extends Object
implements Closeable, Flushable

This class encapsulates all the Indexes at retrieval time. It is loaded by giving a path and prefix. This looks for an index properties file at path/prefix.properties. Essentially, the properties file then specifies which index structures the index supports. The index then loads these so they can be used in retrieval.

Arbitrary properties can be defined in the index properties files, and in particular, properties are used to record index statistics and the contructor type and values of various index objects.

The Index will apply methods on specially marked interfaces. Currently, the only interface supported is IndexConfigurable. Moreover, structures implementing java.io.Closeable will have their close method called when the Index is closed.

Author:

Craig Macdonald & Vassilis Plachouras

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	Index.UpdatingCollectionStatistics This collection statistics parses the associated index properties for each call.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	dirtyProperties Have the properties of this index changed, suggesting a flush() is necessary when closing
protected static Class<?>[]	EMPTY_CLASS_ARRAY empty class array
protected static String	lastLoadError
protected static org.slf4j.Logger	logger The logger used
protected static int	MINIMUM_INDEX_TERRIER_VERSION
protected Properties	properties properties of this index
protected static String	PROPERTIES_SUFFIX
protected static boolean	RETRIEVAL_LOADING_PROFILE

Constructor Summary

Constructors

Modifier	Constructor and Description
	Index() Empty Index constructor - this should never be used! Use Index.createIndex() instead
protected	Index(long a, long b, long c) A constructor for child classes that doesnt open the file

Method Summary

Modifier and Type	Method and Description
void	addIndexStructure(String structureName, Class<?> className, Class<?>[] paramTypes, String[] paramValues) tell the index about a new index structure it provides.
void	addIndexStructure(String structureName, String className) add an index structure to this index.
void	addIndexStructure(String structureName, String className, String[] paramTypes, String[] paramValues) add an index structure to this index.
void	addIndexStructure(String structureName, String className, String paramTypes, String paramValues) add an index structure to this index.
void	addIndexStructureInputStream(String structureName, Class<?> className, Class<?>[] paramTypes, String[] paramValues) tell the index about a new input stream index structure it provides.
void	addIndexStructureInputStream(String structureName, String className) tell the index about a new input stream index structure it provides.
void	addIndexStructureInputStream(String structureName, String className, String[] paramTypes, String[] paramValues) tell the index about a new input stream index structure it provides.
void	addIndexStructureInputStream(String structureName, String className, String paramTypes, String paramValues) tell the index about a new input stream index structure it provides.
static boolean	allExists(String... files) returns true if all named files exist
abstract void	close() Closes the data structures of the index.
static IndexOnDisk	createIndex() Constructs a new Index object.
static IndexOnDisk	createIndex(String path, String prefix) Factory method for load an index.
static IndexOnDisk	createNewIndex(String path, String prefix) Factory method create a new index.
static boolean	existsIndex(String path, String prefix) Returns true if it is likely that an index exists at the specified location
abstract void	flush() Write any dirty data structures down to disk
abstract CollectionStatistics	getCollectionStatistics() Get the collection statistics
abstract PostingIndex<?>	getDirectIndex() Return the DirectIndex associated with this index
abstract DocumentIndex	getDocumentIndex() Return the DocumentIndex associated with this index
int	getEnd() Returns the last docid in this index
static boolean	getIndexLoadingProfileAsRetrieval() Get RETRIEVAL_LOADING_PROFILE
String	getIndexProperty(String key, String defaultValue) get an arbitrary property in the index
IndexRef	getIndexRef()
abstract Object	getIndexStructure(String structureName) Obtains the named index structure, using an already loaded one if possible.
abstract Object	getIndexStructureInputStream(String structureName) Return the input stream associated with the specified structure of this index
int	getIntIndexProperty(String key, int defaultValue) get an arbitrary int property from the index
abstract PostingIndex<?>	getInvertedIndex() Returns the InvertedIndex to use for this index
static String	getLastIndexLoadError() Returns the last warning given by an index being loaded.
abstract Lexicon<String>	getLexicon() Return the Lexicon associated with this index
abstract MetaIndex	getMetaIndex() Get the Meta Index structure
Properties	getProperties() Get the index properties
int	getStart() Returns the first docid in this index
boolean	hasIndexStructure(String structureName) Does this index have an index structure with the specified name?
boolean	hasIndexStructureInputStream(String structureName) Does this index have an index structure input stream with the specified name?
protected static String	join(String[] input, String joinString) joins a series of strings together with a delimiter
static void	main(String[] args) main
static void	setIndexLoadingProfileAsRetrieval(boolean yes) Set RETRIEVAL_LOADING_PROFILE
void	setIndexProperty(String key, String value) set an arbitrary property in the index
abstract String	toString() Returns a String representation of this index

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MINIMUM_INDEX_TERRIER_VERSION

protected static final int MINIMUM_INDEX_TERRIER_VERSION

See Also:

Constant Field Values

PROPERTIES_SUFFIX

protected static final String PROPERTIES_SUFFIX

See Also:

Constant Field Values

RETRIEVAL_LOADING_PROFILE

protected static boolean RETRIEVAL_LOADING_PROFILE

logger

protected static final org.slf4j.Logger logger

The logger used

lastLoadError

protected static String lastLoadError

EMPTY_CLASS_ARRAY

protected static final Class<?>[] EMPTY_CLASS_ARRAY

empty class array

properties

protected final Properties properties

properties of this index

dirtyProperties

protected boolean dirtyProperties

Have the properties of this index changed, suggesting a flush() is necessary when closing

Constructor Detail

Index

public Index()

Empty Index constructor - this should never be used! Use Index.createIndex() instead

Index

protected Index(long a,
                long b,
                long c)

A constructor for child classes that doesnt open the file

Method Detail

allExists

public static boolean allExists(String... files)

returns true if all named files exist

createIndex

public static IndexOnDisk createIndex()

Constructs a new Index object. Don't call this method, call the createIndex(String, String) factory method to construct an Index object.

createIndex

public static IndexOnDisk createIndex(String path,
                                      String prefix)

Factory method for load an index. This method should be used in order to load an existing index in the applications.

Parameters:

path - String the path in which the data structures will be created.

prefix - String the prefix of the files to be created.

createNewIndex

public static IndexOnDisk createNewIndex(String path,
                                         String prefix)

Factory method create a new index. This method should be used in order to load a new index in the applications.

Parameters:

path - String the path in which the data structures will be created.

prefix - String the prefix of the files to be created.

existsIndex

public static boolean existsIndex(String path,
                                  String prefix)

Returns true if it is likely that an index exists at the specified location

Parameters:

path -

prefix -

Returns:

true if a .properties or a .log files exists

getIndexLoadingProfileAsRetrieval

public static boolean getIndexLoadingProfileAsRetrieval()

Get RETRIEVAL_LOADING_PROFILE

Returns:

retrieval loading profile

getLastIndexLoadError

public static String getLastIndexLoadError()

Returns the last warning given by an index being loaded.

join

protected static String join(String[] input,
                             String joinString)

joins a series of strings together with a delimiter

main

public static void main(String[] args)

main

Parameters:

args -

setIndexLoadingProfileAsRetrieval

public static void setIndexLoadingProfileAsRetrieval(boolean yes)

Set RETRIEVAL_LOADING_PROFILE

Parameters:

yes -

addIndexStructure

public void addIndexStructure(String structureName,
                              Class<?> className,
                              Class<?>[] paramTypes,
                              String[] paramValues)

tell the index about a new index structure it provides. Class and parameter types specified as Class objects instead of Strings.

addIndexStructure

public void addIndexStructure(String structureName,
                              String className)

add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation parameters are "String,String", which are "path,prefix".

Parameters:

structureName -

className -

addIndexStructure

public void addIndexStructure(String structureName,
                              String className,
                              String paramTypes,
                              String paramValues)

add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation type parameters or values are non-default.

addIndexStructure

public void addIndexStructure(String structureName,
                              String className,
                              String[] paramTypes,
                              String[] paramValues)

add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation type parameters or values are non-default.

addIndexStructureInputStream

public void addIndexStructureInputStream(String structureName,
                                         Class<?> className,
                                         Class<?>[] paramTypes,
                                         String[] paramValues)

tell the index about a new input stream index structure it provides.

addIndexStructureInputStream

public void addIndexStructureInputStream(String structureName,
                                         String className)

tell the index about a new input stream index structure it provides.

addIndexStructureInputStream

public void addIndexStructureInputStream(String structureName,
                                         String className,
                                         String paramTypes,
                                         String paramValues)

tell the index about a new input stream index structure it provides.

addIndexStructureInputStream

public void addIndexStructureInputStream(String structureName,
                                         String className,
                                         String[] paramTypes,
                                         String[] paramValues)

tell the index about a new input stream index structure it provides.

close

public abstract void close()
                    throws IOException

Closes the data structures of the index.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

flush

public abstract void flush()
                    throws IOException

Write any dirty data structures down to disk

Specified by:

flush in interface Flushable

Throws:

IOException

getCollectionStatistics

public abstract CollectionStatistics getCollectionStatistics()

Get the collection statistics

getDirectIndex

public abstract PostingIndex<?> getDirectIndex()

Return the DirectIndex associated with this index

getDocumentIndex

public abstract DocumentIndex getDocumentIndex()

Return the DocumentIndex associated with this index

getIndexProperty

public String getIndexProperty(String key,
                               String defaultValue)

get an arbitrary property in the index

Parameters:

key - Key of the property to get

defaultValue - value of the property to use if property is not set

Returns:

Value of the property

getIndexRef

public IndexRef getIndexRef()

getIndexStructure

public abstract Object getIndexStructure(String structureName)

Obtains the named index structure, using an already loaded one if possible.

Parameters:

structureName - name of the required structure

Returns:

desired object or null if not found

getIndexStructureInputStream

public abstract Object getIndexStructureInputStream(String structureName)

Return the input stream associated with the specified structure of this index

Parameters:

structureName - The name of the structure of which you want the inputstream. Eg "lexicon"

Returns:

Required structure, or null if not found

getIntIndexProperty

public int getIntIndexProperty(String key,
                               int defaultValue)

get an arbitrary int property from the index

getInvertedIndex

public abstract PostingIndex<?> getInvertedIndex()

Returns the InvertedIndex to use for this index

getLexicon

public abstract Lexicon<String> getLexicon()

Return the Lexicon associated with this index

getMetaIndex

public abstract MetaIndex getMetaIndex()

Get the Meta Index structure

getProperties

public Properties getProperties()

Get the index properties

hasIndexStructure

public boolean hasIndexStructure(String structureName)

Does this index have an index structure with the specified name?

Parameters:

structureName - name of the required structure

Returns:

true if the index has an appropriately named structure

hasIndexStructureInputStream

public boolean hasIndexStructureInputStream(String structureName)

Does this index have an index structure input stream with the specified name?

Parameters:

structureName - name of the required structure

Returns:

true if the index has an appropriately named structure

setIndexProperty

public void setIndexProperty(String key,
                             String value)

set an arbitrary property in the index

Parameters:

key - Key to of the property to set

value - Value of the property to set

toString

public abstract String toString()

Returns a String representation of this index

Overrides:

toString in class Object

getStart

public int getStart()

Returns the first docid in this index

getEnd

public int getEnd()

Returns the last docid in this index