Package org.terrier.structures

Class PropertiesIndex

  • All Implemented Interfaces:
    java.io.Closeable, java.io.Flushable, java.lang.AutoCloseable
    Direct Known Subclasses:
    IndexOnDisk
    public abstract class PropertiesIndex
extends Index
implements java.io.Closeable, java.io.Flushable
    This class encapsulates all the Indexes at retrieval time. It is loaded by giving a path and prefix. This looks for an index properties file at path/prefix.properties. Essentially, the properties file then specifies which index structures the index supports. The index then loads these so they can be used in retrieval.

    Arbitrary properties can be defined in the index properties files, and in particular, properties are used to record index statistics and the contructor type and values of various index objects.

    The Index will apply methods on specially marked interfaces. Currently, the only interface supported is IndexConfigurable. Moreover, structures implementing java.io.Closeable will have their close method called when the Index is closed.

    Author:
    Craig Macdonald & Vassilis Plachouras

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
        PropertiesIndex()
      Empty Index constructor - this should never be used! Use Index.createIndex() instead
      protected PropertiesIndex​(long a, long b, long c)
      A constructor for child classes that doesnt open the file

    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      void addIndexStructure​(java.lang.String structureName, java.lang.Class<?> className, java.lang.Class<?>[] paramTypes, java.lang.String[] paramValues)
      tell the index about a new index structure it provides.
      void addIndexStructure​(java.lang.String structureName, java.lang.String className)
      add an index structure to this index.
      void addIndexStructure​(java.lang.String structureName, java.lang.String className, java.lang.String[] paramTypes, java.lang.String[] paramValues)
      add an index structure to this index.
      void addIndexStructure​(java.lang.String structureName, java.lang.String className, java.lang.String paramTypes, java.lang.String paramValues)
      add an index structure to this index.
      void addIndexStructureInputStream​(java.lang.String structureName, java.lang.Class<?> className, java.lang.Class<?>[] paramTypes, java.lang.String[] paramValues)
      tell the index about a new input stream index structure it provides.
      void addIndexStructureInputStream​(java.lang.String structureName, java.lang.String className)
      tell the index about a new input stream index structure it provides.
      void addIndexStructureInputStream​(java.lang.String structureName, java.lang.String className, java.lang.String[] paramTypes, java.lang.String[] paramValues)
      tell the index about a new input stream index structure it provides.
      void addIndexStructureInputStream​(java.lang.String structureName, java.lang.String className, java.lang.String paramTypes, java.lang.String paramValues)
      tell the index about a new input stream index structure it provides.
      static boolean allExists​(java.lang.String... files)
      returns true if all named files exist
      abstract void close()
      Closes the data structures of the index.
      static IndexOnDisk createIndex()
      Constructs a new Index object.
      static IndexOnDisk createIndex​(java.lang.String path, java.lang.String prefix)
      Factory method for load an index.
      static IndexOnDisk createNewIndex​(java.lang.String path, java.lang.String prefix)
      Factory method create a new index.
      static boolean existsIndex​(java.lang.String path, java.lang.String prefix)
      Returns true if it is likely that an index exists at the specified location
      abstract void flush()
      Write any dirty data structures down to disk
      abstract CollectionStatistics getCollectionStatistics()
      Get the collection statistics
      abstract PostingIndex<?> getDirectIndex()
      Return the DirectIndex associated with this index
      abstract DocumentIndex getDocumentIndex()
      Return the DocumentIndex associated with this index
      int getEnd()
      Returns the last docid in this index
      static boolean getIndexLoadingProfileAsRetrieval()
      Get RETRIEVAL_LOADING_PROFILE
      java.lang.String getIndexProperty​(java.lang.String key, java.lang.String defaultValue)
      get an arbitrary property in the index
      IndexRef getIndexRef()
      Returns a direct IndexRef to this index
      abstract java.lang.Object getIndexStructure​(java.lang.String structureName)
      Obtains the named index structure, using an already loaded one if possible.
      abstract java.lang.Object getIndexStructureInputStream​(java.lang.String structureName)
      Return the input stream associated with the specified structure of this index
      int getIntIndexProperty​(java.lang.String key, int defaultValue)
      get an arbitrary int property from the index
      abstract PostingIndex<?> getInvertedIndex()
      Returns the InvertedIndex to use for this index
      static java.lang.String getLastIndexLoadError()
      Returns the last warning given by an index being loaded.
      abstract Lexicon<java.lang.String> getLexicon()
      Return the Lexicon associated with this index
      abstract MetaIndex getMetaIndex()
      Get the Meta Index structure
      java.util.Properties getProperties()
      Get the index properties
      int getStart()
      Returns the first docid in this index
      boolean hasIndexStructure​(java.lang.String structureName)
      Does this index have an index structure with the specified name?
      boolean hasIndexStructureInputStream​(java.lang.String structureName)
      Does this index have an index structure input stream with the specified name?
      protected static java.lang.String join​(java.lang.String[] input, java.lang.String joinString)
      joins a series of strings together with a delimiter
      static void setIndexLoadingProfileAsRetrieval​(boolean yes)
      Set RETRIEVAL_LOADING_PROFILE
      void setIndexProperty​(java.lang.String key, java.lang.String value)
      set an arbitrary property in the index
      abstract java.lang.String toString()
      Returns a String representation of this index

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Field Detail

      • MINIMUM_INDEX_TERRIER_VERSION

        protected static final int MINIMUM_INDEX_TERRIER_VERSION
        See Also:
        Constant Field Values

      • PROPERTIES_SUFFIX

        protected static final java.lang.String PROPERTIES_SUFFIX
        See Also:
        Constant Field Values

      • RETRIEVAL_LOADING_PROFILE

        protected static boolean RETRIEVAL_LOADING_PROFILE

      • logger

        protected static final org.slf4j.Logger logger
        The logger used

      • lastLoadError

        protected static java.lang.String lastLoadError

      • EMPTY_CLASS_ARRAY

        protected static final java.lang.Class<?>[] EMPTY_CLASS_ARRAY
        empty class array

      • properties

        protected final java.util.Properties properties
        properties of this index

      • dirtyProperties

        protected boolean dirtyProperties
        Have the properties of this index changed, suggesting a flush() is necessary when closing

    • Constructor Detail

      • PropertiesIndex

        public PropertiesIndex()
        Empty Index constructor - this should never be used! Use Index.createIndex() instead

      • PropertiesIndex

        protected PropertiesIndex​(long a,
                          long b,
                          long c)
        A constructor for child classes that doesnt open the file

    • Method Detail

      • allExists

        public static boolean allExists​(java.lang.String... files)
        returns true if all named files exist

      • createIndex

        public static IndexOnDisk createIndex()
        Constructs a new Index object. Don't call this method, call the createIndex(String, String) factory method to construct an Index object.

      • createIndex

        public static IndexOnDisk createIndex​(java.lang.String path,
                                      java.lang.String prefix)
        Factory method for load an index. This method should be used in order to load an existing index in the applications.
        Parameters:
        path - String the path in which the data structures will be created.
        prefix - String the prefix of the files to be created.

      • createNewIndex

        public static IndexOnDisk createNewIndex​(java.lang.String path,
                                         java.lang.String prefix)
        Factory method create a new index. This method should be used in order to load a new index in the applications.
        Parameters:
        path - String the path in which the data structures will be created.
        prefix - String the prefix of the files to be created.

      • existsIndex

        public static boolean existsIndex​(java.lang.String path,
                                  java.lang.String prefix)
        Returns true if it is likely that an index exists at the specified location
        Parameters:
        path -
        prefix -
        Returns:
        true if a .properties or a .log files exists

      • getIndexLoadingProfileAsRetrieval

        public static boolean getIndexLoadingProfileAsRetrieval()
        Get RETRIEVAL_LOADING_PROFILE
        Returns:
        retrieval loading profile

      • getLastIndexLoadError

        public static java.lang.String getLastIndexLoadError()
        Returns the last warning given by an index being loaded.

      • join

        protected static java.lang.String join​(java.lang.String[] input,
                                       java.lang.String joinString)
        joins a series of strings together with a delimiter

      • setIndexLoadingProfileAsRetrieval

        public static void setIndexLoadingProfileAsRetrieval​(boolean yes)
        Set RETRIEVAL_LOADING_PROFILE
        Parameters:
        yes -

      • addIndexStructure

        public void addIndexStructure​(java.lang.String structureName,
                              java.lang.Class<?> className,
                              java.lang.Class<?>[] paramTypes,
                              java.lang.String[] paramValues)
        tell the index about a new index structure it provides. Class and parameter types specified as Class objects instead of Strings.

      • addIndexStructure

        public void addIndexStructure​(java.lang.String structureName,
                              java.lang.String className)
        add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation parameters are "String,String", which are "path,prefix".
        Parameters:
        structureName -
        className -

      • addIndexStructure

        public void addIndexStructure​(java.lang.String structureName,
                              java.lang.String className,
                              java.lang.String paramTypes,
                              java.lang.String paramValues)
        add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation type parameters or values are non-default.

      • addIndexStructure

        public void addIndexStructure​(java.lang.String structureName,
                              java.lang.String className,
                              java.lang.String[] paramTypes,
                              java.lang.String[] paramValues)
        add an index structure to this index. Structure will be called structureName, and instantiated by a class called className. Instantiation type parameters or values are non-default.

      • addIndexStructureInputStream

        public void addIndexStructureInputStream​(java.lang.String structureName,
                                         java.lang.Class<?> className,
                                         java.lang.Class<?>[] paramTypes,
                                         java.lang.String[] paramValues)
        tell the index about a new input stream index structure it provides.

      • addIndexStructureInputStream

        public void addIndexStructureInputStream​(java.lang.String structureName,
                                         java.lang.String className)
        tell the index about a new input stream index structure it provides.

      • addIndexStructureInputStream

        public void addIndexStructureInputStream​(java.lang.String structureName,
                                         java.lang.String className,
                                         java.lang.String paramTypes,
                                         java.lang.String paramValues)
        tell the index about a new input stream index structure it provides.

      • addIndexStructureInputStream

        public void addIndexStructureInputStream​(java.lang.String structureName,
                                         java.lang.String className,
                                         java.lang.String[] paramTypes,
                                         java.lang.String[] paramValues)
        tell the index about a new input stream index structure it provides.

      • close

        public abstract void close()
                    throws java.io.IOException
        Closes the data structures of the index.
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Overrides:
        close in class Index
        Throws:
        java.io.IOException

      • flush

        public abstract void flush()
                    throws java.io.IOException
        Write any dirty data structures down to disk
        Specified by:
        flush in interface java.io.Flushable
        Throws:
        java.io.IOException

      • getDirectIndex

        public abstract PostingIndex<?> getDirectIndex()
        Return the DirectIndex associated with this index
        Specified by:
        getDirectIndex in class Index

      • getIndexProperty

        public java.lang.String getIndexProperty​(java.lang.String key,
                                         java.lang.String defaultValue)
        get an arbitrary property in the index
        Parameters:
        key - Key of the property to get
        defaultValue - value of the property to use if property is not set
        Returns:
        Value of the property

      • getIndexRef

        public IndexRef getIndexRef()
        Description copied from class: Index
        Returns a direct IndexRef to this index
        Specified by:
        getIndexRef in class Index

      • getIndexStructure

        public abstract java.lang.Object getIndexStructure​(java.lang.String structureName)
        Obtains the named index structure, using an already loaded one if possible.
        Overrides:
        getIndexStructure in class Index
        Parameters:
        structureName - name of the required structure
        Returns:
        desired object or null if not found

      • getIndexStructureInputStream

        public abstract java.lang.Object getIndexStructureInputStream​(java.lang.String structureName)
        Return the input stream associated with the specified structure of this index
        Overrides:
        getIndexStructureInputStream in class Index
        Parameters:
        structureName - The name of the structure of which you want the inputstream. Eg "lexicon"
        Returns:
        Required structure, or null if not found

      • getIntIndexProperty

        public int getIntIndexProperty​(java.lang.String key,
                               int defaultValue)
        get an arbitrary int property from the index

      • getInvertedIndex

        public abstract PostingIndex<?> getInvertedIndex()
        Returns the InvertedIndex to use for this index
        Specified by:
        getInvertedIndex in class Index

      • getLexicon

        public abstract Lexicon<java.lang.String> getLexicon()
        Return the Lexicon associated with this index
        Specified by:
        getLexicon in class Index

      • getMetaIndex

        public abstract MetaIndex getMetaIndex()
        Get the Meta Index structure
        Specified by:
        getMetaIndex in class Index

      • getProperties

        public java.util.Properties getProperties()
        Get the index properties

      • hasIndexStructure

        public boolean hasIndexStructure​(java.lang.String structureName)
        Does this index have an index structure with the specified name?
        Overrides:
        hasIndexStructure in class Index
        Parameters:
        structureName - name of the required structure
        Returns:
        true if the index has an appropriately named structure

      • hasIndexStructureInputStream

        public boolean hasIndexStructureInputStream​(java.lang.String structureName)
        Does this index have an index structure input stream with the specified name?
        Overrides:
        hasIndexStructureInputStream in class Index
        Parameters:
        structureName - name of the required structure
        Returns:
        true if the index has an appropriately named structure

      • setIndexProperty

        public void setIndexProperty​(java.lang.String key,
                             java.lang.String value)
        set an arbitrary property in the index
        Parameters:
        key - Key to of the property to set
        value - Value of the property to set

      • toString

        public abstract java.lang.String toString()
        Returns a String representation of this index
        Overrides:
        toString in class java.lang.Object

      • getStart

        public int getStart()
        Returns the first docid in this index
        Overrides:
        getStart in class Index

      • getEnd

        public int getEnd()
        Returns the last docid in this index
        Overrides:
        getEnd in class Index