Package org.terrier.realtime.memory

Class MemoryMetaIndex

  • All Implemented Interfaces:
    java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, MetaIndex
    Direct Known Subclasses:
    MemoryCompressedMetaIndex, MemoryMetaIndexMap
    public class MemoryMetaIndex
extends MetaIndexBuilder
implements MetaIndex, java.io.Serializable
    An in-memory version of a Meta-data index. It stores additional information about each document, e.g. the docno or title. Access to the memory versions of the meta index are faster than the on-disk versions, but can use up large amounts of RAM to store.

    Properties

    • indexer.meta.forward.keys - key names to store in the meta index
    • indexer.meta.forward.keylens - max key lengths for keys to store (this is ignored unless metaindex.crop is set)
    • metaindex.compressed.crop.long - should the content to store be cropped down to the length specified in indexer.meta.forward.keylens?
    Since:
    4.0
    Author:
    Richard McCreadie, Stuart Mackie, Craig Macdonald
    See Also:
    Serialized Form

    • Constructor Summary

      Constructors 
      Constructor Description
      MemoryMetaIndex()  
      MemoryMetaIndex​(java.lang.String[] metaKeys, int[] metaLengths)  
      MemoryMetaIndex​(java.lang.String[] metaKeys, int[] metaLengths, java.lang.String[] revKeys)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected int _binarySearch​(java.lang.String key, java.lang.String value)
      performs a binary search on the metaindex, if they keys happen to be in lexographical order
      void close()
      Delete contents of metadata index (but keep the configured keys).
      java.lang.String[] getAllItems​(int docid)
      Obtain all metadata for specified document.
      int getDocument​(java.lang.String key, java.lang.String value)
      Obtain docid where document has specified metadata value in the specified type.
      java.lang.String getItem​(java.lang.String key, int docid)
      Obtain metadata of specified type for specified document.
      java.lang.String[] getItems​(java.lang.String[] keys, int docid)
      Obtain metadata of specified types for specified document.
      java.lang.String[][] getItems​(java.lang.String[] keys, int[] docids)
      Obtain metadata of specified types for specified documents.
      java.lang.String[] getItems​(java.lang.String key, int[] docids)
      Obtain metadata of specified type for specified documents.
      int[] getKeyLengths()  
      java.lang.String[] getKeys()
      Returns the keys of this meta index
      java.lang.String[] getReverseKeys()
      Returns the reverse keys of this meta index
      java.util.Iterator<java.lang.String[]> iterator()
      Return iterator over meta-data index.
      int size()
      How many documents in this metaindex
      void writeDocumentEntry​(java.lang.String[] data)
      This method has been made public for creating
      void writeDocumentEntry​(java.util.Map<java.lang.String,​java.lang.String> data)
      Write meta-data for document.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • MemoryMetaIndex

        public MemoryMetaIndex()

      • MemoryMetaIndex

        public MemoryMetaIndex​(java.lang.String[] metaKeys,
                       int[] metaLengths)

      • MemoryMetaIndex

        public MemoryMetaIndex​(java.lang.String[] metaKeys,
                       int[] metaLengths,
                       java.lang.String[] revKeys)

    • Method Detail

      • size

        public int size()
        How many documents in this metaindex
        Specified by:
        size in interface MetaIndex

      • getKeys

        public java.lang.String[] getKeys()
        Returns the keys of this meta index
        Specified by:
        getKeys in interface MetaIndex

      • getItem

        public java.lang.String getItem​(java.lang.String key,
                                int docid)
                         throws java.io.IOException
        Obtain metadata of specified type for specified document.
        Specified by:
        getItem in interface MetaIndex
        Throws:
        java.io.IOException

      • getAllItems

        public java.lang.String[] getAllItems​(int docid)
                               throws java.io.IOException
        Obtain all metadata for specified document.
        Specified by:
        getAllItems in interface MetaIndex
        Throws:
        java.io.IOException

      • getItems

        public java.lang.String[] getItems​(java.lang.String key,
                                   int[] docids)
                            throws java.io.IOException
        Obtain metadata of specified type for specified documents.
        Specified by:
        getItems in interface MetaIndex
        Throws:
        java.io.IOException

      • getItems

        public java.lang.String[] getItems​(java.lang.String[] keys,
                                   int docid)
                            throws java.io.IOException
        Obtain metadata of specified types for specified document.
        Specified by:
        getItems in interface MetaIndex
        Throws:
        java.io.IOException

      • getItems

        public java.lang.String[][] getItems​(java.lang.String[] keys,
                                     int[] docids)
                              throws java.io.IOException
        Obtain metadata of specified types for specified documents. Return array is indexed by documents, then by metakeys.
        Specified by:
        getItems in interface MetaIndex
        Throws:
        java.io.IOException

      • writeDocumentEntry

        public void writeDocumentEntry​(java.util.Map<java.lang.String,​java.lang.String> data)
        Write meta-data for document.
        Specified by:
        writeDocumentEntry in class MetaIndexBuilder

      • writeDocumentEntry

        public void writeDocumentEntry​(java.lang.String[] data)
        This method has been made public for creating
        Specified by:
        writeDocumentEntry in class MetaIndexBuilder
        Parameters:
        data -

      • _binarySearch

        protected int _binarySearch​(java.lang.String key,
                            java.lang.String value)
                     throws java.io.IOException
        performs a binary search on the metaindex, if they keys happen to be in lexographical order
        Throws:
        java.io.IOException

      • getDocument

        public int getDocument​(java.lang.String key,
                       java.lang.String value)
                throws java.io.IOException
        Description copied from interface: MetaIndex
        Obtain docid where document has specified metadata value in the specified type. Returns -1 if the value cannot be found for the specified key type.
        Specified by:
        getDocument in interface MetaIndex
        Throws:
        java.io.IOException

      • getReverseKeys

        public java.lang.String[] getReverseKeys()
        Description copied from interface: MetaIndex
        Returns the reverse keys of this meta index
        Specified by:
        getReverseKeys in interface MetaIndex

      • close

        public void close()
           throws java.io.IOException
        Delete contents of metadata index (but keep the configured keys).
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException

      • iterator

        public java.util.Iterator<java.lang.String[]> iterator()
        Return iterator over meta-data index.

      • getKeyLengths

        public int[] getKeyLengths()