org.terrier.matching

Class PostingListManager

java.lang.Object

org.terrier.matching.PostingListManager

All Implemented Interfaces:

Closeable, AutoCloseable

public class PostingListManager
extends Object
implements Closeable

The PostingListManager is reponsible for opening the appropriate posting lists IterablePosting given the MatchingQueryTerms object. Moreover, it knows how each Posting should be scored.

Plugins are also supported by PostingListManager. Each plugin class should implement the PostingListManagerPlugin interface, and be named explicitly in the matching.postinglist.manager.plugins property.

Properties:

• ignore.low.idf.terms - should terms with low IDF (i.e. very frequent) be ignored? Defaults to true, i.e. ignored
• matching.postinglist.manager.plugins - Comma delimited list of PostingListManagerPlugin classes to load.

Example Usage

Following code shows how term-at-a-time matching may occur using the PostingListManager:

 
 MatchingQueryTerms mqt;
 Index index;
 PostingListManager plm = new PostingListManager(index, index.getCollectionStatistics(), mqt);
 plm.prepare(false);
 for(int term = 0;term < plm.size(); term++)
 {
   IterablePosting ip = plm.get(term);
   while(ip.next() != IterablePosting.EOL)
   {
     double score = plm.score(term);
     int id = ip.getId();
   }
 }
 plm.close();
 

Since:

3.5

Author:

Nicola Tonellotto and Craig Macdonald

See Also:

Matching

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	PostingListManager.PostingListManagerPlugin Interface for plugins to further alter the posting lists managed by the PostingListManager

Field Summary

Fields

Modifier and Type	Field and Description
protected CollectionStatistics	collectionStatistics statistics of the collection
protected static boolean	IGNORE_LOW_IDF_TERMS A property that enables to ignore the terms with a low IDF.
protected Index	index underlying index
protected PostingIndex<Pointer>	invertedIndex inverted index of the index
protected Lexicon<String>	lexicon lexicon for the index
protected static org.apache.log4j.Logger	logger
protected int	numTerms number of terms
protected gnu.trove.TDoubleArrayList	termKeyFreqs String form for each term
protected List<WeightingModel[]>	termModels weighting models for each term
protected List<IterablePosting>	termPostings posting lists for each term
protected List<EntryStatistics>	termStatistics EntryStatistics for each term
protected List<String>	termStrings String form for each term

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	PostingListManager(Index _index, CollectionStatistics cs) Create a posting list manager for the given index and statistics
	PostingListManager(Index _index, CollectionStatistics _cs, MatchingQueryTerms mqt) Create a posting list manager for the given index and statistics, and populated using the specified MatchingQueryTerms.
	PostingListManager(Index _index, CollectionStatistics _cs, MatchingQueryTerms mqt, boolean splitSynonyms) Create a posting list manager for the given index and statistics, and populated using the specified MatchingQueryTerms.

Method Summary

Methods

Modifier and Type	Method and Description
void	addSingleTerm(String queryTerm, double weight, EntryStatistics entryStats, WeightingModel[] wmodels) Add a single term to those to be matched for this query.
void	addSingleTermAlternatives(String[] terms, String stringForm, double weight, EntryStatistics[] entryStats, WeightingModel[] wmodels) Adds a synonym group to the matching process.
void	addSingleTermAlternatives(String[] terms, String stringForm, double weight, EntryStatistics entryStats, WeightingModel[] wmodels) Adds a synonym group to the matching process.
void	close()
double	getKeyFrequency(int i)
int	getNumTerms() Returns the number of postings lists (that are terms) for this query
IterablePosting	getPosting(int i) Returns the IterablePosting corresponding to the specified term
EntryStatistics	getStatistics(int i) Returns the EntryStatistics corresponding to the specified term
String	getTerm(int i)
static EntryStatistics	mergeStatistics(EntryStatistics[] entryStats) Knows how to merge several EntryStatistics for a single effective term
void	prepare(boolean firstMove) Counts the number of terms active.
double	score(int i) Returns the score using all weighting models for the current posting of the specified term
int	size() Returns the number of posting lists for this query

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

IGNORE_LOW_IDF_TERMS

protected static boolean IGNORE_LOW_IDF_TERMS

A property that enables to ignore the terms with a low IDF. Controlled by ignore.low.idf.terms property, defualts to true.

termPostings

protected final List<IterablePosting> termPostings

posting lists for each term

termModels

protected final List<WeightingModel[]> termModels

weighting models for each term

termStatistics

protected final List<EntryStatistics> termStatistics

EntryStatistics for each term

termStrings

protected final List<String> termStrings

String form for each term

termKeyFreqs

protected final gnu.trove.TDoubleArrayList termKeyFreqs

String form for each term

numTerms

protected int numTerms

number of terms

index

protected Index index

underlying index

lexicon

protected Lexicon<String> lexicon

lexicon for the index

invertedIndex

protected PostingIndex<Pointer> invertedIndex

inverted index of the index

collectionStatistics

protected CollectionStatistics collectionStatistics

statistics of the collection

Constructor Detail

PostingListManager

protected PostingListManager(Index _index,
                  CollectionStatistics cs)
                      throws IOException

Create a posting list manager for the given index and statistics

Throws:

IOException

PostingListManager

public PostingListManager(Index _index,
                  CollectionStatistics _cs,
                  MatchingQueryTerms mqt)
                   throws IOException

Create a posting list manager for the given index and statistics, and populated using the specified MatchingQueryTerms.

Parameters:

_index - - index to obtain postings from

_cs - - collection statistics to obtain

mqt - - MatchingQueryTerms object calculated for the query

Throws:

IOException

PostingListManager

public PostingListManager(Index _index,
                  CollectionStatistics _cs,
                  MatchingQueryTerms mqt,
                  boolean splitSynonyms)
                   throws IOException

Create a posting list manager for the given index and statistics, and populated using the specified MatchingQueryTerms.

Parameters:

_index - - index to obtain postings from

_cs - - collection statistics to obtain

mqt - - MatchingQueryTerms object calculated for the query

splitSynonyms - - allows the splitting of synonym groups (i.e. singleTermAlternatives) to be disabled

Throws:

IOException

Method Detail

addSingleTerm

public void addSingleTerm(String queryTerm,
                 double weight,
                 EntryStatistics entryStats,
                 WeightingModel[] wmodels)
                   throws IOException

Add a single term to those to be matched for this query. Those with more occurrences than the number of documents will be ignored if IGNORE_LOW_IDF_TERMS is enabled.

Parameters:

queryTerm - String form of the query term

weight - influence of this query term in scoring

entryStats - statistics to be used for this query term. If null, these will be obtained from the local Lexicon

wmodels - weighting models to be applied for this query term

Throws:

IOException

mergeStatistics

public static EntryStatistics mergeStatistics(EntryStatistics[] entryStats)

Knows how to merge several EntryStatistics for a single effective term

addSingleTermAlternatives

public void addSingleTermAlternatives(String[] terms,
                             String stringForm,
                             double weight,
                             EntryStatistics[] entryStats,
                             WeightingModel[] wmodels)
                               throws IOException

Adds a synonym group to the matching process. EntryStatistics for all terms in the group will be combined using mergeStatistics()

Parameters:

terms - String of the terms in the synonym group

weight - influence of this synonym group during retrieval

entryStats - statistics of the terms in the synonym group. If null, these will be obtained from the local Lexicon.

wmodels - WeightingModels for the synonym group (NOT one per member).

Throws:

IOException

addSingleTermAlternatives

public void addSingleTermAlternatives(String[] terms,
                             String stringForm,
                             double weight,
                             EntryStatistics entryStats,
                             WeightingModel[] wmodels)
                               throws IOException

Adds a synonym group to the matching process.

Parameters:

terms - String of the terms in the synonym group

weight - influence of this synonym group during retrieval

entryStats - statistics of the whole synonym group. If null, these will be obtained from the local Lexicon for all terms in the group will be combined using mergeStatistics()

wmodels - WeightingModels for the synonym group (NOT one per member).

Throws:

IOException

prepare

public void prepare(boolean firstMove)
             throws IOException

Counts the number of terms active. If firstMove is true, it will move each posting to the first posting.

Parameters:

firstMove - move all postings to the start?

Throws:

IOException

getStatistics

public EntryStatistics getStatistics(int i)

Returns the EntryStatistics corresponding to the specified term

Parameters:

i - term to obtain statistics for

Returns:

Statistics for this i-1th term

getPosting

public IterablePosting getPosting(int i)

Returns the IterablePosting corresponding to the specified term

Parameters:

i - term to obtain the posting list for

Returns:

Posting list for this i-1th term

size

public int size()

Returns the number of posting lists for this query

getNumTerms

public int getNumTerms()

Returns the number of postings lists (that are terms) for this query

score

public double score(int i)

Returns the score using all weighting models for the current posting of the specified term

Parameters:

i - Which term to score

Returns:

score obtained from all weighting models for that term

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

getTerm

public String getTerm(int i)

getKeyFrequency

public double getKeyFrequency(int i)