uk.ac.gla.terrier.structures
Class ExpansionTerms

java.lang.Object
  uk.ac.gla.terrier.structures.ExpansionTerms

public class ExpansionTerms
extends java.lang.Object

This class implements a data structure of terms in the top-retrieved documents.

Properties:

• expansion.mindocuments - the minimum number of documents a term must exist in before it can be considered to be informative. Defaults to 2. For more information, see Giambattista Amati: Information Theoretic Approach to Information Extraction. FQAS 2006: 519-529 DOI 10.1007/11766254_44

Version:

$Revision: 1.40 $

Author:

Gianni Amati, Ben He, Vassilis Plachouras, Craig Macdonald

Nested Class Summary

static class

ExpansionTerms.ExpansionTerm This class implements a data structure for a term in the top-retrieved documents.

Field Summary

double

normaliser The parameter-free term weight normaliser.

Constructor Summary

ExpansionTerms(CollectionStatistics collStats, double totalLength, Lexicon lexicon)
Constructs an instance of ExpansionTerms.

ExpansionTerms(int numberOfDocuments, long numberOfTokens, double averageDocumentLength, double totalLength, Lexicon lexicon)
Constructs an instance of ExpansionTerms.

Method Summary

void	assignWeights(QueryExpansionModel QEModel) Assign weight to terms that are stored in ExpansionTerm[] terms.
void	deleteTerm(int termid) Remove the records for a given term
double	getDocumentFrequency(int termId) Returns the number of the top-ranked documents a given term occurs in.
SingleTermQuery[]	getExpandedTerms(int numberOfExpandedTerms, QueryExpansionModel QEModel) This method implements the functionality of assigning expansion weights to the terms in the top-retrieved documents, and returns the most informative terms among them.
double	getExpansionProbability(int termId) Returns the probability of a given termid occurring in the expansion documents.
double	getExpansionWeight(int termId) Returns the weight of a term with the given term identifier.
double	getExpansionWeight(int termId, QueryExpansionModel model) Returns the weight of a term with the given term identifier, computed by the specified query expansion model.
double	getExpansionWeight(java.lang.String term) Returns the weight of a given term.
double	getExpansionWeight(java.lang.String term, QueryExpansionModel model) Returns the weight of a given term, computed by the specified query expansion model.
double	getFrequency(int termId) Returns the frequency of a given term in the top-ranked documents.
double	getFrequency(java.lang.String term) Returns the frequency of a given term in the top-ranked documents.
int	getNumberOfUniqueTerms() Returns the unique number of terms found in all the top-ranked documents
double	getOriginalExpansionWeight(java.lang.String term) Returns the un-normalised weight of a given term.
int[]	getTermIds() Returns the termids of all terms found in the top-ranked documents
void	insertTerm(int termID, double withinDocumentFrequency) Add a term in the X top-retrieved documents as a candidate of the expanded terms.
void	setOriginalQueryTerms(MatchingQueryTerms query) Set the original query terms.
void	setTotalDocumentLength(double totalLength) Allows the totalDocumentLength to be set after the fact

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

normaliser

public double normaliser

The parameter-free term weight normaliser.

Constructor Detail

ExpansionTerms

public ExpansionTerms(CollectionStatistics collStats,
                      double totalLength,
                      Lexicon lexicon)

Constructs an instance of ExpansionTerms.

Parameters:

totalLength - The sum of the length of the top-retrieved documents.

lexicon - Lexicon The lexicon used for retrieval.

ExpansionTerms

public ExpansionTerms(int numberOfDocuments,
                      long numberOfTokens,
                      double averageDocumentLength,
                      double totalLength,
                      Lexicon lexicon)

Constructs an instance of ExpansionTerms.

Parameters:

totalLength - The sum of the length of the top-retrieved documents.

lexicon - Lexicon The lexicon used for retrieval.

Method Detail

setTotalDocumentLength

public void setTotalDocumentLength(double totalLength)

Allows the totalDocumentLength to be set after the fact

getTermIds

public int[] getTermIds()

Returns the termids of all terms found in the top-ranked documents

getNumberOfUniqueTerms

public int getNumberOfUniqueTerms()

Returns the unique number of terms found in all the top-ranked documents

getExpandedTerms

public SingleTermQuery[] getExpandedTerms(int numberOfExpandedTerms,
                                          QueryExpansionModel QEModel)

This method implements the functionality of assigning expansion weights to the terms in the top-retrieved documents, and returns the most informative terms among them. Conservative Query Expansion (ConservativeQE) is used if the number of expanded terms is set to 0. In this case, no new query terms are added to the query, only the existing ones reweighted.

Parameters:

numberOfExpandedTerms - int The number of terms to extract from the top-retrieved documents. ConservativeQE is set if this parameter is set to 0.

QEModel - QueryExpansionModel the model used for query expansion

Returns:

TermTreeNode[] The expanded terms.

setOriginalQueryTerms

public void setOriginalQueryTerms(MatchingQueryTerms query)

Set the original query terms.

Parameters:

query - The original query.

deleteTerm

public void deleteTerm(int termid)

Remove the records for a given term

getExpansionWeight

public double getExpansionWeight(java.lang.String term,
                                 QueryExpansionModel model)

Returns the weight of a given term, computed by the specified query expansion model.

Parameters:

term - String the term to set the weight for.

model - QueryExpansionModel the used query expansion model.

Returns:

double the weight of the specified term.

getExpansionWeight

public double getExpansionWeight(java.lang.String term)

Returns the weight of a given term.

Parameters:

term - String the term to get the weight for.

Returns:

double the weight of the specified term.

getOriginalExpansionWeight

public double getOriginalExpansionWeight(java.lang.String term)

Returns the un-normalised weight of a given term.

Parameters:

term - String the given term.

Returns:

The un-normalised term weight.

getFrequency

public double getFrequency(java.lang.String term)

Returns the frequency of a given term in the top-ranked documents.

Parameters:

term - String the term to get the frequency for.

Returns:

double the frequency of the specified term in the top-ranked documents.

getFrequency

public double getFrequency(int termId)

Returns the frequency of a given term in the top-ranked documents.

Parameters:

termId - int the id of the term to get the frequency for.

Returns:

double the frequency of the specified term in the top-ranked documents.

getDocumentFrequency

public double getDocumentFrequency(int termId)

Returns the number of the top-ranked documents a given term occurs in.

Parameters:

termId - int the id of the term to get the frequency for.

Returns:

double the document frequency of the specified term in the top-ranked documents.

assignWeights

public void assignWeights(QueryExpansionModel QEModel)

Assign weight to terms that are stored in ExpansionTerm[] terms.

Parameters:

QEModel - QueryExpansionModel the used query expansion model.

getExpansionWeight

public double getExpansionWeight(int termId,
                                 QueryExpansionModel model)

Returns the weight of a term with the given term identifier, computed by the specified query expansion model.

Parameters:

termId - int the term identifier to set the weight for.

model - QueryExpansionModel the used query expansion model.

Returns:

double the weight of the specified term.

getExpansionWeight

public double getExpansionWeight(int termId)

Returns the weight of a term with the given term identifier.

Parameters:

termId - int the term identifier to set the weight for.

Returns:

double the weight of the specified term.

getExpansionProbability

public double getExpansionProbability(int termId)

Returns the probability of a given termid occurring in the expansion documents. Returns the quotient document frequency in the expansion documents, divided by the total length of all the expansion documents.

Parameters:

termId - int the term identifier to obtain the probability

Returns:

double the probability of the term

insertTerm

public void insertTerm(int termID,
                       double withinDocumentFrequency)

Add a term in the X top-retrieved documents as a candidate of the expanded terms.

Parameters:

termID - int the integer identifier of a term

withinDocumentFrequency - double the within document frequency of a term