org.terrier.evaluation
Class TRECQrelsInMemory

java.lang.Object
  org.terrier.evaluation.TRECQrelsInMemory

public class TRECQrelsInMemory
extends java.lang.Object

Loads the relevance assessments in memory, for performing evaluation of runs.

Author:

Ben He & Vassilis Plachouras

Nested Class Summary

static class

TRECQrelsInMemory.QrelsHashSet Models the set of relevant documents for one query.

Field Summary

protected java.io.File[]	fqrels An array with the qrels files.
protected static org.apache.log4j.Logger	logger
TRECQrelsInMemory.QrelsHashSet[]	qrelsPerQuery Each element in the array contains the docids of the relevant documents with respect to a query.
int	totalNumberOfRelevantDocs The total number of relevant documents.

Constructor Summary

TRECQrelsInMemory()
* A default constructor that creates an instance of the class and loads in memory the relevance assessments from the files that are specified in the file specified by the property TREC_QRELS.

TRECQrelsInMemory(java.lang.String qrelsFilename)
A constructor that creates an instance of the class and loads in memory the relevance assessments from the given file.

TRECQrelsInMemory(java.lang.String[] qrelsFilenames)
A constructor that creates an instance of the class and loads in memory the relevance assessments from the given files.

Method Summary

gnu.trove.THashSet<java.lang.String>	getAllRelevantDocuments() Get the identifiers of all relevant documents in the pool.
java.lang.String[]	getAllRelevantDocumentsToArray() Get the identifiers of all relevant documents in the pool.
int	getGrade(java.lang.String qid, java.lang.String docno, int def) Returns the grade of a document for a given query.
gnu.trove.THashSet<java.lang.String>	getNonRelevantDocuments() Get all the pooled non-relevant documents.
gnu.trove.THashSet<java.lang.String>	getNonRelevantDocuments(java.lang.String queryid) Get the pooled non-relevant documents for the given query.
java.lang.String[]	getNonRelevantDocumentsToArray() Get all the pooled non-relevant documents.
java.lang.String[]	getNonRelevantDocumentsToArray(java.lang.String queryid) Get the pooled non-relevant documents for a given query.
int	getNumberOfQueries() Returns the total number of queries contained in the loaded relevance assessments.
int	getNumberOfRelevant(java.lang.String queryid) Returns the numbe of relevant documents for a given query.
java.lang.String[]	getQueryids() Get ids of the queries that appear in the pool.
gnu.trove.THashSet<java.lang.String>	getRelevantDocuments(int grade) Get the identifiers of all relevant documents in the pool with a given relevance grade.
java.lang.String[]	getRelevantDocuments(int[] grades) Get the identifiers of all relevant documents with the given relevance grades.
gnu.trove.THashSet<java.lang.String>	getRelevantDocuments(java.lang.String queryid) Get the pooled relevant documents for the given query.
gnu.trove.THashSet<java.lang.String>	getRelevantDocuments(java.lang.String queryid, int grade) Get the pooled relevant documents for the given query.
gnu.trove.THashSet<java.lang.String>	getRelevantDocuments(java.lang.String queryid, int[] grades) Get the pooled relevant documents for the given query.
java.lang.String[]	getRelevantDocumentsToArray(int grade) Get the identifiers of all relevant documents in the pool with the given relevance grade.
java.lang.String[]	getRelevantDocumentsToArray(java.lang.String queryid) Get the pooled relevant documents for a given query.
java.lang.String[]	getRelevantDocumentsToArray(java.lang.String queryid, int grade) Get the pooled relevant documents for a given query.
java.lang.String[]	getRelevantDocumentsToArray(java.lang.String queryid, int[] grades) Get the pooled relevant documents for the given query.
boolean	isRelevant(java.lang.String queryid, java.lang.String docno) Check if the given document is relevant for a given query.
protected void	loadQrelsFile() Load in memory the relevance assessment files that are specified in the array fqrels.
boolean	queryExistInQrels(java.lang.String queryid) Checks whether there is a query with a given identifier in the relevance assessments.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

qrelsPerQuery

public TRECQrelsInMemory.QrelsHashSet[] qrelsPerQuery

Each element in the array contains the docids of the relevant documents with respect to a query.

fqrels

protected java.io.File[] fqrels

An array with the qrels files.

totalNumberOfRelevantDocs

public int totalNumberOfRelevantDocs

The total number of relevant documents.

Constructor Detail

TRECQrelsInMemory

public TRECQrelsInMemory(java.lang.String qrelsFilename)

A constructor that creates an instance of the class and loads in memory the relevance assessments from the given file.

Parameters:

qrelsFilename - String The full path of the qrels file to load.

TRECQrelsInMemory

public TRECQrelsInMemory(java.lang.String[] qrelsFilenames)

A constructor that creates an instance of the class and loads in memory the relevance assessments from the given files.

Parameters:

qrelsFilenames - String[] The full path of the qrels files to load.

TRECQrelsInMemory

public TRECQrelsInMemory()

* A default constructor that creates an instance of the class and loads in memory the relevance assessments from the files that are specified in the file specified by the property TREC_QRELS.

Method Detail

getQueryids

public java.lang.String[] getQueryids()

Get ids of the queries that appear in the pool.

Returns:

The ids of queries in the pool.

getRelevantDocuments

public java.lang.String[] getRelevantDocuments(int[] grades)

Get the identifiers of all relevant documents with the given relevance grades.

Parameters:

grades - The relevance grades.

Returns:

The identifiers of all relevant documents with the given relevance grades.

getAllRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getAllRelevantDocuments()

Get the identifiers of all relevant documents in the pool.

Returns:

The identifiers of all relevant documents in the pool.

getRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getRelevantDocuments(int grade)

Get the identifiers of all relevant documents in the pool with a given relevance grade.

Returns:

The identifiers of all relevant documents in the pool.

getAllRelevantDocumentsToArray

public java.lang.String[] getAllRelevantDocumentsToArray()

Get the identifiers of all relevant documents in the pool.

Returns:

The identifiers of all relevant documents in the pool.

getRelevantDocumentsToArray

public java.lang.String[] getRelevantDocumentsToArray(int grade)

Get the identifiers of all relevant documents in the pool with the given relevance grade.

Returns:

The identifiers of all relevant documents in the pool.

getNonRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getNonRelevantDocuments(java.lang.String queryid)

Get the pooled non-relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled non-relevant documents for the given query.

getNonRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getNonRelevantDocuments()

Get all the pooled non-relevant documents.

Returns:

All the pooled non-relevant documents.

getNonRelevantDocumentsToArray

public java.lang.String[] getNonRelevantDocumentsToArray()

Get all the pooled non-relevant documents.

Returns:

All the pooled non-relevant documents.

getRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getRelevantDocuments(java.lang.String queryid)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getRelevantDocuments(java.lang.String queryid,
                                                                 int grade)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocuments

public gnu.trove.THashSet<java.lang.String> getRelevantDocuments(java.lang.String queryid,
                                                                 int[] grades)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocumentsToArray

public java.lang.String[] getRelevantDocumentsToArray(java.lang.String queryid,
                                                      int[] grades)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getNonRelevantDocumentsToArray

public java.lang.String[] getNonRelevantDocumentsToArray(java.lang.String queryid)

Get the pooled non-relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled non-relevant documents for the given query.

getRelevantDocumentsToArray

public java.lang.String[] getRelevantDocumentsToArray(java.lang.String queryid)

Get the pooled relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled relevant documents for the given query.

getRelevantDocumentsToArray

public java.lang.String[] getRelevantDocumentsToArray(java.lang.String queryid,
                                                      int grade)

Get the pooled relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled relevant documents for the given query.

getNumberOfQueries

public int getNumberOfQueries()

Returns the total number of queries contained in the loaded relevance assessments.

Returns:

int The number of unique queries in the loaded relevance assessments.

getNumberOfRelevant

public int getNumberOfRelevant(java.lang.String queryid)

Returns the numbe of relevant documents for a given query.

Parameters:

queryid - String The identifier of a query.

Returns:

int The number of relevant documents for the given query.

loadQrelsFile

protected void loadQrelsFile()

Load in memory the relevance assessment files that are specified in the array fqrels.

queryExistInQrels

public boolean queryExistInQrels(java.lang.String queryid)

Checks whether there is a query with a given identifier in the relevance assessments.

Parameters:

queryid - String the identifier of a query.

Returns:

boolean true if the given query exists in the qrels file, otherwise it returns false.

isRelevant

public boolean isRelevant(java.lang.String queryid,
                          java.lang.String docno)

Check if the given document is relevant for a given query.

Parameters:

queryid - String a query identifier.

docno - String a document identifier.

Returns:

boolean true if the given document is relevant for the given query, or otherwise false.

getGrade

public int getGrade(java.lang.String qid,
                    java.lang.String docno,
                    int def)

Returns the grade of a document for a given query.

Parameters:

qid - String the identifier of the desired query.

docno - String a document identifier.

def - The default value to be returned if the document is not found.

Returns:

int The grade of the document for the given query.