org.terrier.evaluation

Class TRECQrelsInMemory

java.lang.Object

org.terrier.evaluation.TRECQrelsInMemory

public class TRECQrelsInMemory
extends Object

Loads the relevance assessments in memory, for performing evaluation of runs.

Author:

Ben He & Vassilis Plachouras

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TRECQrelsInMemory.QrelsHashSet Models the set of relevant documents for one query.

Field Summary

Fields

Modifier and Type	Field and Description
protected File[]	fqrels An array with the qrels files.
protected static org.slf4j.Logger	logger
TRECQrelsInMemory.QrelsHashSet[]	qrelsPerQuery Each element in the array contains the docids of the relevant documents with respect to a query.
int	totalNumberOfRelevantDocs The total number of relevant documents.

Constructor Summary

Constructors

Constructor and Description
TRECQrelsInMemory() * A default constructor that creates an instance of the class and loads in memory the relevance assessments from the files that are specified in the file specified by the property TREC_QRELS.
TRECQrelsInMemory(String qrelsFilename) A constructor that creates an instance of the class and loads in memory the relevance assessments from the given file.
TRECQrelsInMemory(String[] qrelsFilenames) A constructor that creates an instance of the class and loads in memory the relevance assessments from the given files.

Method Summary

Modifier and Type	Method and Description
gnu.trove.THashSet<String>	getAllRelevantDocuments() Get the identifiers of all relevant documents in the pool.
String[]	getAllRelevantDocumentsToArray() Get the identifiers of all relevant documents in the pool.
int	getGrade(String qid, String docno, int def) Returns the grade of a document for a given query.
gnu.trove.THashSet<String>	getNonRelevantDocuments() Get all the pooled non-relevant documents.
gnu.trove.THashSet<String>	getNonRelevantDocuments(String queryid) Get the pooled non-relevant documents for the given query.
String[]	getNonRelevantDocumentsToArray() Get all the pooled non-relevant documents.
String[]	getNonRelevantDocumentsToArray(String queryid) Get the pooled non-relevant documents for a given query.
int	getNumberOfQueries() Returns the total number of queries contained in the loaded relevance assessments.
int	getNumberOfRelevant(String queryid) Returns the numbe of relevant documents for a given query.
String[]	getQueryids() Get ids of the queries that appear in the pool.
gnu.trove.THashSet<String>	getRelevantDocuments(int grade) Get the identifiers of all relevant documents in the pool with a given relevance grade.
String[]	getRelevantDocuments(int[] grades) Get the identifiers of all relevant documents with the given relevance grades.
gnu.trove.THashSet<String>	getRelevantDocuments(String queryid) Get the pooled relevant documents for the given query.
gnu.trove.THashSet<String>	getRelevantDocuments(String queryid, int grade) Get the pooled relevant documents for the given query.
gnu.trove.THashSet<String>	getRelevantDocuments(String queryid, int[] grades) Get the pooled relevant documents for the given query.
String[]	getRelevantDocumentsToArray(int grade) Get the identifiers of all relevant documents in the pool with the given relevance grade.
String[]	getRelevantDocumentsToArray(String queryid) Get the pooled relevant documents for a given query.
String[]	getRelevantDocumentsToArray(String queryid, int grade) Get the pooled relevant documents for a given query.
String[]	getRelevantDocumentsToArray(String queryid, int[] grades) Get the pooled relevant documents for the given query.
boolean	isRelevant(String queryid, String docno) Check if the given document is relevant for a given query.
protected void	loadQrelsFile() Load in memory the relevance assessment files that are specified in the array fqrels.
static String	parseTRECQueryNo(String queryid)
boolean	queryExistInQrels(String queryid) Checks whether there is a query with a given identifier in the relevance assessments.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.slf4j.Logger logger

qrelsPerQuery

public TRECQrelsInMemory.QrelsHashSet[] qrelsPerQuery

Each element in the array contains the docids of the relevant documents with respect to a query.

fqrels

protected File[] fqrels

An array with the qrels files.

totalNumberOfRelevantDocs

public int totalNumberOfRelevantDocs

The total number of relevant documents.

Constructor Detail

TRECQrelsInMemory

public TRECQrelsInMemory(String qrelsFilename)

A constructor that creates an instance of the class and loads in memory the relevance assessments from the given file.

Parameters:

qrelsFilename - String The full path of the qrels file to load.

TRECQrelsInMemory

public TRECQrelsInMemory(String[] qrelsFilenames)

A constructor that creates an instance of the class and loads in memory the relevance assessments from the given files.

Parameters:

qrelsFilenames - String[] The full path of the qrels files to load.

TRECQrelsInMemory

public TRECQrelsInMemory()

* A default constructor that creates an instance of the class and loads in memory the relevance assessments from the files that are specified in the file specified by the property TREC_QRELS.

Method Detail

getQueryids

public String[] getQueryids()

Get ids of the queries that appear in the pool.

Returns:

The ids of queries in the pool.

getRelevantDocuments

public String[] getRelevantDocuments(int[] grades)

Get the identifiers of all relevant documents with the given relevance grades.

Parameters:

grades - The relevance grades.

Returns:

The identifiers of all relevant documents with the given relevance grades.

getAllRelevantDocuments

public gnu.trove.THashSet<String> getAllRelevantDocuments()

Get the identifiers of all relevant documents in the pool.

Returns:

The identifiers of all relevant documents in the pool.

getRelevantDocuments

public gnu.trove.THashSet<String> getRelevantDocuments(int grade)

Get the identifiers of all relevant documents in the pool with a given relevance grade.

Returns:

The identifiers of all relevant documents in the pool.

getAllRelevantDocumentsToArray

public String[] getAllRelevantDocumentsToArray()

Get the identifiers of all relevant documents in the pool.

Returns:

The identifiers of all relevant documents in the pool.

getRelevantDocumentsToArray

public String[] getRelevantDocumentsToArray(int grade)

Get the identifiers of all relevant documents in the pool with the given relevance grade.

Returns:

The identifiers of all relevant documents in the pool.

getNonRelevantDocuments

public gnu.trove.THashSet<String> getNonRelevantDocuments(String queryid)

Get the pooled non-relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled non-relevant documents for the given query.

getNonRelevantDocuments

public gnu.trove.THashSet<String> getNonRelevantDocuments()

Get all the pooled non-relevant documents.

Returns:

All the pooled non-relevant documents.

getNonRelevantDocumentsToArray

public String[] getNonRelevantDocumentsToArray()

Get all the pooled non-relevant documents.

Returns:

All the pooled non-relevant documents.

getRelevantDocuments

public gnu.trove.THashSet<String> getRelevantDocuments(String queryid)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocuments

public gnu.trove.THashSet<String> getRelevantDocuments(String queryid,
                                                       int grade)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocuments

public gnu.trove.THashSet<String> getRelevantDocuments(String queryid,
                                                       int[] grades)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getRelevantDocumentsToArray

public String[] getRelevantDocumentsToArray(String queryid,
                                            int[] grades)

Get the pooled relevant documents for the given query.

Parameters:

queryid - The id of the given query.

Returns:

A hashset containing the docnos of the pooled relevant documents for the given query.

getNonRelevantDocumentsToArray

public String[] getNonRelevantDocumentsToArray(String queryid)

Get the pooled non-relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled non-relevant documents for the given query.

getRelevantDocumentsToArray

public String[] getRelevantDocumentsToArray(String queryid)

Get the pooled relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled relevant documents for the given query.

getRelevantDocumentsToArray

public String[] getRelevantDocumentsToArray(String queryid,
                                            int grade)

Get the pooled relevant documents for a given query.

Parameters:

queryid - The id of the given query.

Returns:

An array of the docnos of the pooled relevant documents for the given query.

getNumberOfQueries

public int getNumberOfQueries()

Returns the total number of queries contained in the loaded relevance assessments.

Returns:

int The number of unique queries in the loaded relevance assessments.

getNumberOfRelevant

public int getNumberOfRelevant(String queryid)

Returns the numbe of relevant documents for a given query.

Parameters:

queryid - String The identifier of a query.

Returns:

int The number of relevant documents for the given query.

loadQrelsFile

protected void loadQrelsFile()

Load in memory the relevance assessment files that are specified in the array fqrels.

parseTRECQueryNo

public static String parseTRECQueryNo(String queryid)

queryExistInQrels

public boolean queryExistInQrels(String queryid)

Checks whether there is a query with a given identifier in the relevance assessments.

Parameters:

queryid - String the identifier of a query.

Returns:

boolean true if the given query exists in the qrels file, otherwise it returns false.

isRelevant

public boolean isRelevant(String queryid,
                          String docno)

Check if the given document is relevant for a given query.

Parameters:

queryid - String a query identifier.

docno - String a document identifier.

Returns:

boolean true if the given document is relevant for the given query, or otherwise false.

getGrade

public int getGrade(String qid,
                    String docno,
                    int def)

Returns the grade of a document for a given query.

Parameters:

qid - String the identifier of the desired query.

docno - String a document identifier.

def - The default value to be returned if the document is not found.

Returns:

int The grade of the document for the given query.