org.terrier.evaluation
Class AdhocEvaluation

java.lang.Object
  org.terrier.evaluation.Evaluation
      org.terrier.evaluation.AdhocEvaluation

Direct Known Subclasses:

AdhocFullQueryEvaluation

public class AdhocEvaluation
extends Evaluation

Performs the evaluation for TREC's tasks, except the named page task. The evaluation measures include the mean average precision and other measures such as precision at 10, precision at 10%, and so on....

Author:

Gianni Amati, Ben He

Nested Class Summary

Nested classes/interfaces inherited from class org.terrier.evaluation.Evaluation

Evaluation.Record

Field Summary

protected double[]	averagePrecisionOfEachQuery The average precision of each query.
protected static org.apache.log4j.Logger	logger
protected int	maxNumberRetrieved The maximum number of documents retrieved for a query.
protected double	meanAveragePrecision Average Precision.
protected double	meanRelevantPrecision Relevant Precision.
protected int	numberOfEffQuery The number of effective queries.
protected static int[]	PRECISION_PERCENTAGES
protected static int[]	PRECISION_RANKS
protected gnu.trove.TIntDoubleHashMap	precisionAtRank Precision at rank number of documents
protected gnu.trove.TIntDoubleHashMap	precisionAtRecall
protected java.lang.String[]	queryNo The query number of each query.
protected int	totalNumberOfRelevant The total number of relevant documents in the qrels file for the queries processed in the task.
protected int	totalNumberOfRelevantRetrieved The total number of relevant documents retrieved in the task.
protected int	totalNumberOfRetrieved The total number of documents retrieved in the task.

Fields inherited from class org.terrier.evaluation.Evaluation

qrels

Constructor Summary

AdhocEvaluation()
Create adhoc evaluation

AdhocEvaluation(java.lang.String qrelsFile)
Create adhoc evaluation

AdhocEvaluation(java.lang.String[] qrelsFiles)
Create adhoc evaluation

Method Summary

void	evaluate(java.lang.String resultFilename) Evaluates the given result file.
void	initialise() Initialise variables.
void	writeEvaluationResult(java.io.PrintWriter out) Output the evaluation result to the specific file.
void	writeEvaluationResultOfEachQuery(java.lang.String resultEvalFilename) Output the evaluation result of each query to the specific file.

Methods inherited from class org.terrier.evaluation.Evaluation

writeEvaluationResult, writeEvaluationResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

PRECISION_RANKS

protected static final int[] PRECISION_RANKS

PRECISION_PERCENTAGES

protected static final int[] PRECISION_PERCENTAGES

maxNumberRetrieved

protected int maxNumberRetrieved

The maximum number of documents retrieved for a query.

numberOfEffQuery

protected int numberOfEffQuery

The number of effective queries. An effective query is a query that has corresponding relevant documents in the qrels file.

totalNumberOfRetrieved

protected int totalNumberOfRetrieved

The total number of documents retrieved in the task.

totalNumberOfRelevant

protected int totalNumberOfRelevant

The total number of relevant documents in the qrels file for the queries processed in the task.

totalNumberOfRelevantRetrieved

protected int totalNumberOfRelevantRetrieved

The total number of relevant documents retrieved in the task.

precisionAtRank

protected gnu.trove.TIntDoubleHashMap precisionAtRank

Precision at rank number of documents

precisionAtRecall

protected gnu.trove.TIntDoubleHashMap precisionAtRecall

meanAveragePrecision

protected double meanAveragePrecision

Average Precision.

meanRelevantPrecision

protected double meanRelevantPrecision

Relevant Precision.

averagePrecisionOfEachQuery

protected double[] averagePrecisionOfEachQuery

The average precision of each query.

queryNo

protected java.lang.String[] queryNo

The query number of each query.

Constructor Detail

AdhocEvaluation

public AdhocEvaluation()

Create adhoc evaluation

AdhocEvaluation

public AdhocEvaluation(java.lang.String qrelsFile)

Create adhoc evaluation

Parameters:

qrelsFile -

AdhocEvaluation

public AdhocEvaluation(java.lang.String[] qrelsFiles)

Create adhoc evaluation

Parameters:

qrelsFiles -

Method Detail

initialise

public void initialise()

Initialise variables.

evaluate

public void evaluate(java.lang.String resultFilename)

Evaluates the given result file.

Specified by:

evaluate in class Evaluation

Parameters:

resultFilename - String the filename of the result file to evaluate.

writeEvaluationResultOfEachQuery

public void writeEvaluationResultOfEachQuery(java.lang.String resultEvalFilename)

Output the evaluation result of each query to the specific file.

Specified by:

writeEvaluationResultOfEachQuery in class Evaluation

Parameters:

resultEvalFilename - String the name of the file in which to save the evaluation results.

writeEvaluationResult

public void writeEvaluationResult(java.io.PrintWriter out)

Output the evaluation result to the specific file.

Specified by:

writeEvaluationResult in class Evaluation

Parameters:

out - java.io.PrintWriter the stream to which the results are printed.