Package org.terrier.evaluation

Class NamedPageEvaluation

java.lang.Object

org.terrier.evaluation.TerrierEvaluation

org.terrier.evaluation.NamedPageEvaluation

All Implemented Interfaces:

Evaluation

public class NamedPageEvaluation
extends TerrierEvaluation

Performs the evaluation for TREC's named/home page finding task. The evaluation measure is the average reciprocal rank of those relevant documents that are among the top 50 documents.

Author:

Vassilis Plachouras

Nested Class Summary

Nested classes/interfaces inherited from class org.terrier.evaluation.TerrierEvaluation

TerrierEvaluation.Record

Field Summary

Fields

Modifier and Type	Field	Description
protected double	arr	The average reciprocal rank.
protected int	inTop1	The number of queries where the answer is found in the top first rank.
protected int	inTop10	The number of queries where the answer is found in the top 10 docs.
protected int	inTop20	The number of queries for which the answer is found in the top 20 docs.
protected int	inTop5	The number of queries where the answer is found in the top 5 docs.
protected int	inTop50	The number of queries where the answer is found in the top 50 docs.
protected int	notInTop50	The number of queries where the answer was not found in the top 50 documents.
protected int	numberOfQueries	total number of queries.
protected gnu.trove.TIntHashSet	queryNumbers	The query numbers' vector.
protected gnu.trove.TIntIntHashMap	rankRelevantDocument	the rank of the correct answer for each query evaluated.
protected gnu.trove.TIntDoubleHashMap	recipRank	The reciprocal rank of each query.

Fields inherited from class org.terrier.evaluation.TerrierEvaluation

logger, qrels

Constructor Summary

Constructors

Constructor	Description
NamedPageEvaluation()

Method Summary

Modifier and Type	Method	Description
void	evaluate(java.lang.String resultFilename)	Evaluates the given result file.
void	writeEvaluationResult(java.io.PrintWriter out)	Output the evaluation result to the specific file.
void	writeEvaluationResultOfEachQuery(java.lang.String resultEvalFilename)	Output the evaluation result of each query to the specific file.

Methods inherited from class org.terrier.evaluation.TerrierEvaluation

writeEvaluationResult, writeEvaluationResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

arr

protected double arr

The average reciprocal rank.

numberOfQueries

protected int numberOfQueries

total number of queries.

rankRelevantDocument

protected gnu.trove.TIntIntHashMap rankRelevantDocument

the rank of the correct answer for each query evaluated.

inTop50

protected int inTop50

The number of queries where the answer is found in the top 50 docs.

inTop20

protected int inTop20

The number of queries for which the answer is found in the top 20 docs.

inTop10

protected int inTop10

The number of queries where the answer is found in the top 10 docs.

inTop5

protected int inTop5

The number of queries where the answer is found in the top 5 docs.

inTop1

protected int inTop1

The number of queries where the answer is found in the top first rank.

notInTop50

protected int notInTop50

The number of queries where the answer was not found in the top 50 documents.

queryNumbers

protected gnu.trove.TIntHashSet queryNumbers

The query numbers' vector.

recipRank

protected gnu.trove.TIntDoubleHashMap recipRank

The reciprocal rank of each query.

Constructor Detail

NamedPageEvaluation

public NamedPageEvaluation()

Method Detail

evaluate

public void evaluate(java.lang.String resultFilename)

Evaluates the given result file.

Specified by:

evaluate in interface Evaluation

Specified by:

evaluate in class TerrierEvaluation

Parameters:

resultFilename - String the filename of the result file to evaluate.

writeEvaluationResultOfEachQuery

public void writeEvaluationResultOfEachQuery(java.lang.String resultEvalFilename)

Output the evaluation result of each query to the specific file.

Specified by:

writeEvaluationResultOfEachQuery in interface Evaluation

Specified by:

writeEvaluationResultOfEachQuery in class TerrierEvaluation

Parameters:

resultEvalFilename - String the name of the file in which to save the evaluation results.

writeEvaluationResult

public void writeEvaluationResult(java.io.PrintWriter out)

Output the evaluation result to the specific file.

Specified by:

writeEvaluationResult in interface Evaluation

Specified by:

writeEvaluationResult in class TerrierEvaluation

Parameters:

out - PrintWriter the name of the stream to output the result.