org.terrier.evaluation

Class NamedPageEvaluation

java.lang.Object

org.terrier.evaluation.Evaluation

org.terrier.evaluation.NamedPageEvaluation

public class NamedPageEvaluation
extends Evaluation

Performs the evaluation for TREC's named/home page finding task. The evaluation measure is the average reciprocal rank of those relevant documents that are among the top 50 documents.

Author:

Vassilis Plachouras

Nested Class Summary

Nested classes/interfaces inherited from class org.terrier.evaluation.Evaluation

Evaluation.Record

Field Summary

Fields

Modifier and Type	Field and Description
protected double	arr The average reciprocal rank.
protected int	inTop1 The number of queries where the answer is found in the top first rank.
protected int	inTop10 The number of queries where the answer is found in the top 10 docs.
protected int	inTop20 The number of queries for which the answer is found in the top 20 docs.
protected int	inTop5 The number of queries where the answer is found in the top 5 docs.
protected int	inTop50 The number of queries where the answer is found in the top 50 docs.
protected int	notInTop50 The number of queries where the answer was not found in the top 50 documents.
protected int	numberOfQueries total number of queries.
protected gnu.trove.TIntHashSet	queryNumbers The query numbers' vector.
protected gnu.trove.TIntIntHashMap	rankRelevantDocument the rank of the correct answer for each query evaluated.
protected gnu.trove.TIntDoubleHashMap	recipRank The reciprocal rank of each query.

Fields inherited from class org.terrier.evaluation.Evaluation

logger, qrels

Constructor Summary

Constructors

Constructor and Description
NamedPageEvaluation()

Method Summary

Methods

Modifier and Type	Method and Description
void	evaluate(String resultFilename) Evaluates the given result file.
void	writeEvaluationResult(PrintWriter out) Output the evaluation result to the specific file.
void	writeEvaluationResultOfEachQuery(String resultEvalFilename) Output the evaluation result of each query to the specific file.

Methods inherited from class org.terrier.evaluation.Evaluation

writeEvaluationResult, writeEvaluationResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

arr

protected double arr

The average reciprocal rank.

numberOfQueries

protected int numberOfQueries

total number of queries.

rankRelevantDocument

protected gnu.trove.TIntIntHashMap rankRelevantDocument

the rank of the correct answer for each query evaluated.

inTop50

protected int inTop50

The number of queries where the answer is found in the top 50 docs.

inTop20

protected int inTop20

The number of queries for which the answer is found in the top 20 docs.

inTop10

protected int inTop10

The number of queries where the answer is found in the top 10 docs.

inTop5

protected int inTop5

The number of queries where the answer is found in the top 5 docs.

inTop1

protected int inTop1

The number of queries where the answer is found in the top first rank.

notInTop50

protected int notInTop50

The number of queries where the answer was not found in the top 50 documents.

queryNumbers

protected gnu.trove.TIntHashSet queryNumbers

The query numbers' vector.

recipRank

protected gnu.trove.TIntDoubleHashMap recipRank

The reciprocal rank of each query.

Constructor Detail

NamedPageEvaluation

public NamedPageEvaluation()

Method Detail

evaluate

public void evaluate(String resultFilename)

Evaluates the given result file.

Specified by:

evaluate in class Evaluation

Parameters:

resultFilename - String the filename of the result file to evaluate.

writeEvaluationResultOfEachQuery

public void writeEvaluationResultOfEachQuery(String resultEvalFilename)

Output the evaluation result of each query to the specific file.

Specified by:

writeEvaluationResultOfEachQuery in class Evaluation

Parameters:

resultEvalFilename - String the name of the file in which to save the evaluation results.

writeEvaluationResult

public void writeEvaluationResult(PrintWriter out)

Output the evaluation result to the specific file.

Specified by:

writeEvaluationResult in class Evaluation

Parameters:

out - PrintWriter the name of the stream to output the result.