[TR-59] Reset problem in Terrier evaluation tool Created: 10/Sep/09  Updated: 05/Mar/10  Resolved: 16/Sep/09

Status: Resolved
Project: Terrier Core
Component/s: None
Affects Version/s: 3.0
Fix Version/s: 3.0

Type: Bug Priority: Major
Reporter: Iadh Ounis Assignee: Craig Macdonald
Resolution: Fixed  
Labels: None

Nicola reported today a bug with the Terrier evaluation tool. For some queries, the system reports an MAP of 2 (yes 2) at precision @1. It appears that it is a resetting problem when multiple evaluation files are used.

Comment by Craig Macdonald [ 16/Sep/09 ]

Not a reset problem actually. A problem calculating precision at rank.

if (relevantRetrieved[j].rank <= precisionRank)

should read

if (relevantRetrieved[j].rank < precisionRank)

because ranks are 0 based.

I have also added test cases for the AdhocEvaluation class. The test cases check various measures (hand calculated) for rankings upto size 4. Without the patch, the test fails. With it, the test passes.

Comment by Iadh Ounis [ 16/Sep/09 ]

Well spotted Craig!

Generated at Sat Aug 15 11:06:46 BST 2020 using JIRA 7.1.1#71004-sha1:d6b2c0d9b7051e9fb5e4eb8ce177ca56d91d7bd8.