Terrier Users :  Terrier Forum terrier.org
General discussion about using/developing applications using Terrier 
Why all Precision is 0.0000 but retrieved results is not zero?
Posted by: serina ()
Date: December 29, 2017 08:13AM

Hello terrier team and terrier users,
I was indexing the CACM collection with terrier 4.1,
When I evaluate this index with qrels file, all of precision was 0.0000
For example:
Number of queries = 52
Retrieved = 47873
Relevant = 796
Relevant retrieved = 0
____________________________________
Average Precision: 0.0000
R Precision : 0.0000
____________________________________
Precision at 1 : 0.0000
Precision at 2 : 0.0000
Precision at 3 : 0.0000
Precision at 4 : 0.0000
Precision at 5 : 0.0000
Precision at 10 : 0.0000
Precision at 15 : 0.0000
Precision at 20 : 0.0000
Precision at 30 : 0.0000
Precision at 50 : 0.0000
Precision at 100 : 0.0000
Precision at 200 : 0.0000
Precision at 500 : 0.0000
Precision at 1000 : 0.0000



But when I Retrieval this index the result show very document with high score is retrieved.
Here a part of result of Retrieval:

1 Q0 1938 0 9.375360262342909 TF_IDF
1 Q0 2371 1 8.541978912092546 TF_IDF
1 Q0 1071 2 8.188493872644084 TF_IDF
1 Q0 1410 3 8.092355889480462 TF_IDF
1 Q0 2535 4 7.534875454756727 TF_IDF
1 Q0 2036 5 7.486214306987502 TF_IDF
1 Q0 1391 6 7.265419796783676 TF_IDF
1 Q0 1571 7 7.15587774862721 TF_IDF
1 Q0 1605 8 7.115016673394125 TF_IDF
1 Q0 2151 9 7.101664548739161 TF_IDF
1 Q0 2621 10 7.01294899145803 TF_IDF
1 Q0 1827 11 6.980662303797203 TF_IDF
1 Q0 1506 12 6.931377845386175 TF_IDF
1 Q0 1657 13 6.854031467007401 TF_IDF
1 Q0 1844 14 6.8379775545814505 TF_IDF
1 Q0 322 15 6.802840528717799 TF_IDF
1 Q0 2950 16 6.751785168031093 TF_IDF
1 Q0 3023 17 6.695282927944731 TF_IDF
1 Q0 2522 18 6.673598552301519 TF_IDF
1 Q0 1572 19 6.626633682430349 TF_IDF
1 Q0 1680 20 6.606058353114983 TF_IDF
1 Q0 2951 21 6.602278258031578 TF_IDF
1 Q0 2357 22 6.533407046142173 TF_IDF
1 Q0 971 23 6.455705036366826 TF_IDF
1 Q0 2948 24 6.394563633094594 TF_IDF


My terrier.properties file is:

#default controls for query expansion
querying.postprocesses.order=QueryExpansion
querying.postprocesses.controls=qe:QueryExpansion
#default controls for the web-based interface. SimpleDecorate
#is the simplest metadata decorator. For more control, see Decorate.
querying.postfilters.order=SimpleDecorate,SiteFilter,Scope
querying.postfilters.controls=decorateconfused smileyimpleDecorate,siteconfused smileyiteFilter,scopeconfused smileycope

#default and allowed controls
querying.default.controls=
querying.allowed.controls=scope,qe,qemodel,start,end,site,scope

#document tags specification
#for processing the contents of
#the documents, ignoring DOCHDR

#query tags specification
TrecQueryTags.doctag=DOC
TrecQueryTags.idtag=DOCNO
TrecQueryTags.process=DOC,DOCNO
TrecQueryTags.skip=


#stop-words file
stopwords.filename=stopword-list.txt

#the processing stages a term goes through
termpipelines=Stopwords,PorterStemmer

trec.collection.class=SimpleFileCollection
indexing.simplefilecollection.extensionsparsers=txt
indexing.simplefilecollection.defaultparser=FileDocument
indexer.meta.forward.keylens=100

trec.topics=C:/Users/Serina/Collection/cacm.query
trec.model=TF_IDF

expansion.documents = 5
expansion.terms = 250

trec.qrels=C:/Users/Serina/Collection/cacm.rel


Please help me, thank you very much.
Thank you so much for your help to a newbie like me!

Options: ReplyQuote
Re: Why all Precision is 0.0000 but retrieved results is not zero?
Posted by: craigm ()
Date: December 29, 2017 03:08PM

Think you had a typo in your question about "very document"
Can you manually check that a relevant is retrieved?

Craig

Options: ReplyQuote
Re: Why all Precision is 0.0000 but retrieved results is not zero?
Posted by: serina ()
Date: December 29, 2017 10:54PM

Hello sir.
Yes please look at part of my judgment file:

1 Q0 CACM-1410 1
1 Q0 CACM-1572 1
1 Q0 CACM-1605 1
1 Q0 CACM-2020 1
1 Q0 CACM-2358 1
2 Q0 CACM-2434 1
2 Q0 CACM-2863 1
2 Q0 CACM-3078 1
3 Q0 CACM-1134 1
3 Q0 CACM-1613 1
3 Q0 CACM-1807 1
3 Q0 CACM-1947 1

For question 1 in this file 5 document is better that from that part of retrieved file in above: doc 1410 in score 3,and doc 1572 in scor 19.
I find all other doc in retrieved result.
Please help me.

Options: ReplyQuote
Re: Why all Precision is 0.0000 but retrieved results is not zero?
Posted by: serina ()
Date: December 31, 2017 07:42AM

hello terrier users,

my problem is solved.
I modify my qrels file.

thanks very much.

Options: ReplyQuote


Sorry, only registered users may post in this forum.
This forum powered by Phorum.