Terrier Users :  Terrier Forum terrier.org
General discussion about using/developing applications using Terrier 
How to access original query terms?
Posted by: Nader ()
Date: November 21, 2017 07:23PM

Hello,
I'm trying to modify and debug the DFRBagExpansionTerms.java file and when I try to access the hashmap originalTermids it is empty (it has a size of zero). Any idea why this is so?

Thanks!

Options: ReplyQuote
Re: How to access original query terms?
Posted by: craigm ()
Date: November 23, 2017 12:50PM

Not sure. originalTermIds is set using expansionTerms.setOriginalQueryTErms() in QueryExpansion.java.

[github.com]

Can you check the query has EntryStatistics attached for each term? Some debugging required.

Craig

Options: ReplyQuote
Re: How to access original query terms?
Posted by: Nader ()
Date: November 27, 2017 11:15PM

Hello Craig,
After doing some debugging, here is what I think is the problem:

In the ExpansionTerms.java file, line 82
github line 82 : the object te of class EntryStatistics is always assigned a value of null. Thus the if statement in the next line(83) is never evaluated to true and therefore the originalTermids don't get assigned any value.

Now the reason that the EntryStatistics object has a null value is because when the properties of a term is initialized at line 337 ( github line 337 ) of MatchingQueryTerms.java with the QueryTermProperties constructors in lines 92-95 ( github line 92 ) of MatchingQueryTerms.java, it doesn't assign any value to the stats variable. Therefore, it will have a value of null by default.

If you think this is a bug, what would you suggest to resolve it?

Many thanks for your time!

Options: ReplyQuote
Re: How to access original query terms?
Posted by: craigm ()
Date: November 29, 2017 12:11AM

Hi,

Yes, I'll need to look into this. Matching should "decorate" the MatchingQueryTerms such that the entryStatistics are not present for each term.

Craig

Options: ReplyQuote
Re: How to access original query terms?
Posted by: Nader ()
Date: November 29, 2017 04:14PM

Good I look forward to what will you find out.

Thanks!

Options: ReplyQuote
Re: How to access original query terms?
Posted by: craigm ()
Date: November 29, 2017 05:59PM

This is a regression caused by the PostingListManager. I have a patch that I'm testing, but I want to know what the impact on effectiveness is.

You should be able to workaround by setting:
-Dtrec.matching=org.terrier.matching.daat.FullNoPLM

Hope this helps

Craig

Options: ReplyQuote
Re: How to access original query terms?
Posted by: Slattery ()
Date: December 06, 2017 03:32PM

Hi Craig, when do you think the patch will be out? Are you certain that it will fix this?

Options: ReplyQuote
Re: How to access original query terms?
Posted by: craigm ()
Date: December 07, 2017 06:12PM

Did you try the workaround? It should work fine.

Craig

Options: ReplyQuote
Re: How to access original query terms?
Posted by: craigm ()
Date: December 07, 2017 07:02PM

See

[terrier.org]

Works fine for me.

Craig

Options: ReplyQuote


Sorry, only registered users may post in this forum.
This forum powered by Phorum.