Inconsistent scores using DirichletLM between Terrier 3.5 and 4.2
Date: September 05, 2017 04:28PM
I'm trying to upgrade to the latest version of Terrier and facing some difficulties.
I'm using the DirichletLM weighting model and getting inconsistent scoring of documents between Terrier versions 3.5 and 4.2. After debugging for some time I found the problem was actually in indexing. Specifically in the method createDirectIndex of file BlockIndexer.java
Here is the line where they differ at:
In Terrier 4.2:
new LexiconBuilder(currentIndex, "lexicon", new LexiconMap(), BasicLexiconEntry.class.getName(), termCodes);
In Terrier 3.5:
new LexiconBuilder(currentIndex, "lexicon", new BlockLexiconMap(), BlockLexiconEntry.class.getName());
Specifically, in the second and third arguments of call to the constructor.
Any thoughts about this issue?