Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-324

Evaluation fails to parse .res files

          Details

          • Type: Bug
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 4.0
          • Fix Version/s: 4.1
          • Component/s: .evaluation
          • Labels:
            None

            Description

            This is running the quick-start example, except over an index of TREC-8 adhoc:

            % ./bin/trec_terrier.sh -r -Dtrec.model=PL2 -c 10.99 -Dtrec.topics=./cd45-cr/topics.401-450
            ...
            INFO - Finished topics, executed 50 queries in 1.393 seconds, results written to /Users/soboroff/terrier-4.0/var/results/PL2c10.99_0.res

            % ./bin/trec_terrier.sh -e -Dtrec.qrels=./cd45-cr/adhoc.qrels
            Setting TERRIER_HOME to /Users/soboroff/terrier-4.0
            INFO - Evaluating result file: /Users/soboroff/terrier-4.0/var/results/PL2c10.99_0.res
            A problem occurred: java.lang.NumberFormatException: For input string: "10.91742797093212"
            java.lang.NumberFormatException: For input string: "10.91742797093212"
            at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
            at java.lang.Integer.parseInt(Integer.java:492)
            at java.lang.Integer.parseInt(Integer.java:527)
            at org.terrier.evaluation.AdhocEvaluation.evaluate(AdhocEvaluation.java:175)
            at org.terrier.applications.TrecTerrier.run(TrecTerrier.java:525)
            at org.terrier.applications.TrecTerrier.applyOptions(TrecTerrier.java:588)
            at org.terrier.applications.TrecTerrier.main(TrecTerrier.java:245)

              Attachments

                Activity

                Descending order - Click to sort in ascending order
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                Fixed 730a83ec

                Show
                richardm Richard McCreadie added a comment - Fixed 730a83ec
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                Patch to add diagnostic DOCNOs.

                Show
                richardm Richard McCreadie added a comment - Patch to add diagnostic DOCNOs.
                Hide
                Permalink
                isoboroff Ian Soboroff added a comment -

                I'll try to cook up a patch time permitting.

                Show
                isoboroff Ian Soboroff added a comment - I'll try to cook up a patch time permitting.
                Hide
                Permalink
                isoboroff Ian Soboroff added a comment -

                Yes, indexing some non-TREC docs in the collection.spec was the initial cause, but maybe when trying to write out TREC runfiles, if a docno is absent something suitably diagnostic could be output in that field instead?

                Show
                isoboroff Ian Soboroff added a comment - Yes, indexing some non-TREC docs in the collection.spec was the initial cause, but maybe when trying to write out TREC runfiles, if a docno is absent something suitably diagnostic could be output in that field instead?
                Hide
                Permalink
                craigm Craig Macdonald added a comment -

                I don't suppose you know which document didnt had an empty DOCNO tag?

                Perhaps you listed a readme file in the collection.spec file that got indexed?

                Use bin/trec_terrier.sh --printmeta to see the docnos contained in the index in order.

                Thanks

                Craig

                Show
                craigm Craig Macdonald added a comment - I don't suppose you know which document didnt had an empty DOCNO tag? Perhaps you listed a readme file in the collection.spec file that got indexed? Use bin/trec_terrier.sh --printmeta to see the docnos contained in the index in order. Thanks Craig
                Hide
                Permalink
                isoboroff Ian Soboroff added a comment -

                It is unhappy because the .res line in question is missing a docid.

                Show
                isoboroff Ian Soboroff added a comment - It is unhappy because the .res line in question is missing a docid.
                Hide
                Permalink
                isoboroff Ian Soboroff added a comment -

                Oops, attached an extra file for another run I was trying.

                Show
                isoboroff Ian Soboroff added a comment - Oops, attached an extra file for another run I was trying.

                  People

                  • Assignee:
                    richardm Richard McCreadie
                    Reporter:
                    isoboroff Ian Soboroff
                  • Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: