Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-324

Evaluation fails to parse .res files

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0
    • Fix Version/s: 4.1
    • Component/s: .evaluation
    • Labels:
      None

      Description

      This is running the quick-start example, except over an index of TREC-8 adhoc:

      % ./bin/trec_terrier.sh -r -Dtrec.model=PL2 -c 10.99 -Dtrec.topics=./cd45-cr/topics.401-450
      ...
      INFO - Finished topics, executed 50 queries in 1.393 seconds, results written to /Users/soboroff/terrier-4.0/var/results/PL2c10.99_0.res

      % ./bin/trec_terrier.sh -e -Dtrec.qrels=./cd45-cr/adhoc.qrels
      Setting TERRIER_HOME to /Users/soboroff/terrier-4.0
      INFO - Evaluating result file: /Users/soboroff/terrier-4.0/var/results/PL2c10.99_0.res
      A problem occurred: java.lang.NumberFormatException: For input string: "10.91742797093212"
      java.lang.NumberFormatException: For input string: "10.91742797093212"
      at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
      at java.lang.Integer.parseInt(Integer.java:492)
      at java.lang.Integer.parseInt(Integer.java:527)
      at org.terrier.evaluation.AdhocEvaluation.evaluate(AdhocEvaluation.java:175)
      at org.terrier.applications.TrecTerrier.run(TrecTerrier.java:525)
      at org.terrier.applications.TrecTerrier.applyOptions(TrecTerrier.java:588)
      at org.terrier.applications.TrecTerrier.main(TrecTerrier.java:245)

        Attachments

          Activity

          isoboroff Ian Soboroff created issue -
          Hide
          isoboroff Ian Soboroff added a comment -

          Oops, attached an extra file for another run I was trying.

          Show
          isoboroff Ian Soboroff added a comment - Oops, attached an extra file for another run I was trying.
          Hide
          isoboroff Ian Soboroff added a comment -

          It is unhappy because the .res line in question is missing a docid.

          Show
          isoboroff Ian Soboroff added a comment - It is unhappy because the .res line in question is missing a docid.
          Hide
          craigm Craig Macdonald added a comment -

          I don't suppose you know which document didnt had an empty DOCNO tag?

          Perhaps you listed a readme file in the collection.spec file that got indexed?

          Use bin/trec_terrier.sh --printmeta to see the docnos contained in the index in order.

          Thanks

          Craig

          Show
          craigm Craig Macdonald added a comment - I don't suppose you know which document didnt had an empty DOCNO tag? Perhaps you listed a readme file in the collection.spec file that got indexed? Use bin/trec_terrier.sh --printmeta to see the docnos contained in the index in order. Thanks Craig
          Hide
          isoboroff Ian Soboroff added a comment -

          Yes, indexing some non-TREC docs in the collection.spec was the initial cause, but maybe when trying to write out TREC runfiles, if a docno is absent something suitably diagnostic could be output in that field instead?

          Show
          isoboroff Ian Soboroff added a comment - Yes, indexing some non-TREC docs in the collection.spec was the initial cause, but maybe when trying to write out TREC runfiles, if a docno is absent something suitably diagnostic could be output in that field instead?
          Hide
          isoboroff Ian Soboroff added a comment -

          I'll try to cook up a patch time permitting.

          Show
          isoboroff Ian Soboroff added a comment - I'll try to cook up a patch time permitting.
          craigm Craig Macdonald made changes -
          Field Original Value New Value
          Fix Version/s 4.1 [ 10070 ]
          craigm Craig Macdonald made changes -
          Assignee Craig Macdonald [ craigm ] Richard McCreadie [ richardm ]
          Hide
          richardm Richard McCreadie added a comment -

          Patch to add diagnostic DOCNOs.

          Show
          richardm Richard McCreadie added a comment - Patch to add diagnostic DOCNOs.
          richardm Richard McCreadie made changes -
          Attachment Terrier324.patch [ 10452 ]
          Hide
          richardm Richard McCreadie added a comment -

          Fixed 730a83ec

          Show
          richardm Richard McCreadie added a comment - Fixed 730a83ec
          richardm Richard McCreadie made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              richardm Richard McCreadie
              Reporter:
              isoboroff Ian Soboroff
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: