Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-235

LexiconBuilder fails on empty term

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.5
    • Fix Version/s: 3.6
    • Component/s: .indexing
    • Labels:
      None

      Description

      INFO - Collection #0 took 55 seconds to build the runs for 1666 documents

      INFO - Key docno values are sorted in meta index, consider binary searching zdat
      a file
      INFO - Merging 1 runs...
      INFO - Collection #0 took 0 seconds to merge

      INFO - Collection #0 total time 55
      INFO - Optimising structure lexicon
      INFO - Optimsing lexicon with 68611 entries
      A problem occurred: java.nio.BufferUnderflowException
      java.nio.BufferUnderflowException
              at java.nio.Buffer.nextGetIndex(Unknown Source)
              at java.nio.HeapByteBuffer.get(Unknown Source)
              at org.apache.hadoop.io.Text.bytesToCodePoint(Text.java:536)
              at org.apache.hadoop.io.Text.charAt(Text.java:121)
              at org.terrier.structures.FSOMapFileLexicon.optimise(FSOMapFileLexicon.j
      ava:528)
              at org.terrier.structures.FSOMapFileLexicon.optimise(FSOMapFileLexicon.j
      ava:473)
              at org.terrier.structures.indexing.LexiconBuilder.optimise(LexiconBuilde
      r.java:830)
              at org.terrier.indexing.BasicIndexer.finishedInvertedIndexBuild(BasicInd
      exer.java:449)
              at org.terrier.indexing.BasicSinglePassIndexer.createInvertedIndex(Basic
      SinglePassIndexer.java:302)
              at org.terrier.indexing.BasicSinglePassIndexer.createDirectIndex(BasicSi
      nglePassIndexer.java:155)
              at org.terrier.indexing.Indexer.index(Indexer.java:346)
              at org.terrier.applications.TRECIndexing.createSinglePass(TRECIndexing.j
      ava:220)
              at org.terrier.applications.TrecTerrier.run(TrecTerrier.java:382)
              at org.terrier.applications.TrecTerrier.applyOptions(TrecTerrier.java:56
      4)
              at org.terrier.applications.TrecTerrier.main(TrecTerrier.java:235)

      C:\terrier-3.5\bin>

        Attachments

          Activity

          mhidy Abdelkader EL MAHDAOUY created issue -
          mhidy Abdelkader EL MAHDAOUY made changes -
          Field Original Value New Value
          Status Open [ 1 ] Patch Available [ 10000 ]
          mhidy Abdelkader EL MAHDAOUY made changes -
          Status Patch Available [ 10000 ] Open [ 1 ]
          craigm Craig Macdonald made changes -
          Fix Version/s 3.6 [ 10060 ]
          richardm Richard McCreadie made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          richardm Richard McCreadie made changes -
          Summary Optimsing lexicon with 68611 entries : A problem occurred: java.nio.BufferUnderflowException LexiconBuilder fails on empty term

            People

            • Assignee:
              craigm Craig Macdonald
              Reporter:
              mhidy Abdelkader EL MAHDAOUY
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: