Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-78

BitInputFormat: some minor changes

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 3.0
    • Fix Version/s: 3.0
    • Component/s: .structures
    • Labels:
      None

      Description

      Two corner issues:
      1. End splits of less than one byte were not processed
      2. Empty entries were not processed correctly.

      This code is used by Inv2DirectMultiReduce. However, (1) is an unlikely case, as our inverted files are so massive, the chances of the final split being less than 1 byte as very small. (2) cannot happen for an inverted file - terms always have entries.

        Attachments

          Activity

          craigm Craig Macdonald created issue -
          Hide
          craigm Craig Macdonald added a comment -

          Formatting change

          Show
          craigm Craig Macdonald added a comment - Formatting change
          craigm Craig Macdonald made changes -
          Field Original Value New Value
          Description Two corner issues:
          1. End splits of less than one byte were not processed
           * Empty entries were not processed correctly.

          This code is used by Inv2DirectMultiReduce. However, (1) is an unlikely case, as our inverted files are so massive, the chances of the final split being less than 1 byte as very small. (2) cannot happen for an inverted file - terms always have entries.
          Two corner issues:
          1. End splits of less than one byte were not processed
          2. Empty entries were not processed correctly.

          This code is used by Inv2DirectMultiReduce. However, (1) is an unlikely case, as our inverted files are so massive, the chances of the final split being less than 1 byte as very small. (2) cannot happen for an inverted file - terms always have entries.
          Hide
          craigm Craig Macdonald added a comment -

          Committed to trunk

          Show
          craigm Craig Macdonald added a comment - Committed to trunk
          craigm Craig Macdonald made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Assignee Iadh Ounis [ ounis ] Craig Macdonald [ craigm ]
          Hide
          craigm Craig Macdonald added a comment -

          Test case was committed to HadoopShakespeareEndToEndTest for testing splitting of a direct index.

          Show
          craigm Craig Macdonald added a comment - Test case was committed to HadoopShakespeareEndToEndTest for testing splitting of a direct index.
          craigm Craig Macdonald made changes -
          Affects Version/s 3.0 [ 10020 ]
          Fix Version/s 3.0 [ 10020 ]
          craigm Craig Macdonald made changes -
          Project TREC [ 10010 ] Terrier Core [ 10000 ]
          Key TREC-110 TR-78
          Workflow jira [ 10223 ] Terrier Open Source [ 10336 ]
          Affects Version/s 3.0 [ 10030 ]
          Affects Version/s 3.0 [ 10020 ]
          Component/s .structures [ 10007 ]
          Component/s Core [ 10020 ]
          Fix Version/s 3.0 [ 10030 ]
          Fix Version/s 3.0 [ 10020 ]

            People

            • Assignee:
              craigm Craig Macdonald
              Reporter:
              craigm Craig Macdonald
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: