Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-8

Delay index path checking during indexing till HDFS is loaded

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.1
    • Component/s: None
    • Labels:
      None

      Description

      TR-7 gave the Hadoop file system plugin to Terrier the ability to check that files exist. However, when using HOD and HadoopIndexing, indexing would fail in TrecTerrier stating that the index did not exit. This is because, in TrecTerrier, HOD has not been called, and hence the HDFS is not yet available.

        Attachments

          Activity

          Hide
          craigm Craig Macdonald added a comment -

          This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.

          Show
          craigm Craig Macdonald added a comment - This patch remove index checking from TrecTerrier. Instead, HadoopIndexing and TRECIndexing should check the index path themselves. This patch provides such functionality.
          Hide
          craigm Craig Macdonald added a comment -

          Committed.

          Show
          craigm Craig Macdonald added a comment - Committed.

            People

            • Assignee:
              craigm Craig Macdonald
              Reporter:
              craigm Craig Macdonald
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: