Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-137

TRECCollection cannot add properties from the document tags to the meta index at indexing time

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0
    • Fix Version/s: 3.5
    • Component/s: .indexing
    • Labels:
      None

      Description

      Modified the TRECCollection class such that additional document tags like the URL can be saved automatically into the meta index. Initial patch provided.

      Example: I want to index the title field and add the url to the meta index for the collection below:
      <doc>
      <docno>NYTMay2006-0</docno>
      <url>http://events.nytimes.com/2006/05/03/dining/reviews/03rest.html&lt;/url>
      <title>Forum: Dine Out</title>
      </doc>

      Set trec.properties:
      # Which fields do we want to pass to the indexer as properties (in addition to the docno)?
      TrecDocTags.propertytags=url
      # At the indexer which keys to we add to the Meta index?
      indexer.meta.forward.keys=docno,url
      # What are the lengths of those keys (e.g. the url will be much longer)?
      indexer.meta.forward.keylens=20,100

        Attachments

          Activity

          richardm Richard McCreadie created issue -
          richardm Richard McCreadie made changes -
          Field Original Value New Value
          Assignee Iadh Ounis [ ounis ] Richard McCreadie [ richardm ]
          richardm Richard McCreadie made changes -
          Description Modified the TRECCollection class such that additional document tags like the URL can be saved automatically into the meta index. Initial patch provided.

          Example: I want to index the title field and add the url to the meta index for the collection below:
          <doc>
          <docno>NYTMay2006-0</docno>
          <url>http://events.nytimes.com/2006/05/03/dining/reviews/03rest.html&lt;/url>
          <title>Forum: Dine Out</title>
          </doc>

          Set trec.properties:
          # Which fields do we want to pass to the indexer as properties (in addition to the docno)?
          TrecDocTags.propertytags=url
          # At the indexer which keys to we add to the Meta index?
          indexer.meta.forward.keys=docno,desc
          # What are the lengths of those keys (e.g. the url will be much longer)?
          indexer.meta.forward.keylens=20,100
          Modified the TRECCollection class such that additional document tags like the URL can be saved automatically into the meta index. Initial patch provided.

          Example: I want to index the title field and add the url to the meta index for the collection below:
          <doc>
          <docno>NYTMay2006-0</docno>
          <url>http://events.nytimes.com/2006/05/03/dining/reviews/03rest.html&lt;/url>
          <title>Forum: Dine Out</title>
          </doc>

          Set trec.properties:
          # Which fields do we want to pass to the indexer as properties (in addition to the docno)?
          TrecDocTags.propertytags=url
          # At the indexer which keys to we add to the Meta index?
          indexer.meta.forward.keys=docno,url
          # What are the lengths of those keys (e.g. the url will be much longer)?
          indexer.meta.forward.keylens=20,100
          craigm Craig Macdonald made changes -
          Affects Version/s 3.1 [ 10021 ]
          Affects Version/s 3.0 [ 10020 ]
          craigm Craig Macdonald made changes -
          Affects Version/s 3.0 [ 10020 ]
          Affects Version/s 3.1 [ 10021 ]
          Fix Version/s 3.1 [ 10021 ]
          Fix Version/s 3.0 [ 10020 ]
          richardm Richard McCreadie made changes -
          Attachment TREC-178 v2.patch [ 10261 ]
          richardm Richard McCreadie made changes -
          Attachment TREC-178v3.patch [ 10262 ]
          craigm Craig Macdonald made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          craigm Craig Macdonald made changes -
          Project TREC [ 10010 ] Terrier Core [ 10000 ]
          Key TREC-178 TR-137
          Workflow jira [ 10426 ] Terrier Open Source [ 10530 ]
          Affects Version/s 3.0 [ 10030 ]
          Affects Version/s 3.0 [ 10020 ]
          Component/s .indexing [ 10002 ]
          Component/s Core [ 10020 ]
          Fix Version/s 3.1 [ 10040 ]
          Fix Version/s 3.1 [ 10021 ]

            People

            • Assignee:
              richardm Richard McCreadie
              Reporter:
              richardm Richard McCreadie
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: