Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-299

Document the use of DAAT for fast (default?) retrieval

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5, 3.6
    • Fix Version/s: 4.0
    • Component/s: None
    • Labels:
      None

      Description

      On ClueWeb12 segment 00, mean query response times can fall from 4.5seconds to 0.76seconds if DAAT is used instead of TAAT.

        Attachments

          Activity

          craigm Craig Macdonald created issue -
          Hide
          craigm Craig Macdonald added a comment -

          I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents.

          Do we need a wiki page on "making Terrier fast"

          Show
          craigm Craig Macdonald added a comment - I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents. Do we need a wiki page on "making Terrier fast"
          Hide
          craigm Craig Macdonald added a comment -

          Committed change to taat package, r3793

          Show
          craigm Craig Macdonald added a comment - Committed change to taat package, r3793
          Hide
          richardm Richard McCreadie added a comment -

          Changed the basic default retrieval matching to daat in Manager.

          r3846

          Show
          richardm Richard McCreadie added a comment - Changed the basic default retrieval matching to daat in Manager. r3846
          Hide
          richardm Richard McCreadie added a comment -

          Issue covered. Resolving.

          Show
          richardm Richard McCreadie added a comment - Issue covered. Resolving.
          richardm Richard McCreadie made changes -
          Field Original Value New Value
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          craigm Craig Macdonald added a comment -

          Sorry, where did we agree to change the default?

          Show
          craigm Craig Macdonald added a comment - Sorry, where did we agree to change the default?
          craigm Craig Macdonald made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          richardm Richard McCreadie added a comment -

          I thought that we talked about this.

          Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare.

          Linking issue with TREC-365 for quantification.

          Show
          richardm Richard McCreadie added a comment - I thought that we talked about this. Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare. Linking issue with TREC-365 for quantification.
          richardm Richard McCreadie made changes -
          Link This issue blocks TREC-365 [ TREC-365 ]
          Hide
          richardm Richard McCreadie added a comment -

          From TREC-365

          TAAT vs DAAT (50 queries, 300k index)

          Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load)
          Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load)

          Tagging issue for resolution.

          Show
          richardm Richard McCreadie added a comment - From TREC-365 TAAT vs DAAT (50 queries, 300k index) Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load) Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load) Tagging issue for resolution.
          Hide
          richardm Richard McCreadie added a comment -

          Resolved

          Show
          richardm Richard McCreadie added a comment - Resolved
          richardm Richard McCreadie made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          richardm Richard McCreadie made changes -
          Project TREC [ 10010 ] Terrier Core [ 10000 ]
          Key TREC-372 TR-299
          Workflow jira [ 10817 ] Terrier Open Source [ 10862 ]
          Affects Version/s 3.6 [ 10060 ]
          Affects Version/s 3.5 [ 10040 ]
          Affects Version/s 3.6 [ 10061 ]
          Affects Version/s 3.5 [ 10021 ]
          Component/s Core [ 10020 ]
          Fix Version/s 4.0 [ 10051 ]
          Fix Version/s 4.0 [ 10050 ]

            People

            • Assignee:
              richardm Richard McCreadie
              Reporter:
              craigm Craig Macdonald
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: