Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-299

Document the use of DAAT for fast (default?) retrieval

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5, 3.6
    • Fix Version/s: 4.0
    • Component/s: None
    • Labels:
      None

      Description

      On ClueWeb12 segment 00, mean query response times can fall from 4.5seconds to 0.76seconds if DAAT is used instead of TAAT.

        Attachments

          Activity

          Hide
          craigm Craig Macdonald added a comment -

          I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents.

          Do we need a wiki page on "making Terrier fast"

          Show
          craigm Craig Macdonald added a comment - I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents. Do we need a wiki page on "making Terrier fast"
          Hide
          craigm Craig Macdonald added a comment -

          Committed change to taat package, r3793

          Show
          craigm Craig Macdonald added a comment - Committed change to taat package, r3793
          Hide
          richardm Richard McCreadie added a comment -

          Changed the basic default retrieval matching to daat in Manager.

          r3846

          Show
          richardm Richard McCreadie added a comment - Changed the basic default retrieval matching to daat in Manager. r3846
          Hide
          richardm Richard McCreadie added a comment -

          Issue covered. Resolving.

          Show
          richardm Richard McCreadie added a comment - Issue covered. Resolving.
          Hide
          craigm Craig Macdonald added a comment -

          Sorry, where did we agree to change the default?

          Show
          craigm Craig Macdonald added a comment - Sorry, where did we agree to change the default?
          Hide
          richardm Richard McCreadie added a comment -

          I thought that we talked about this.

          Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare.

          Linking issue with TREC-365 for quantification.

          Show
          richardm Richard McCreadie added a comment - I thought that we talked about this. Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare. Linking issue with TREC-365 for quantification.
          Hide
          richardm Richard McCreadie added a comment -

          From TREC-365

          TAAT vs DAAT (50 queries, 300k index)

          Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load)
          Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load)

          Tagging issue for resolution.

          Show
          richardm Richard McCreadie added a comment - From TREC-365 TAAT vs DAAT (50 queries, 300k index) Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load) Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load) Tagging issue for resolution.
          Hide
          richardm Richard McCreadie added a comment -

          Resolved

          Show
          richardm Richard McCreadie added a comment - Resolved

            People

            • Assignee:
              richardm Richard McCreadie
              Reporter:
              craigm Craig Macdonald
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: