Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-299

Document the use of DAAT for fast (default?) retrieval

          Details

          • Type: Improvement
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 3.5, 3.6
          • Fix Version/s: 4.0
          • Component/s: None
          • Labels:
            None

            Description

            On ClueWeb12 segment 00, mean query response times can fall from 4.5seconds to 0.76seconds if DAAT is used instead of TAAT.

              Attachments

                Activity

                Descending order - Click to sort in ascending order
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                Resolved

                Show
                richardm Richard McCreadie added a comment - Resolved
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                From TREC-365

                TAAT vs DAAT (50 queries, 300k index)

                Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load)
                Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load)

                Tagging issue for resolution.

                Show
                richardm Richard McCreadie added a comment - From TREC-365 TAAT vs DAAT (50 queries, 300k index) Terrier 4.0 with reverse meta + TAAT = 1.134 seconds (1.58 seconds inc. index load) Terrier 4.0 with reverse meta + DAAT = 0.772 seconds (1.218 seconds inc. index load) Tagging issue for resolution.
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                I thought that we talked about this.

                Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare.

                Linking issue with TREC-365 for quantification.

                Show
                richardm Richard McCreadie added a comment - I thought that we talked about this. Most indexing tasks tackled these days will be more efficient when using daat than taat, since 'small' indices are increasingly rare. Linking issue with TREC-365 for quantification.
                Hide
                Permalink
                craigm Craig Macdonald added a comment -

                Sorry, where did we agree to change the default?

                Show
                craigm Craig Macdonald added a comment - Sorry, where did we agree to change the default?
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                Issue covered. Resolving.

                Show
                richardm Richard McCreadie added a comment - Issue covered. Resolving.
                Hide
                Permalink
                richardm Richard McCreadie added a comment -

                Changed the basic default retrieval matching to daat in Manager.

                r3846

                Show
                richardm Richard McCreadie added a comment - Changed the basic default retrieval matching to daat in Manager. r3846
                Hide
                Permalink
                craigm Craig Macdonald added a comment -

                Committed change to taat package, r3793

                Show
                craigm Craig Macdonald added a comment - Committed change to taat package, r3793
                Hide
                Permalink
                craigm Craig Macdonald added a comment -

                I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents.

                Do we need a wiki page on "making Terrier fast"

                Show
                craigm Craig Macdonald added a comment - I would also suggest making taat.Full give a warning on construction if the index is more than a few million documents. Do we need a wiki page on "making Terrier fast"

                  People

                  • Assignee:
                    richardm Richard McCreadie
                    Reporter:
                    craigm Craig Macdonald
                  • Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: