Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-157

TRECTerrier scripting files: trec.models qemodels trec.topics.list trec.qrels

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5
    • Component/s: .applications, .utility, tests
    • Labels:
      None

      Description

      Since time memorial, users of Terrier have been able to attempt various weighting models and topics sets etc by adding entries in trec.models qemodels trec.topics.list and trec.qrels

      However, for each of these, there are now properties, and these can be set on the command line. No one in Glasgow now really uses these files. The proposed action is to remove this functionality, in favour of users using properties. If they want to run mutliple experiments, they can use scripting languages (e.g. Bash).

      Discuss.

        Attachments

          Issue Links

            Activity

            Hide
            craigm Craig Macdonald added a comment -

            Ben says he rarely if ever uses this functionality.

            Show
            craigm Craig Macdonald added a comment - Ben says he rarely if ever uses this functionality.
            Hide
            craigm Craig Macdonald added a comment -

            I also have feedback from Gianni that he is happy for this functionality to be only placed in terrier.properties. In this case, I think we have an agreement.

            Show
            craigm Craig Macdonald added a comment - I also have feedback from Gianni that he is happy for this functionality to be only placed in terrier.properties. In this case, I think we have an agreement.
            Hide
            rodrygo Rodrygo L. T. Santos added a comment -

            I personally don't use this property, and I guess most people don't use it either. However, does it cause any harm to justify its removal?

            Show
            rodrygo Rodrygo L. T. Santos added a comment - I personally don't use this property, and I guess most people don't use it either. However, does it cause any harm to justify its removal?
            Hide
            richardm Richard McCreadie added a comment -

            It may be the case that this piece of functionality is unnessesary. I believe that it is both cleaner and more desirable to use properties on the command line to set such parameters. However, just because there is a better way to do something does not mean it should be the only way. While I myself cannot think of a case where it would not be more desirable to use the command line, this does not mean that one does not exist. Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set?

            Show
            richardm Richard McCreadie added a comment - It may be the case that this piece of functionality is unnessesary. I believe that it is both cleaner and more desirable to use properties on the command line to set such parameters. However, just because there is a better way to do something does not mean it should be the only way. While I myself cannot think of a case where it would not be more desirable to use the command line, this does not mean that one does not exist. Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set?
            Hide
            craigm Craig Macdonald added a comment -

            I have several reasons for proposing to remove it:

            1. It was designed when Terrier was primarily for comparing weighting models. Nowadays, people compare many many more things than weighting models (e.g. different parameter values), and its functionality doesn't help those people. Indeed, it only really helps someone comparing weighting models.

            2. Its confusing for the documentation, as users have several places to configure Terrier: the terrier.properties (either on the command line, or in the file) vs these separate files for TRECQuerying. It would be much easier if we just had the corresponding properties listed in terrier.properties

            3. Many of these files already have their corresponding properties, and the move is towards properties (i.e. we haven't added any more of these extra configuration files), hence, why just keep these few.

            4. With properties or files, clearly there can be a conflict, meaning more documentation about how the system resolves these conflicts.

            5. Finally, these files are needed for TRECQuerying to work, which means more configuration files being shipped around.

            Show
            craigm Craig Macdonald added a comment - I have several reasons for proposing to remove it: 1. It was designed when Terrier was primarily for comparing weighting models. Nowadays, people compare many many more things than weighting models (e.g. different parameter values), and its functionality doesn't help those people. Indeed, it only really helps someone comparing weighting models. 2. Its confusing for the documentation, as users have several places to configure Terrier: the terrier.properties (either on the command line, or in the file) vs these separate files for TRECQuerying. It would be much easier if we just had the corresponding properties listed in terrier.properties 3. Many of these files already have their corresponding properties, and the move is towards properties (i.e. we haven't added any more of these extra configuration files), hence, why just keep these few. 4. With properties or files, clearly there can be a conflict, meaning more documentation about how the system resolves these conflicts. 5. Finally, these files are needed for TRECQuerying to work, which means more configuration files being shipped around.
            Hide
            craigm Craig Macdonald added a comment -

            Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set?

            I think that removing functionality which has been outgrown (or is broken - but not completely in this case) is a way of ensuring that the platform matures. Does Windows 7 ship with File Manager or Program Manager from Windows 3.0 - no, because something better has been designed. True that sometimes progress has opposition, however if no-one is using a particular piece of functionality, then there has to be a decision based on a cost-benefit analysis - it has a cost to maintain (c.f TR-25) but there is no benefit in investing in its maintenance - such investment benefits no-one. (Windows 95 did have File Manager btw, but it will have been removed by Windows 7).

            Show
            craigm Craig Macdonald added a comment - Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set? I think that removing functionality which has been outgrown (or is broken - but not completely in this case) is a way of ensuring that the platform matures. Does Windows 7 ship with File Manager or Program Manager from Windows 3.0 - no, because something better has been designed. True that sometimes progress has opposition, however if no-one is using a particular piece of functionality, then there has to be a decision based on a cost-benefit analysis - it has a cost to maintain (c.f TR-25 ) but there is no benefit in investing in its maintenance - such investment benefits no-one. (Windows 95 did have File Manager btw, but it will have been removed by Windows 7).
            Hide
            richardm Richard McCreadie added a comment -

            That seems fine. However, under the same logic, I would question the utility of the /etc subfolder, as it will subsequently contain only the propertes file? (which could be moved up a dir)

            Show
            richardm Richard McCreadie added a comment - That seems fine. However, under the same logic, I would question the utility of the /etc subfolder, as it will subsequently contain only the propertes file? (which could be moved up a dir)
            Hide
            craigm Craig Macdonald added a comment - - edited

            That would be a major change that I wouldn't attempt for 3.1. Perhaps someone can come to the defense of the etc/ folder?

            Show
            craigm Craig Macdonald added a comment - - edited That would be a major change that I wouldn't attempt for 3.1. Perhaps someone can come to the defense of the etc/ folder?
            Hide
            craigm Craig Macdonald added a comment -

            Attached updated patch from Nut. This removed qemodels, trec.qrels trec.topics.list files from the etc/ folder, and removes iterating functionality from TRECQuerying.

            Show
            craigm Craig Macdonald added a comment - Attached updated patch from Nut. This removed qemodels, trec.qrels trec.topics.list files from the etc/ folder, and removes iterating functionality from TRECQuerying.
            Hide
            craigm Craig Macdonald added a comment -

            Committed to trunk. Thanks Nut!

            Show
            craigm Craig Macdonald added a comment - Committed to trunk. Thanks Nut!
            Hide
            nutli Nut Limsopatham added a comment -

            Patch for html update

            Show
            nutli Nut Limsopatham added a comment - Patch for html update

              People

              • Assignee:
                nutli Nut Limsopatham
                Reporter:
                craigm Craig Macdonald
              • Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: