[TR-157] TRECTerrier scripting files: trec.models qemodels trec.topics.list trec.qrels Created: 18/Feb/11  Updated: 10/Jun/11  Resolved: 05/Apr/11

Status: Resolved
Project: Terrier Core
Component/s: .applications, .utility, tests
Affects Version/s: None
Fix Version/s: 3.5

Type: Improvement Priority: Minor
Reporter: Craig Macdonald Assignee: Nut Limsopatham
Resolution: Fixed  
Labels: None

Attachments: Text File patchTR126.txt     File TR-157_html_document.patch     File TREC-216.v2.patch    
Issue Links:
Block
blocks TR-25 TRECQueryingExpansion required trec.t... Resolved

 Description   
Since time memorial, users of Terrier have been able to attempt various weighting models and topics sets etc by adding entries in trec.models qemodels trec.topics.list and trec.qrels

However, for each of these, there are now properties, and these can be set on the command line. No one in Glasgow now really uses these files. The proposed action is to remove this functionality, in favour of users using properties. If they want to run mutliple experiments, they can use scripting languages (e.g. Bash).

Discuss.

 Comments   
Comment by Craig Macdonald [ 18/Feb/11 ]

Ben says he rarely if ever uses this functionality.

Comment by Craig Macdonald [ 21/Feb/11 ]

I also have feedback from Gianni that he is happy for this functionality to be only placed in terrier.properties. In this case, I think we have an agreement.

Comment by Rodrygo L. T. Santos [ 21/Feb/11 ]

I personally don't use this property, and I guess most people don't use it either. However, does it cause any harm to justify its removal?

Comment by Richard McCreadie [ 21/Feb/11 ]

It may be the case that this piece of functionality is unnessesary. I believe that it is both cleaner and more desirable to use properties on the command line to set such parameters. However, just because there is a better way to do something does not mean it should be the only way. While I myself cannot think of a case where it would not be more desirable to use the command line, this does not mean that one does not exist. Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set?

Comment by Craig Macdonald [ 21/Feb/11 ]

I have several reasons for proposing to remove it:

1. It was designed when Terrier was primarily for comparing weighting models. Nowadays, people compare many many more things than weighting models (e.g. different parameter values), and its functionality doesn't help those people. Indeed, it only really helps someone comparing weighting models.

2. Its confusing for the documentation, as users have several places to configure Terrier: the terrier.properties (either on the command line, or in the file) vs these separate files for TRECQuerying. It would be much easier if we just had the corresponding properties listed in terrier.properties

3. Many of these files already have their corresponding properties, and the move is towards properties (i.e. we haven't added any more of these extra configuration files), hence, why just keep these few.

4. With properties or files, clearly there can be a conflict, meaning more documentation about how the system resolves these conflicts.

5. Finally, these files are needed for TRECQuerying to work, which means more configuration files being shipped around.

Comment by Craig Macdonald [ 21/Feb/11 ]

Moreover, removing functionality because we think that alternative is better feels a bit presumptious, what kind of precedent does this set?

I think that removing functionality which has been outgrown (or is broken - but not completely in this case) is a way of ensuring that the platform matures. Does Windows 7 ship with File Manager or Program Manager from Windows 3.0 - no, because something better has been designed. True that sometimes progress has opposition, however if no-one is using a particular piece of functionality, then there has to be a decision based on a cost-benefit analysis - it has a cost to maintain (c.f TR-25) but there is no benefit in investing in its maintenance - such investment benefits no-one. (Windows 95 did have File Manager btw, but it will have been removed by Windows 7).

Comment by Richard McCreadie [ 21/Feb/11 ]

That seems fine. However, under the same logic, I would question the utility of the /etc subfolder, as it will subsequently contain only the propertes file? (which could be moved up a dir)

Comment by Craig Macdonald [ 21/Feb/11 ]

That would be a major change that I wouldn't attempt for 3.1. Perhaps someone can come to the defense of the etc/ folder?

Comment by Craig Macdonald [ 05/Apr/11 ]

Attached updated patch from Nut. This removed qemodels, trec.qrels trec.topics.list files from the etc/ folder, and removes iterating functionality from TRECQuerying.

Comment by Craig Macdonald [ 05/Apr/11 ]

Committed to trunk. Thanks Nut!

Comment by Nut Limsopatham [ 10/Jun/11 ]

Patch for html update

Generated at Sat Dec 16 03:25:20 GMT 2017 using JIRA 7.1.1#71004-sha1:d6b2c0d9b7051e9fb5e4eb8ce177ca56d91d7bd8.