I have several reasons for proposing to remove it:
1. It was designed when Terrier was primarily for comparing weighting models. Nowadays, people compare many many more things than weighting models (e.g. different parameter values), and its functionality doesn't help those people. Indeed, it only really helps someone comparing weighting models.
2. Its confusing for the documentation, as users have several places to configure Terrier: the terrier.properties (either on the command line, or in the file) vs these separate files for TRECQuerying. It would be much easier if we just had the corresponding properties listed in terrier.properties
3. Many of these files already have their corresponding properties, and the move is towards properties (i.e. we haven't added any more of these extra configuration files), hence, why just keep these few.
4. With properties or files, clearly there can be a conflict, meaning more documentation about how the system resolves these conflicts.
5. Finally, these files are needed for TRECQuerying to work, which means more configuration files being shipped around.