But compression tests only have that issue because there are degrees of success for those tests, even for the same compression algorithm. I don't think they're hiding the test vectors because they're worried about gaming the tests by purposely failing to process valid inputs to achieve better metrics, just writing overly specific heuristics.
For straightforward corner case acceptance tests (which I would assume covers most of the coreutils test suite) there's not really a danger of overfitting unless the developers are literally writing if statements that match a single input from the test and provide the correct output.
For straightforward corner case acceptance tests (which I would assume covers most of the coreutils test suite) there's not really a danger of overfitting unless the developers are literally writing if statements that match a single input from the test and provide the correct output.