I suppose you could do something like, enumerate every possible combination of inputs and check that some property holds for all of them. Or, maybe you could instead randomly select a number of combinations of inputs and check that a property holds for each of those random combinations, but that wouldn't be guaranteed to find the inputs for which the specification isn't satisfied.
I guess maybe if the test passed to the function to be tested, mock values, such that the function is effectively evaluated symbolically (where any branching that depends on the inputs to the function, would maybe have the mocked object specify what the result of the conditional should be, with different tests for different cases?) ?
Or.. Can you explain how you write tests such that they truly function as specifications?
Good question - and there's been lots of work on this area. See for example property testing and fuzz testing, which can do something similar to what your second paragraph suggests.
You should be able to find a property testing library in your favourite language such as Hypothesis (python), Quickcheck (Haskell), Fastcheck (JS/typescript), etc.
A “specification” which is based on just “the desired behavior in a few specific examples” seems to allow undesired behavior without it being clear whether said behavior is allowed by the spec.
That to me doesn’t seem like what a “specification” is.
Now, maybe having good examples is more important for business use cases than having a specification.
I suppose you could do something like, enumerate every possible combination of inputs and check that some property holds for all of them. Or, maybe you could instead randomly select a number of combinations of inputs and check that a property holds for each of those random combinations, but that wouldn't be guaranteed to find the inputs for which the specification isn't satisfied.
I guess maybe if the test passed to the function to be tested, mock values, such that the function is effectively evaluated symbolically (where any branching that depends on the inputs to the function, would maybe have the mocked object specify what the result of the conditional should be, with different tests for different cases?) ?
Or.. Can you explain how you write tests such that they truly function as specifications?