clang+llvm releases do a three stage build and compare the output of the third stage with the second. This is fairly effective at rooting out many sources of indeterminate behavior.
> In broad strokes they do but often they’ll have randomness creep in, such as different build hashes or iteration order of associative containers.
In order to find defects related to codegen being altered by incidental ordering of objects in containers, a build option called LLVM_ENABLE_REVERSE_ITERATION was created. Builders periodically run with this mode to verify that the regression test suite still passes when containers iterate in reverse.
That said, it's true that there are probably remaining sources of variation among builds. There is a significant effort [1] to find these and avoid them.
Not all containers right? Just unordered containers. It’s a bit weird of a strategy though - most other applications use a random seed for unordered hash tables.
Rather than change the implementation of the DenseMap/DenseSet/StringMap/StringSet containers to mask such codegen defects, the community opted to create a test strategy to find the defects instead.
I would guess that this may also end up favoring the execution performance of the compiler over other design choices, but I could be wrong about that one.
The iteration order is undefined for these containers. The whole point of the reverse iteration is to find when reproducibility is dependent on undefined iteration order. If the reverse iteration accomplishes this, so would a random seed.
> In broad strokes they do but often they’ll have randomness creep in, such as different build hashes or iteration order of associative containers.
In order to find defects related to codegen being altered by incidental ordering of objects in containers, a build option called LLVM_ENABLE_REVERSE_ITERATION was created. Builders periodically run with this mode to verify that the regression test suite still passes when containers iterate in reverse.
That said, it's true that there are probably remaining sources of variation among builds. There is a significant effort [1] to find these and avoid them.
[1] https://reproducible-builds.org/