Their benchmarks are not to be trusted (after reading the source).
- They cheat, they rewrite code to use coden specific features to "win" (ie, parallelization and GPU optimizations)
- They don't warm up. They are simply running their competition directly rather than allowing any sort of warmup. (In other words, they are measuring cold boot and startup time)
Now, if they want to argue about startup time or whatever mattering for performance then fine. However, the representation of "20x faster!" is simply deception.
TBF, they are upfront about cheating
> Further, some of the benchmarks are identical in both Python and Codon, some are changed slightly to work with Codon's type system, and some use Codon-specific features like parallelism or GPU.
- They cheat, they rewrite code to use coden specific features to "win" (ie, parallelization and GPU optimizations)
- They don't warm up. They are simply running their competition directly rather than allowing any sort of warmup. (In other words, they are measuring cold boot and startup time)
Now, if they want to argue about startup time or whatever mattering for performance then fine. However, the representation of "20x faster!" is simply deception.
TBF, they are upfront about cheating
> Further, some of the benchmarks are identical in both Python and Codon, some are changed slightly to work with Codon's type system, and some use Codon-specific features like parallelism or GPU.