Code Coverage for Zig with Callgrind

[todsacerdoti]

https://www.ryanliptak.com/blog/code-coverage-zig-callgrind/

[tiehuis]

This is a nice write-up. Since zig has quite extensive compile-time functionality, one drawback is it is often easy to miss untested branches, especially for non-native targets if they are not regularly tested. It would be quite beneficial to be able to combine coverage reports from different targets to get a full picture of the tested code.

I have previously used kcov [1] to perform coverage testing which due to zig having good DWARF debugging information worked with no fanfare. However, I'm not too familiar with the shortcomings of this method vs. the more fully instrumented binary approach provided.

[1] https://github.com/SimonKagstrom/kcov

[squeek502 → tiehuis]

Cool, I wasn't aware of kcov. Looks like kcov sets breakpoints on every line to get its coverage information, which seems like it'd probably be comparable to what callgrind is doing. The difference would be in execution speed, and after testing kcov, it looks like it wins that handily:

- Running the zig std lib tests normally takes ~5 seconds

- Running the zig std lib tests with callgrind takes ~2 minutes

- Running the zig std lib tests with kcov takes ~9 seconds

There are also slight disagreements in terms of the results:

- grindcov reports 43480 of 52586 lines executed or 82.68% coverage

- kcov reports 44103 of 52335 lines executed or 84.3% coverage

This difference might be down to kcov following child processes while grindcov doesn't, but I'm not totally sure.

kcov is probably the better choice overall.

[touisteur → tiehuis]

GNATCoverage (Ada-only) has interesting binary coverage capabilities. Qemu trace, valgrind instrumentation, nexus... If interested I built once A gdb-branch-trace script, a pintool to generate the trace or an Intel-PT-based prototype. Dyninst and dynamo rio too. Neat exercise

[gravypod]

I wish there was a repo where someone figured out how to generate coverage data for "all languages" into a single coverage format and write standardized merging and rendering tools, etc.

It's one of those fundamental things that should just work the same across every ecosystem.

[daitangio]

About memory allocator:

Sorry for the possibly silly question but I did not understand if Zig has a Rust similar concept of "borrow" for pointers, of it is based on static analysis of allocator?

Zig seems to me easier to use than Rust on this respect, but I cannot understand the difference (because I do not know them very well)

[michaelcampbell]

meta: Unless you're using TeX, drop the full justification. It looks awful, IMO, and makes it somewhat harder to read.

Reasonable people can disagree.

Reader mode was a savior, here.

[squeek502 → michaelcampbell]

Noted. Full-justification was inherited from the default style of the static site generator I'm using. I'll get rid of it.

EDIT: Done.

[michaelcampbell → squeek502]

(IMO), 1000% better.