Postgresql testing works the same way it has worked for the last 20-30 years. Each test consists of a SQL script, an expected result file, and a result file. A shell script runs the SQL script through the pg commandline, and dumps the results to the results file. Diff then tells you if the test pass. Also the shell scripts were rewritten in C for cross platform support. Not available via SQL, ie vacum? No tests. Concurrent modifications? No tests. Testing query plans that don't use order by have fun.
They have a new test suite built around TAP (https://testanything.org/) and perl so now they have automated tests for more things like certs, and replication.
- pg_regress, which is as you say a set of SQL scripts run on a running server with generated output compared to the expected output. There could be multiple expected outputs, see for example float tests.
- pg_isolation_test, which is a part you are misunderstanding. This facility has been introduced with SSI (serializable transactions) to test concurrency.
- TAP tests (author of the replication/backup/recovery part here), which has roughly three test suites:
-- src/bin/ to test all the in-core binaries
-- src/test/ssl to test SSL configurations. This has proved to be useful lately when moving all the SSL parameters to be reloadable.
-- src/test/recovery to test all the kind of replication scenarios.
There are as well modules in src/test/modules that use TAP tests to run regressions on pg_dump, commit timestamps and extension interactions.
The query VACUUM is being tested via SQL. Now if you are referring to autovacuum, there are no tests directly designed with this purpose in mind. autovacuum is running in any instance running the regression tests as it is always enabled by default.
I'm hoping for one too. I have become frighteningly reliant on the amazing HN community for their tl;dr's.
I literally started watching the video so I could do one and immediately felt guilty because I realize I need to add better tests to my latest. So back to coding instead.
