All hand written over 10 years. They catch a lot of stuff, but stuff still slips through. Generally the area of the breakage even has tests over it but didn’t manage to catch the particular issue or they had mocked out parts which ended up being the issue.
I can’t see how static analysts could solve it really.
imho, this is a huge reason why ruby fell off the popularity charts over time. i predicted this pretty early on and got a lot of flack from the ruby community as a result.
nobody wanted to write tests to ensure that something would compile... let alone tell you what 'type' an object was. i can't tell you how many hours i wasted watching my pair trying to figure out what 'type' an object was, by using `print`. luckily i was working as a very expensive consultant.
of course these things could be bolted onto ruby, but it never felt natural. running 40k tests to see if something compiles is absurd, takes forever and results in super brittle tests that have to be reworked when code changes. often people would use meta programming in the tests, which only made figuring out the errors even harder since the line numbers would be all wonky in traces.
sorry you're saddled with that legacy of well intention and poor execution.
Also, out of curiosity, are all of those tests hand written, or is there some generation in there?