Part of the problem here is that it is very hard to audit something OpenSSL's size. It has 450,000 lines of code. Are you going to skim, let alone audit, a mathematics-heavy codebase of that size? No.
Some of that is OpenSSL's fault, but a lot of it is inherent in implementing a 200-page (before extensions!) standard. GnuTLS is 170 KLOC, CyaSSL is 205 KLOC, and even PolarSSL, commonly thought of as the gold standard in trimming the fat, is 55 KLOC.
If you want to reduce the level of paranoia, you would have to pair down the size and scope of TLS by an order of magnitude. For the record, I support that kind of thing, but you would have to cut a lot of things people actually use, not to mention breaking backwards compatibility.
While I understand where you've coming from I think your attitude is dangerous. Implementing the SSL/TLS spec(s) is always going to be difficult while they are in widespread use. That said refactoring the existing libraries (to make them more readable and understandable) and then auditing them (to gain confidence in their efficacy) is especially important given the nature of their applications.
Doing the hard work of getting this right, or accepting the status quo and getting it wrong, will have a very real impact on the future of many, many people. Should the time come that changes have to be made to the standards themselves then we can debate the merits of breaking backwards compatibility. I value pragmatism more than most (I think) but there comes a point where the cost of not changing is higher than the cost of changing. I think we're there.
Some of that is OpenSSL's fault, but a lot of it is inherent in implementing a 200-page (before extensions!) standard. GnuTLS is 170 KLOC, CyaSSL is 205 KLOC, and even PolarSSL, commonly thought of as the gold standard in trimming the fat, is 55 KLOC.
If you want to reduce the level of paranoia, you would have to pair down the size and scope of TLS by an order of magnitude. For the record, I support that kind of thing, but you would have to cut a lot of things people actually use, not to mention breaking backwards compatibility.