Not to be disparaging, but I wouldn't call this a "comprehensive" "guide" to public key cryptography. In social conversation I can usually help non-technical people understand the basic principles of public key cryptography at greater depth than this and in less time. This article doesn't mention RSA or integer factorization anywhere, which is the most straightforward template for explaining how asymmetric problems work. It also comes close to conflating one way functions with trapdoor functions - these are not the same thing, but the article uses the same definition for both.
Despite the underlying mathematical complexity of public key cryptography it's a fairly neat and easy concept to understand. We don't need to call it "mathematical magic" because it's already intuitive to us. Ask someone to tell you all the numbers which multiply to 81. Next ask them to multiply 9 by 9, then ask them which exercise was easier. On paper, you can demonstrate that multiplication is both faster and conceptually easier than long division. Now you have taken an abstract concept - the "one way function" - and grounded it in something the person learned as a child. It doesn't take much more effort to explain how you can use very large and difficult-to-factor numbers (large semiprimes) to obfuscate information. We all naturally struggle to divide more than we struggle to multiply. Conveniently, so do computers.
But that's just a one way function, which is insufficient for public key cryptography. There is no relation between multiplication and factorization suitable for creating public and private keys, or else we wouldn't need the entire RSA algorithm. One way functions are just hash functions (or the theoretical basis thereof). A trapdoor function takes this further and augments a one way function such that it's still infeasible to invert the output back to the input, except if you have some piece of information. This is why hash functions are relatively easier mathematical primitives to understand, because establishing a trapdoor requires significantly more mathematical structure to get right while still maintaining security. They're far more delicate. You need enough mathematical structure to bypass the infeasible inversion without compromising the infeasibility of the inversion entirely. It's like poking a hole in an egg without breaking the whole shell.
Hash functions are OWFs, but OWFs are not necessarily hash functions. Among other things, for an OWF inverting the function is only hard on average, and recovering some information about the "preimage" may not be hard (the only guarantee is that there is something about the input that is hard to compute). Also, there are common uses of hash functions that are not even theoretically sound (anything in the random oracle model), so we should be careful when discussing the "theoretical basis" for hash functions...
Yes, hash functions only approximate one way functions since we haven't technically created one - your point here is what I was referring to by mentioning the random oracle model.
But the random oracle model is fully theoretically sound - I'm not sure where you got the impression it isn't. Do you think everything outside the standard model of cryptography isn't theoretically sound? What, then, is your definition of soundness?
ROM is not sound because there are constructions that are provably secure that cannot be securely instantiated. I do not deny that it is pragmatic or that the problems have not been relevant in practice, but the ROM is just a heuristic.
Despite the underlying mathematical complexity of public key cryptography it's a fairly neat and easy concept to understand. We don't need to call it "mathematical magic" because it's already intuitive to us. Ask someone to tell you all the numbers which multiply to 81. Next ask them to multiply 9 by 9, then ask them which exercise was easier. On paper, you can demonstrate that multiplication is both faster and conceptually easier than long division. Now you have taken an abstract concept - the "one way function" - and grounded it in something the person learned as a child. It doesn't take much more effort to explain how you can use very large and difficult-to-factor numbers (large semiprimes) to obfuscate information. We all naturally struggle to divide more than we struggle to multiply. Conveniently, so do computers.
But that's just a one way function, which is insufficient for public key cryptography. There is no relation between multiplication and factorization suitable for creating public and private keys, or else we wouldn't need the entire RSA algorithm. One way functions are just hash functions (or the theoretical basis thereof). A trapdoor function takes this further and augments a one way function such that it's still infeasible to invert the output back to the input, except if you have some piece of information. This is why hash functions are relatively easier mathematical primitives to understand, because establishing a trapdoor requires significantly more mathematical structure to get right while still maintaining security. They're far more delicate. You need enough mathematical structure to bypass the infeasible inversion without compromising the infeasibility of the inversion entirely. It's like poking a hole in an egg without breaking the whole shell.