jQuery is usually cached in your browser because of how common it is. From what I understand, the slowing is negligible. Part of a developer's role is to mitigate site performance with the speed of process. Inlining of critical CSS is one way to balance out initial page load, for example.
This is part myth. The chances your site has the same version of jQuery that the visitor's browser cache is are pretty low. iirc, someone once said it was less than 5%.
