I think that's sort of what he's talking about. Had you not had the experience of calculus and diff eq at a university and your only exposure to it was at Stanford with middling grades - perhaps it'd probably lead to more existential questions of whether you really wanted to do STEM in the first place.
Right, I quickly made peace with being a mediocre student at Stanford (and, still ended up getting into MIT for a physics PhD, where I was also a mediocre student but by then I didn't care :) ). I think the people who probably suffer the most at are probably premeds, where it really might be easier to get into med school by going to an "easier" school.
In the grand scheme of things, I was probably pretty good at math as I got through a rigorous Mech E program. But I struggled with the more math intensive classes (Fluids/Navier-Stokes) and I'm pretty sure I could have dealt with an undergrad in physics/math--much less a grad program--about as well as I could have flown by flapping my arms in the air. But things ended up fine.