Right, I quickly made peace with being a mediocre student at Stanford (and, still ended up getting into MIT for a physics PhD, where I was also a mediocre student but by then I didn't care :) ). I think the people who probably suffer the most at are probably premeds, where it really might be easier to get into med school by going to an "easier" school.
In the grand scheme of things, I was probably pretty good at math as I got through a rigorous Mech E program. But I struggled with the more math intensive classes (Fluids/Navier-Stokes) and I'm pretty sure I could have dealt with an undergrad in physics/math--much less a grad program--about as well as I could have flown by flapping my arms in the air. But things ended up fine.
