Yes, we've given credits (or a refund, at the customer's preference) in cases where lack of visibility or inaccurate docs led to over-provisioning of dynos.
There's actually very few cases where people paid more money than they would have otherwise. Heroku is a service with your bill prorated to the second. For the most part, if people don't like the performance (which is measurable externally via benchmarks and browser inspectors), they leave the service. Many people who hit problems with visibility and performance did exactly that.
Naturally, we'll be working hard to try to recapture their business, as well as to remove any reasons that existing customers might leave as they hit performance or visibility problems scaling up in the future.
