Higher orbits are slower, not faster. Very low orbits as you say though are less an issue due to atmospheric drag deorbiting debris. A bit higher and debris will stay up much longer, even higher up though and there's so much room we can't possibly put out enough material to fill the orbits yet. The mid point of that scale is the dangerous region.
Higher altitude orbits are slower in angular velocity, but faster in linear velocity, assuming a circular orbit. Faster debris ends up in higher orbit.
Higher altitude circular orbits are slower in linear velocity too. Look it up or work it out by equating the centripetal force with the gravitational attraction, which is the condition for a circular orbit.
