I don’t have references for this, but my understanding is that you have more to worry about space debris that is higher/faster than lower/slower. The atmospheric drag is such that any out of control craft at [some threshold] or under will deorbit themselves fairly quickly, and the debris will burn up entirely. I believe that threshold is around 450-500km, but it’s also probably not a literal height, more of a gradient of risk. Higher/faster, they will be up there for longer (years? decades?) and it’s harder to predict where any broken pieces will go over time.
Higher orbits are slower, not faster. Very low orbits as you say though are less an issue due to atmospheric drag deorbiting debris. A bit higher and debris will stay up much longer, even higher up though and there's so much room we can't possibly put out enough material to fill the orbits yet. The mid point of that scale is the dangerous region.
Higher altitude orbits are slower in angular velocity, but faster in linear velocity, assuming a circular orbit. Faster debris ends up in higher orbit.
Higher altitude circular orbits are slower in linear velocity too. Look it up or work it out by equating the centripetal force with the gravitational attraction, which is the condition for a circular orbit.
