That actually happened 100s of years after the political fall of the empire and it also happened to a large extent in the East where the empire persisted.
So the political fall of the West as the primary factor in de-urbanization is not really a good argument.
Except it's exactly like that. Huge numbers of people died; population cratered.