From that, The dominant view among scientists concerning the origin of anatomically modern humans is the "Out of Africa" or recent African origin hypothesis, which argues that Homo sapiens arose in Africa and migrated out of the continent around 50,000 to 100,000 years ago, replacing populations of Homo erectus in Asia and Homo neanderthalensis in Europe. Scientists supporting an alternative multiregional hypothesis argue that Homo sapiens evolved as geographically separate but interbreeding populations stemming from a worldwide migration of Homo erectus out of Africa nearly 2.5 million years ago.
The difference is that the species that we call Homo sapiens - us - is on the order of 100,000 years old.
There have been catastrophic events in that time that some believe acted as population bottlenecks, in particular the Toba eruption 70,000 years ago: http://en.wikipedia.org/wiki/Lake_Toba
Of course, there is much debate about the effect this had.
[edit] In a recent Seminar About Long Term Thinking, a biologist said that there is evidence the human population was reduced to a few thousand by the Toba event. That is the extent of my knowledge on the subject.
"There is some evidence, based on mitochondrial DNA, that the human race may have passed through a genetic bottleneck around this time, reducing genetic diversity below what would be expected from the age of the species. According to the Toba catastrophe theory proposed by Stanley H. Ambrose of the University of Illinois at Urbana-Champaign in 1998, human populations may have been reduced to only a few tens of thousands of individuals by the Toba eruption."
http://en.wikipedia.org/wiki/Human_evolution
Populations would have been small, but an extra 2.1 million years is a long time.