Dwarves are short people, so there's really no mystery there. As for gnomes and trolls, they could be explained as birth defects. The myth of "changelings" are probably these birth defects that might not be noticeable in a newly born child, but later become obvious.
I think the real cultural memory of the Neanderthals and other species related to us is in the myths of Titans and Giants. People who were on earth first and who battled furiously with man-like gods for dominance.
I think the real cultural memory of the Neanderthals and other species related to us is in the myths of Titans and Giants. People who were on earth first and who battled furiously with man-like gods for dominance.