It's interesting that you should mention smell, since I was born without a sense of smell - the term is anosmia. It is very poorly studied, and most studies deal with people who lose their sense of smell due to neurological diseases or head trauma.
In my case it's clearly genetic (my mom and her dad both can't smell), and almost no one studies it. From what I've read I'd much rather be born without a sense of smell than lose it, since that can be very traumatic.
I was specifically thinking of anosmia and ageusia. There are many invisible disabilities that most people aren't aware of; anosmia is one of the ones where most people who don't have the issue don't even see why it might be a problem until they've really thought about it.
Couple of articles about or by people who have experienced the loss:
Anosmia is much more serious when it's the result of trauma or some other disease - it makes food taste radically different, and is likely to cause depression. That said, I've never had a sense of smell, so I don't know what I'm missing. And the world is much less smell-focused than vision or hearing, so if you have to lose a sense, smell is near the top of the list.
Sounds like for you it's a drawback but not a trauma. It's also not a problem I have personal experience with, though I have with others.
I guess my point was a lot of people hearing about traumatic loss think its no big deal because you can still work without it and don't immediately make the connection between "losing your sense of smell" and "losing the ability to taste chocolate like you're used to".
In my case it's clearly genetic (my mom and her dad both can't smell), and almost no one studies it. From what I've read I'd much rather be born without a sense of smell than lose it, since that can be very traumatic.