One minor nit: it's important to distinguish between phonemic and phonetic transcriptions in IPA. The former are what's usually found in dictionaries and does not account for variance in dialects and speakers. The latter attempts to represent utterances as they are actually produced by speakers.
There's still some nuance that is lost in transcription but phonetic IPA transcriptions can achieve a pretty close approximation to the real utterances.
This is true, but you make it sound like this is a response to and even a solution for my comment, when I was actually assuming a phonetic transcription: such transcriptions also vary in how narrowly they define the sound involved, and it is the nuance of sound to which I was commenting, as even in a very narrow phonetic transcription (with tons of marks to try to adjust the sounds of the phonemes), you just can't represent what a native speaker sounds like using these symbols without adjusting for language; and I guess we just have to disagree with how "pretty close" the result is, as when I thought about what I would want to use an IPA->speech tool to accomplish, they all involved vowel charts ;P.
> This is true, but you make it sound like this is a response to and even a solution for my comment
I think it's a suitable response. There is enough allophonic variation among speakers of a single language that undoubtedly someone perfectly reciting a narrow IPA transcription could be taken as a plausible native speaker. Even for your purposes (though I'm still not clear on what type of task your envisioning) the IPA could still be useful as an intermediary layer of abstraction, as in storing a mapping by language of IPA vowel symbols to the exact formants required.
There's still some nuance that is lost in transcription but phonetic IPA transcriptions can achieve a pretty close approximation to the real utterances.