Sure, there were a bunch of genius-level physicists capable of building an atom bomb and the US Government didn't use them because it wanted to be mean to people. That's believable.
No, the idea is that in that era an untold number of talented people were barred from contributing to the US due to discriminatory roadblocks at every level in terms of access to education and so on.
Your sarcasm is hilariously misplaced because the person I cited was driven away by stubborn US officials only to become the leading light behind the Chinese nuclear program. All they had to do was leave him alone. So yes, what you are describing literally happened...