Both Goudsmit and Pash wrote books about their Alsos mission to find out about the German atomic bomb program. Goudsmit opens his book with a drawing of the German experimental station for atomic bomb research - a house-sized wooden building.[1] The US's Manhattan Project involved building as much plant as the prewar auto industry. The US intelligence people knew there was a German atomic program, but couldn't find out anything about it, so they assumed it must have really good secrecy. Goudsmit mentions that the V-2 rocket project made no sense unless it could carry an atomic bomb - accuracy was so bad that it couldn't hit a target smaller than Greater London, and it cost as much as a fighter plane.
No, it was just a tiny program.
[1] https://books.google.com/books?id=3v2ttYJ_d2kC&pg=PA2