Biodiversity, I think is the biggest problem. The potato blight spread so rampently because mostly one clone of the exact potato was being grown everywhere. Some sort of rice or corn blight would be catastrophic.
Absolutely true. Also consider the fate of the Gros Michel banana. Biodiversity is vital for our food production, and GMOs definitely carry the risk of creating monocultures.