What would be a good starting point to learn more about the topics described in this article? I felt like I understood the gist, but a lot of things went over my head.
Douglas Hofstadter's "Gödel, Escher Bach" does a very slow gradual walk through Gödel's incompleteness theorem. If you can get into his writing style, it's a good way to really "get" the points.
Chaitin made the very good point that inferential undecidability (incompleteness) is too important a result to depend on the triviality of the existence of the proposition I'mUnprovable. Fortunately, inferential undecidability of strongly-typed theories can be proved without using the nonexistent I'mUnprovable.