One thing I learned about archive.org and robots.txt is that they never actually delete anything. If you accidentally block their bots, then in a month or so, your old content is available again. I've blocked their bots by mistake a few times, and each time my old web content is back. Not a big deal for me, I just feel silly for my old geocities style sites. :-)
