ICO's a perfectly reasonable format for small icons, and PNG was still a new format then.
I've written stuff that handled the ICO format; it's simpler and easier to load than PNG/GIF/JPG (unless you already have libraries for the latter, of course - which wasn't a given in browsers at the time). I don't think their choice was wrong at all, especially since ICO loading was at the time built into the OS, so any Windows browser that wanted to load favicons didn't even have to build their own loader.
EDIT: Plus, if you're complaining about 'favicon.ico', it's no worse than robots.txt. At the time the modern 'link rel="icon"' approach was definitely not in common use so it's no wonder nobody at MS chose it, and people probably would have complained loudly about MS introducing 'some proprietary icon HTML'.
Another reason ICO is good as a favicon format is that it makes it impossible for site authors to accidentally use a 500KB image file as their icon (which you could trivially do with png, gif, jpg, etc.) - ICO is intentionally a file format for small images, tailor-made for what icons were like in that era. So by relying on ICO they ensured that the favicon.ico on each site would be small, cheap to download, easy to cache, and easy to author.
And ICO has the huge advantage that you can provide multiple bitmap sizes (and color depths) in a single file, avoiding the bullshit of having to specify 6 different icons for 6 different size specs.
Technically, ICO is really a container more than an image format, the original ICO just contained bitmap data, since Vista you can also put PNG data in your ICO file (which MSIE recommends for 256x256 icons).
A similar alternative being Apple's (somewhat more flexible) ICNS
You need to get a life and realise how much pain and suffering you have caused us all non m$ users.
To this day its a pain.. even a png auto would suffice..
What would be your plan NOW ? For everyones sanity . check