From eyeballing that chart, x264 Baseline (a very, very good implementation, that can beat some other H.264 encoders in High Profile) never beats Theora 1.1 by more than 20%.
Actually he has figures in the table, but the figures are all given against the winning x264 profile, I've adapted them to compare x264 baseline and Theora 1.1:
Which is a very good showing for Theora, despite his misleading textual comment ("Ogg Theora, is a big disappointment: I never expected it to play in the same league as x264, but even I didn’t think that it would be worse than even Baseline Profile" -- remember x264 Baseline can beat Apple H.264 High Profile amongst others). Particularly with them claiming good improvements between 1.1 and the in-development 1.2 that already exceed the jump demonstrated in 1.1 over the initial 1.0 release. He tried to use the latest Theora code for this test but got the wrong branch unfortunately.
He also, bizarrely, commiserated that Theora is in the same class as the "old" Xvid, which beats the (again very, very good) x264 Baseline at the higher bitrates. One of the better thought out codec tests I've seen, but his commentary is weirdly lacking in context. On that note I've just noticed that he, incorrectly, described Theora as an "MPEG-4 derivate" (like Xvid and DivX) in the intro when you could say that not deriving from patented MPEG technology is perhaps a fundamental feature of VP3 and perhaps the whole point of Theora.
Why compare with Baseline, though, and not Main or High? In e.g. the "desktop computer playing the video through Flash" scenario there's no client-side reason not to use the more advanced modes. The iPhone is limited to Baseline to be able to use accelerated decoding, but is that also the case for other embedded systems such as set-top boxes?
In the case of YouTube et al there's the server-side performance/encoding time consideration, but they should be able to encode everything in a quicker format, e.g. Baseline, and re-encode popular content in High profile as needed.
Well it was aw3c2 who originally made the comparison that you responded to, so you'd have to ask them to find their reasons.
For myself, I'd suggest that Baseline might be the XP of H.264, good enough for most folk and with a big installed base out there. (Main profile would be Vista in this analogy as many people seem to suggest just skipping straight onto High Profile if Baseline isn't appropriate). Would you like to provide one set of images on your blog for most users and another set to iPhone users? I'd guess the same applies to video. Though of course this isn't an issue for centralised hosts like Youtube that host multiple bitrates, profiles and codecs.
It will be interesting, for example, to see if the next-gen iPhone allows you to play High Profile, I think the current 3GS has got enough oomph but is software limited by Apple to baseline, presumably to keep the iPhone line a homogenous target platform to some extent (or to avoid the hit on battery life making them look bad?).
Speaking of XP I'd also love to know what levels of H.264 video many existing computers, especially netbooks, can handle. I'd guess Google is the kind of company to gather this kind of info, and they have an obvious interest with Youtube, but I don't know if they've released any of it publicly. They are famous for conservatism in their encoding settings, though I believe they use High Profile for their 720p encodes now.
From what I know Baseline is what is usually used for SD on the web, so it just makes sense to compare to that. Also many people say "mobile things do not have hardware support for Theora but for H264" and that, as you said, mostly means those devices have Baseline support.
This is an hard pill to swallow for companies who have profited greatly from the interoperability that such licensing affords them.
it is not acceptable to basically bar open source from a huge part of the web
Who exactly is doing that? H.264 is not proposed to be part of the HTML5 spec. It's just what people use.
flash is not part of the standard web ecosystem
In the sense of de jure standards, neither is H.264. In the sense of de facto standards, it is even more of a standard than H.264.