Part of the problem is San Francisco's (and Phoenix/Tampa/etc/etc) fascination with rail. They spend billions and billions of dollars to construct and maintain rail systems that simply don't attract that many riders. And of course since the fares don't come close to covering the cost of the rail, the cities are forced to suck that money away from other areas like buses (which are far cheaper and have the added benefit of being more flexible). So, cities like San Francisco end up with monstrous rail costs and a resulting inability to fund fundamentally better transportation, like buses.
I'd quibble a little with that, or maybe just clarify; there are two kinds of rail systems in the Bay Area -- local (MUNI in SF, VTA in Santa Clara) and regional (Caltrain, BART, Capitol Corridor). I really think regional rail serves a purpose. For all the tomatoes we tend to throw at BART (some of which it certainly deserves), it's a big contributor to keeping cars off the road -- or letting people get by without cars -- for trips that you really couldn't walk, such as between SF and Oakland. When I worked in SF and lived in Foster City, I was taking Caltrain in daily, and while that wasn't without problems it was a hell of a lot nicer than driving -- and once you figure in not only gas prices but parking fees, cheaper.
On the local level, as much as I've generally preferred to ride trains rather than buses, I think you're right -- although buses are helped immensely by having dedicated bus lanes for main arteries, and I'm not sure how feasible that would be in SF.
The problem isn't rail. Chicago and NYC have perfectly functional systems built on rail. The problem is that the rail doesn't go anywhere you want it to go.
You don't see this as a problem? I find that akin to saying "The Soviet Union had a perfectly functioning economy. The problem was just that half its citizens were starving."
One of the best aspects of buses, aside from the cost, are they can go anywhere you want. If a lot more people move into the Mission, you can spend a few million dollars and double the number of buses (as opposed to literally billions of dollars to build a rail line). If people move away from the Tenderloin, you're not stuck with billions of dollars of infrastructure that now isn't needed. Just move the bus routes.
Also, lets look at NYC, one of the most successful rail systems in the world. As OP states, it costs nearly $14B a year to operate and only generates $7B a year. That means people who don't use it are paying $7B/year to the people that do use it. I wouldn't call this "perfectly functional". And as I mentioned above, the more rail you build, inevitably the less resources you have for more efficient and economical transport.
It operates at a loss of $7B, but that $7B is buying at least one thing: people traveling underground instead of aboveground.
Does this reduce congestion, traffic, or pollution? Maybe it allows for increased density or more sensible housing/business districts? More reliable commutes?
Given that the population is roughly 8 million and the gdp is roughly $1 trillion, it seems possible that NYC is coming out ahead on that $7B.
"The rail doesn't go where you want it to go" comment is directed at Muni, not Chicago's or New York's system. Chicago's and New York's largely goes where people want it to go because development springs up around the transit.
The problem with buses is capacity. If you city is relatively sensibly laid out and commuters tend to follow certain predictable routes, you can get far more capacity on a rail line than on roads. Not only does a bus carry a lot fewer people (an NYC reticulated bus can carry about 120, a full NYC subway train about 2,000), but as a practical matter you can't run the buses as fast or as frequently as you can run trains.
Predictability is also a problem. Buses are at the mercy of street traffic. Trains run on a schedule. My commute from northern Virginia to d.c. by highway used to take anywhere from 40 minutes to 80 minutes depending on traffic. My commute from westchester to new york city by train (which is actually 2 miles longer) takes exactly 34 minutes almost every day. About once a month a train will be 5-7 minutes late (which is consistent with the ~95% on-time performance of Metro North). I can leave my house 5 minutes before the expected departure time (with a 3-4 minute walk to the station) because I can count that the train will arrive within a minute of its scheduled time. You can't replicate this by car or bus in a dense metro area.
Pollution is another problem. Air pollution has huge externalize costs in a city because of the density. With electric rail, you can build the power plants out in the country where the pollution affects fewer people. With buses, you're clogging up the air breathed in by several million people in close proximity. Buses are prime contributors to sulfurous and particulate pollution in cities.
Moreover, it's not like bus infrastructure is free. You have to build and maintain the roads, and if you want to replicate the capacity of trains you need to build isolated bus highways through the city. These are not cheap, and have a very damaging effect on the communities they run through by cutting them in half. Highway construction inside a city is an absolute terrible idea from an economic point of view.
It's wholly inappropriate to compare the operating costs of transit to revenues. First, the shortfall in the MTA isn't because it can't charge high enough fares, but because the ticket prices are kept artificially low ($100 for unlimited rides per month!) as a subsidy to low-income people. Second, transit infrastructure generates high positive externalities. You have to look at the net impact on the whole economy instead of looking at just a piece of it. E.g. consider the highway system. It brings in a few tens of billions in revenue each year via gas taxes. Is that the whole of the economic benefit of the system? Are people like me, who don't drive, subsidizing people who do? Imagine if we got rid of the highway system. What would be the impact on GDP? A hell of a lot more than a few tens of billions, I can tell you.
Exactly. If the commissions that regulat how many new housing units were both competent and not corrupt, I reckon the rail options would be far more profitable.
People don't move away from convenient transportation. One good transportation is built the rent around the stations goes up because of the desireability of being near public transportation. The only exception I know of where public transportation lines are shut down because demand for them dropped are in Detroit, and that has everything to do with people moving out of the city entirely and not just moving to a place across town.
Gotta disagree here. Muni is centralized in downtown, which is where most want to be. It then shoots out to cover dense residential neighborhoods. Some possible lines may be missing, but other than maybe the T line, all existing ones are important.
As any rider can attest, it gets pretty busy. Perhaps you are thinking of SF's southern neighbor, San Jose, as an example of a highly underutilized light rail system?
Also, a lot of muni metro is underground, which is essential for keeping in fast beneath the high traffic volumes above.
