I'm the author. A question I have is: how did so many prices ever get negotiated in the first place? What kind of systems are in place to do this kind of micro-negotiation?
The US healthcare system is wildly complicated and inefficient because it is a double-bureaucracy; pubic and private. The government bureaucracy makes a bunch of rules and also provide healthcare through Medicare/Medicaid. The private bureaucracy compete with each other, and hospitals, and pharma companies, ect.
Many of the private health providers are for-profit and lobby against rule changes that would reduce complexity and save the system money. It know this may sound glib, but if you are trying to understand the US healthcare system and something seems strange, usually it's because it makes someone money and they'll fight hard to keep it that way.
> Many of the private health providers are for-profit and lobby against rule changes that would reduce complexity and save the system money.
This is almost certainly an anti-competitive move. By keeping many rules and regulations, you need more staff to deal with them - and smaller insurers have fewer patients to amortize those salaries over.
The Canadian system is a mix of private and public, with a mix of national and provincial funding, some national standards (tied to funding), and provincial jurisdiction.
For procedures covered by provincial health programs, doctors in private practice (GPs, at least some surgeons) give the service to the patient and then bill the government for that amount. For services that aren't covered by provincial health programs, GPs/surgeons bill the patient.
Provincial health care programs cover anything medically necessary. My general understanding is that you cannot pay to have a service done which is covered by a provincial health program. The intent is to prevent the establishment of a two-tier health care system.
It's probably less complicated than the US system, but it's still complicated.
What was negotiated was probably more blanket style discounts like "10% off your published medicare rate for procedures in categories a/b/c" for one customer and "15% off retail price for all categories other than x/y/z but only in these geographic areas" for another customer, and so on.
But, when publishing, they omit the context and just dump every negotiated rate. Because it's technically compliant, but keeps things opaque.
This is probably the data as they have it and instead the people responsible for inputting things in to whatever system this data came from just have a little print out taped next to their screen reminding them customer XPN305 has a 10% discount on codes that start with FJR and so on.
Any time you can convert a problem from an `N x K` problem to an `N + K` problem, there's some asshole administrator trying to turn an `N + K` problem into an `N x K` problem. It wouldn't surprise me if there's huge amounts of redundant information in there.
> In the newly-released data, each "negotiated rate" (or simply "price") is associated with a lot of metadata, but it boils down to: who's paying, who's getting paid, what they're getting paid for, plus some extra fluff to keep track of versioning. The hundreds of billions of prices in the dataset (probably over a trillion) result from all the possible combinations of these things.
They basically denormalized all the dimensions.
Imagine you have a function which takes 5 arguments and returns one value. You could give me the source code and let me run this function. Or you could give me a mapping of every possible combination of the 5 inputs to the returned value. The former could be quite small, but the latter would be a massive number of rows.
> You could give me the source code and let me run this function
If I understand correctly, in this case, that function's source is highly distributed in wetware. It's about as closed-source as it gets; nobody has anywhere near the full source. Each hospital is its own fiefdom!
It would be interesting to see whether it's possible to reconstruct the rules by comparing the negotiated prices to a baseline like Medicare price, and doing classification into discount buckets to recover the categories insurers negotiated.
There is not a universal practice. Predominantly just excel, a lot of emails, conference calls and meetings. Different institutions have distinct personalities and that reflects in how things are done between them. Some people still have actual mainframes involved.
When it comes to large parties, multi-practice groups, health systems, etc, an overall fee schedule or charge master for an existing institution is typically not renegotitated line by line every year but as incremental changes from the previous. Many/most of the parties involved have been working together for decades, some even longer.
Many plans administered by familiar names like Anthem are actually funded and controlled by the large employers the plan services. In those cases the employer plays a role in defining what will and will not be covered and what will be paid and the insurer is a middleman (acting as a third party administrator).
Today in Massachusetts, physicians cannot get paid unless they belong to an organization that negotiates their rates with the insurers. These negotiating entities are like unions but not really. If the insurer and the organization disagree, the insurer simply goes to a different organization to make a contract. Prices were not publicly available so each negotiation resulted in a different fee schedule. On top of that, insurers invent different 'products' with different amounts of 'coverage' for different premiums. Each of these 'products' had their own negotiation, their contracts, and their own subset of physicians who chose to participate. So what do these organizations do for the cut that they take? They reduce the burden of the insurers so they dont have to negotiate with each individual provider.
Hospitals are an entirely different system. They have much more negotiating power and if an insurer has a customer that goes to a hospital emergency room outside of their contract, the insurer has to pay outlandish rates. So it is in the insurer's interest to make a deal. They achieve this by inventing different 'products' with different amounts of 'coverage' for different premiums. Each of these 'products' had their own negotiation and their contracts.
Price transparency is the first good thing that has been mandated. However, this misses the mark. The focus is the patient, not the insurer, the hospital or the physician. Accordingly, patients should be allowed to submit their explanation of benefits and their bills-this is the data that reflects the true cost of healthcare. All of the numbers provided by hospitals, insurers and physicians has been massaged and buried in a forest of minutiae.
They aren't negotiated individually. They are negotiated categorically. They generate individual prices based on some discount rate off of a negotiated max.
Are we sure they negotiated unique prices with each provider? I wouldn't be surprised if they have a dozen of templates that get replicated every time a new entity accept preexisting price sheet. Basically they have dumped a denormalized data set.
My understanding (not an expert by any means) is that we basically have two tiers of negotiation - the fed. govt. has way more leverage but also some amount of corruption that goes into pricing, then afterwards individual hospitals and "networks" of providers will negotiate with the insurer - sometimes after the procedure has already happened - to figure out the final price.
The end result is that you might end up with an individual doctor having to work with the insurance company for pricing, so the same procedure can cost vastly different amounts at hospitals down the road from each other providing the same level of care. To make it worse we also have laws preventing healthcare providers from providing prices upfront, out of a fear that people will forego necessary care they can't afford.
> To make it worse we also have laws preventing healthcare providers from providing prices upfront, out of a fear that people will forego necessary care they can't afford.
What are these laws? This seems so backwards - I know personally I have put off medical care in my past because I had high deductible insurance, and no guarantee that the bill I'd get wouldn't wipe me out, and no way to price shop. Paralysis of unknown.
Now assuming the hospital is compliant the information should be available. To be fair my understanding of the argument for the old law was that you didn't want a hospital with a big sign out front saying "Broken arm repair: $10k" and having people not go in for it when there might be some financial aid they could get afterwards
I'm sure the real reason is that the hospital up the road will set up a sign "Broken arm repair: $9k" to compete and that isn't something the lobbyists want.
It’s not the case. The No Surprises Act requires a good faith estimation for most procedures. Although IIRC it doesn’t apply to people who don’t have insurance, which seems kind of backwards as those people would likely be the most price-sensitive and have the least amount of bargaining power in the market. I guess they also tend to have the least amount of political power, too...
It's an entire pipeline of billing to extract the tax benefit of insurance.
The real economy has way many more prices than this one - from each store of anything in the country that negotiates from straws to bread. The difference is that these ones happen in a system that has a paper trail from the doctor, to the insurance, and this admin burden is only (apparently) worth it because the vast majority of money in healthcare goes through tax-advantaged insurance.
Cash based payment should suffice for 50~70% of healthcare expenditues and it would have more prices and not have expensive and abusive billing processes.
The next question would be 'how long did the average negotiation take' followed by 'how much were the average people on each side of the negotiation paid?" (or are most of these negotiations the result of computers talking to each other? Either way with a few assumptions one could make an estimate of the smallest amount these different prices cost the system. Might be huge
And I, as an Australian citizen, am wondering why we don't have this mess. Afaik in our system, each insurer just has one flat price for how much they cover of each procedure. And each provider has a flat price for how much the procedure costs. No individual negotiations between providers and insurers.
Also! What did they do before they could store 100TB of pricing data? How has pricing (and care quality) changed as a result of being able to do this type of thing?
Possibly the original data is logically compressed. E.g. payer A pays 110% of our standard rates, payer B pays 85% of our standard rates. Those two rows could translate into thousands of CSV lines depending on the number of procedures.
Maybe you have a couple one off negotiations for high volume procedures, but even still the source data could be several orders of magnitude smaller than the dumps.
this would be a great social study. cases where technology has enabled the racketeering and price gouging by corporations with almost no gains in efficiency or output or quality or any metric of value.
I would assume these get negotiated as a large list with each payer so if you have 500 services, and 4 payers, you probably and up with 1k-2k unique prices?
I'm author of Hacking Healthcare for O'Reilly, 20 year health system executive, blah, blah.
It's very easy for people to forget the scale of the US "health system", we are talking 1/5, maybe more, of the entire US economy. If US healthcare spending were a country, it would have the third largest GDP in the world. Accidents of history and the massive federal beauracracy created the crazy monster of ICD/CPT codes that results in the very clumsy way of pricing healthcare services that results in this massive matrix of data.
As pointed out elsewhere there is a tremendous amount of cost distribution that goes into the code matrix and this plays a large role in negotiations with health insurers as well. Ground is given in one set of procedures and lost in others.
This is a big step in shining light into areas that need it to improve the system overall.
Do you consider the amount that the US spends per capita on healthcare relative to other countries for the same standard of care a "failure" of the healthcare industry? Or is there some other reason healthcare "just costs more" here?
Also wondering what you think a solution is - single-payer for better and simpler price negotiations, or some other approach?
My main concern is if we're spending 20% of GDP on something other countries accomplish with 10%, then that's a huge waste, especially in a country with a larger total GDP pool.
Healthcare is such a base layer of the economy, I find comparisons to be extraordinaly difficult between countries. On the most basic level our pathway to becoming a healthcare provider of all sorts is dramatically more expensive and limited than other countries, what healthcare providers are paid is dramatically more than other countries, we invest many times per capita what other countries put into basic medical research, the way are population is taxed is very different than other countries, our patient population is very different from other countries, our expectations are very different from other countries, our scale is dramatically different than other countries, and so on. The US is a singular animal politically in that it is a compact of individual states that especially in regards to healthcare, the federal goverments powers (though it may not seem so at times) are actually quite limited. It's all but impossible to come up with reasonable numerators and denominators for comparison.
They're most obvious with "basic" products like energy and comms - in theory what is delivered is mostly undifferentiated kWh or GB but through the magic of "confusing plans", marketers have succeeded in making comparisons very difficult for regular people.
(You can do it, but you need OCD, a year of billing data and a spreadsheet - which greatly exceeds the cognitive effort most people will invest in choosing a mobile or energy provider).
The US healthcare sector seems to be the largest, most intricate and most successful (in terms of gdp extraction) confusopoly in history.
To be fair, energy isn't just kWh. A Joule delivered during times of peak demand costs more to produce (or has a larger opportunity cost) than a Joule during a lull.
Also a marginal Joule that you can demand at will is different from one that you committed to months in advance.
Similarly for data.
Of course, in practice most plans don't reflect this 'essential' complexity, but are full of accidental complexity to confuse people.
...because the healthcare corporations get congress to sell out citizens. $3.5 BILLION flow through lobbyists every year (all industries). Healthcare being a huge part of that. Congress sells out US citizens & corporations fuel it
Off-topic but "confusopoly" maybe Scott Adams true contribution to human understanding. IIRR he wrote a comic management book and dropped in a throwaway line and invented the term.
Academically this sounds enlightening, but it only takes one cursory walk around a supermarket in the US to see this is unequivocally false. Healthcare is an externality, not a base of anything. From the average customer to the product in the aisle to the marketing - everything is 100% not a direct cost benefit function in terms of healthcare.
Walk around the supermarket that you drive to, hopefully not being maimed or maiming someone else on the way. A ludicrous expenditure of energy to avoid physical activity so that you can buy products to help alleviate the symptoms of the energy expenditure and laziness. Can’t walk, or bike, or take the non-existent public transit. That’s for sure.
I don't think driving is merely to avoid physical expenditure. Driving is a result of rural and suburban living as well as poor urban planning. Lots more people would walk or bike if it was reasonably feasible.
I suspect it's a bit of a chicken/egg problem (based on my opservations from visiting the US). Even in suburban areas there seems to be roads everywhere, often very hard to cross without a car.
Where I live in Europe, the expectation is that the kids will walk or take the bike to school. 95% of the paths they need to travel is covered by walkways (usually separated from the roads, and in many cases with dedicated bike lanes). Everywhere the kids need to cross, there are crosswalks, and every morning before school a team of parents is organized to safeguard these crosswalks.
Meanwhile, driving is slowed down significantly by the efforts above. The roads are underfunded and lines tend to form around the school (some parents still do drive their kids, up to about half when the weather is really bad). But with parents blocking the crosswalk every time a kid is near, bottlenecks do form. Basically, if you're in a car, you're treated as a 2nd rate citizen. :)
Oh, and I suppose one benefit of all this walking is that hospital bills go down. Mostly because of the exercise, but also to some extent due reduction in air polution (which is helped further by most of the remaining cars being electric, most of which, ironically, are of a certain American brand).
Where i live in the suburbs we don't even have sidewalks ffs. I live about 3 blocks from my kids' school, yet they are driven or take the bus to school because kids are not allowed to bike or walk to elementary or middle school, unfortunately.
wow, didn't realize so much of EU can afford electric vehicles. They are more expensive over here in the U.S. - especially compared to a decent used car. Is this a function of the cost of fuel being much higher over there? Or are there subsidies for purchasing electric or something else?
> wow, didn't realize so much of EU can afford electric vehicles. They are more expensive over here in the U.S. - especially compared to a decent used car.
I don't live in the EU, though. You need to move a bit further north and west.
And it's not that electric cars are cheaper here than in the US, rather other cars are taxed at +100% or so (more for "luxery" cars), while electric cars have had low to zero taxes.
Also, fuel has an extra tax of about $1/liter ($3.8/gallon) on top of the normal price (and 25% VAT even on that), meaning even before the current boom, fuel was typically priced around $1.5-$/l (around $5-$7/gallon), and has been approaching $2.5-$3/l (up to about $10/gallon) recently.
Another difference compared to the US is that we have about 25% fewer cars per capita here, despite having about 20% higher nominal GDP per capita than the US (purchasing power of households are lower than in the US, due to taxes and tariffs ). Partly because of car-hostile taxes (except for electric cars) and partly because other means of transportation are subsidized. More of the money is put into buses, trains and sidewalks and less into cars, roads and parking lots.
I'll concede that the comment was vague and relied too heavily on a shared intuition. Though I will admit, the reward became well played dry comedy throughout your sibling comments.
So I'll break down my reasoning a bit. It requires a full blog post to get out, so please forgive the abridged version.
Healthcare is a catch all for all the other problems of society. The top costly conditions in the US are (in order of this barely sourced article): Mental Disorders, Heart Conditions, Trauma-Related Injuries, Diabetes, Cancer, COPD and Asthma.
Every single one of these is plainly racking up unneeded costs by the daily actions of all of us. My quip on the supermarket was a remark on the total view of health (from mental to reproductive care to basic carcinogens to ...).
How many people are mindlessly scrolling on Instagram while performing another task, how many people smell of cigarettes, the marketing of 'sinful goods' (depending on the state), the near impossibility to avoid added sugar in every packaged foodstuff, the number of 'alcohol noses' you can see down a 50ft isle, the parenting of children, the smell of fossil fuel exhaust from the parking lot, the gait of the elderly, injured, or soon-to-be, the accommodations (or lack thereof) for those in wheelchairs and with living assistance, and still the primary food at checkout - And to include everyone in the conversation: think of the anyone working two jobs and has 0 time to prepare fresh food for themselves or anyone else, the eventual cost is in the habitual behaviors made in the constraints of under-compensated labor.... I could keep going and I've left out other observations contributing to other conditions but I think you understand.
The thing I'm trying to say is that there are interventions all over the place - however, the up-front costs (ignoring all else) of a 'double blind randomized trial' for every single one of them to earn the proper authority to define its relative utility to cost is unrealistic at the moment (also most governments do not allow for risk based price of care) - an economic externality.
Couple this externality behavior with a market of near perfect inelasticity for good health (and before someone comments, yes, suicide / assisted euthanasia may not be inelastic in price on this metric) - and you can't say "Healthcare is such a base layer of the economy" - an alternative analysis is "Healthcare is an externality that is priced in a government controlled market"
[Edit] I completely left out the externality of the high reward litigation industry on malpractice and all of the above conditions as evidence of harm - adding pressure on compensation to the highest paid professionals.
Yes, most governments do not allow pricing according to ricks; but, whether insurance will take that risk group and insure them does in fact act as pricing according to risk. An example is diabetes. As if you are not with an elite insurance carrier via elite high income job you have actually not having insurance and dying from diabetes due not being able to afford insulin shots.
It is offset at all by all the workers who are forced to sit their entire shifts? It's probably better on our bodies to stand than to sit for 6 hours at a time, but it starts in schools and ends at desk work. Standing desks are nice, and can help, but not everyone has access to them.
One of the nice things about working from home is that it allows people to escape that kind of environment to a point, but it just enables other types of obsessive micromanagement like "Why hasn't your mouse moved for 15 minutes!" or keeping cameras pointed at you all day long.
What we really need is less micromanaging and an expectation that not everyone is going to be at their desk every minute of the day, but that's a very hard sell in some environments.
> It's probably better on our bodies to stand than to sit for 6 hours at a time, but it starts in schools and ends at desk work.
It definitely isn't, standing puts enormous strains on our bodies. Walking for 6h is much easier and better, but sitting out lying down are much better than just standing.
They're also talking about a population of workers - this includes elderly people with bad hips, bunions, handicapped people who aren't always given proper accommodations, etc.
> our patient population is very different from other countries
> our expectations are very different from other countries
> our scale is dramatically different than other countries
> the way are population is taxed is very different than other countries
do you have evidence to support these claims? what makes the US patient population or their expectations or the US taxation system unique in the world?
Not the OP, but Americans as a whole are very unhealthy (with 42% of the population being obese and over two-thirds being overweight) and culturally have very high expectations of what medicine can do for them, as opposed to making difficult changes to their lifestyle.
A part of the latter is based on the actual superiority of the quality of medical care in this country -- due to the high levels of wealth produced by this (mostly market-oriented) economy and advanced medical technology, doctors can in fact perform miracles here that they cannot elsewhere.
A part of the latter is based on the actual superiority of the quality of medical care in this country...
How is it superior? Sure, some countries fare worse. But folks aren't getting the healthcare they need because of cost, and the results aren't exactly the best in the world. I'm not convinced that "culturally" folks have high expectations either, and sure, you might want to change your lifestyle to lose weight - but at the same time, you might just need medical oversight to do so. Not to mention that a bunch of things medicine helps are not things that lifestyle just fixes.
Yes, folks are being priced out of healthcare, but the healthcare that is being provided is of superior quality than can be found in other countries -- even first world countries with socialized healthcare. I mean it in that narrow sense, that the service that is being delivered is of higher quality.
It sounds like your point is that wider delivery of healthcare would be superior overall. That's fine, but I contend that the best way to achieve that is by increasing the supply of healthcare providers, instead of applying a price ceiling, which leads to shortages (as seen elsewhere on this thread[0]) and quality deterioration.
> e invest many times per capita what other countries put into basic medical research
This is a big thing. I'm in the UK, where healthcare is very socialised, but I very much appreciate the fact that the US invests in making and productionising the next generation of healthcare, which we can then buy in bulk at a discount.
I understand you're the relative expert here, but even so I must disagree with your general thrust.
I've been hospitalized in four different countries. The least sane was America. The sanest was a private hospital in England, but the public hospital in England was fine too. My home country of Canada is sane, reliable, and reliably slow and mediocre bordering on subpar. Cyprus lacked toilet seats, but at least the food was fantastic.
America's healthcare system is bananas. Even trying to come up with a metaphor here is difficult. It's $5k a day stays with Wonderbread, tuna, and bad not-actually-mayo-mayo for lunch. It's well groomed, well respected, monied indentured second and third opinion servants. It's Moloch's own mediation on Moloch[0] sold on the discount rack of the bookstore pharmacy downstairs.
You can think Americans are different. They are not. They move to Canada all the time and we service their bum knees just fine.
You can think Americans do all the medical research in the world. They don't. Plenty comes out of Europe, China, and elsewhere.
You can think your tax code is unique. Ok this one I kinda agree with. It's almost as bananas as your medical system. But it doesn't change the fact that Americans put up with absolute bananaspants insanity for a healthcare system when they're perfectly capable of funding their libraries and roads.
China beats you on scale. And so on.
The basic fact is that Americans have what is essentially a psychopathic medical system at the best of times. One can negotiate with a psychopath, but Kafka returns your offer with a can of stale soup and doesn't even laugh.
If you mean PPP, then the reason is simple. How much a janitor is paid to clean a research lab's washroom doesn't materially affect the quality of the research produced. There are other measures that look at things like papers cited or page rank like algorithms, and on those USA does even worse if I recall correctly.
At the end of the day, though, this is a distraction from the core argument. A valid defence of US healthcare policy is not "but we're good at research." Imagine if China was trying to defend their overbudget and under-effective military by talking up how much they've done for global aeronautical research.
Grandma lost her house because she got kidney stones, but, butterfingers, at least we research stuff! Oh and often times our pharma kills more people than it saves and true justice is never metered out ala opioid epidemic.
There's no reason to come on here and be nasty to someone you've never met because you or some relative of yours had a bad experience.
If it's that big of a deal for you, work to fix it and you can come on here with your credentials and blah blah instead of whining about people who have experience in what they're talking about :)
Civility is a little overrated and what an obnoxiously condescending response to say well go fix it yourself before you talk. Go talk to children like that
About 90% of Americans have medical insurance. Coverage caps were eliminated years ago, and the out-of-pocket maximum is low enough that the medical expenses from a single incident are unlikely to drive middle class people to bankruptcy.
Where bankruptcy becomes an issue is when a medical condition leaves someone unable to work for months. With no income they can burn through savings and credit quickly. Then when they file for bankruptcy of course they have some medical debts, but those medical debts are typically not the primary cause of the bankruptcy and even if the medical debts were eliminated they would still be insolvent.
That's a good point, I may have overstated the issue. However, from what I understand it is not too difficult to find oneself being treated by somebody outside of their insurance network. This could easily lead to a bankruptcy. Furthermore, many do not feel comfortable or confident navigating these byzantine insurance landscapes. This leads to people avoiding medical care regardless of insurance status.
Number one is full price transparency of the whole chain. I work for a medical device company and even the marketing people can't really tell what our stuff costs. There are a ton of middlemen with obscure contracts and very high markups. My ex got one of our devices and I was told by our people that the hospital should have received the device for between 20k-30k (nobody seems to really know) and the hospital charged 80k for the device alone. They also charged another 200k for a one hour surgery with a total hospital stay of six hours.
It's also hard to explain that US patients pay a multiple of the drug price people in other parts of the world pay for the something.
The problem is that if the US wastes 10% of GDP on health care inefficiencies this creates a huge lobby that will fight tooth and mail to keep that money.
The prices as charged in the US regularly make it onto Twitter and Reddit etc where us Europeans wonder how on earth it's possible that something in the US costs $ 800 which is charged at less than $ 100 over here (and then paid by insurance). Same for that $ 30k device being sold for $ 80k.
What we all forget is that nobody is actually paying the US healthcare invoices.
Roughly two options... 1. You have insurance and they negotiated a different (much lower) rate or 2. you don't have insurance and can't pay the amount on the invoice.
In option 2 you either declare bankruptcy and they get nothing or they sell the claim for something like 20% of the invoice to a collections agency.
Not correct. The insurance paid 80k for the device and 300k total for the surgery after one year of fighting almost daily.
Also: if you make a certain amount of money and they hit you with a 100k bill you can't just declare bankruptcy. The court won't allow you to do it because you make too much money.
> you can't just declare bankruptcy. The court won't allow you to do it because you make too much money
Isn't this always the case? My understanding was that bankruptcy is for when you don't have enough income to pay bills, so if you have a bill for any amount but are able to pay it then you wouldn't be allowed to declare bankruptcy
Google search for the McKinsey report on US healthcare spending - I think it was around 2009.
I work in the industry as well and it’s one of the few reports that actually breaks down the spending in a logical way.
They basically adjust US spending by GDP (high GDP countries spend more generally) then compare each category to the OECD average (also adjust by GDP), on a price and volume measure.
The answer is - yes, higher price are a factor, but volume is also a major factor. In hospital spending is actually in line with other countries. Drugs costs more but it doesn’t contribute that much to total spending. In terms of durables (equipment) the US spends less.
The biggest driver? Out patient procedures. Not just price, but Americans get way more out patient procedures done compared to other countries and it accounts for like half of the “excess spend” of the US compared to other countries.
The US is (exaggerating a bit) a nation of obese, sedentary substance abusers. We are sicker on average than other developed countries and thus have a higher demand for healthcare.
We might be able to eke out some minor improvements by tweaking the payment model and eliminating some waste. Those things are worth doing, but they won't fix the fundamental problem. The US won't get healthcare spending down to Japan's levels until Americans start acting like Japanese.
There are some other key factors as well. A large fraction of healthcare spending goes toward treating elderly patients with serious chronic conditions in their last few years of life. Some countries explicitly deny care to such patients because they don't think it's justified on a QALY basis, but Americans seem uncomfortable with rationing on that basis.
And some aspects of the US healthcare system are top notch. For many types of cancers we have the world's best 5-year survival rates. There is a thriving medical tourism business where patients from countries with socialized medicine such as Canada come here to receive rapid treatment instead of waiting for years for something like a hip replacement.
> Do you consider the amount that the US spends per capita on healthcare relative to other countries for the same standard of care a "failure" of the healthcare industry?
That we spend more per capita for approximately the same level of care as most other first world countries is certainly annoying. But sometimes I think we are too focused on that and not putting enough effort into trying to stop the cost from increasing.
I think increasing costs are a more serious problem because the problem of spending so much more than the others is a US problem. That suggests it is just something we are doing wrong, and by making our system more like some of those others we can fix it.
The problem of rising costs also plagues those other countries, and to about the same extent as it does the US. That suggests it is a much harder problem to solve.
Here are some examples of rising costs per capita.
How much costs per capita went up from 2000 to 2018: US 2.3x, Germany 2.1x, France 1.8x, Canada 2.0x, Italy 1.7x, Japan 2.6x, and UK 2.6x.
Costs per capita in 1980, 1990, 2000, 2010, and 2020 divided by 1970 costs:
Data doesn’t make sense. If US costs went up similar to other countries but US is significantly more expensive than other countries today, does it mean costs in US have always been much higher?
Yes (at least as far back as the data at oecd.org goes, which is 1970).
In 1970 US health care spending was $327.0/per capita. France was $192.1, and UK was $124.0. That's 1.7x France and 2.6x UK for the US.
In 2020 it was $11859 for the US, $5468 for France, and $5019 for the UK. That puts 2020 US spending at 2.2x France and 2.4x UK.
From 1970 to 2020 US went up 36x, France 28x, and UK 40x.
It looks like much of the first world has a serious rising health spending problem, with costs rising roughly the same over time everywhere. The US was more expensive long ago, and since the rising costs have been roughly the same the US has stayed more expensive by about the same ratio.
If we could get our spending down to match the rest of the first world, without reducing the level of care, that would be great.
BTW, it is similar if we go by percent of GDP instead of per capita.
US was spending 6.2% of GDP on health in 1970, France 5.2%, and UK 4.0%.
In 2020 that was 18.8% of GDP for the US, 12.2% France, and 12.0% UK.
I’m not the OP and have no deep knowledge, but I’ve often heard cited that the US out-researches other nations, so we incur “R&D” costs for healthcare that other nations use. Eg pharmaceuticals are researched in the US while the patents are used in other nations through a cost structure that doesn’t allow the original researching party to recoup costs.
On-shoring that research also seems to be an advantage -Looking at the astounding amount of research that poured into covid post 2020 would show that we have a huge dormant muscle that can be flexed in unison during an emergency.
I’m not trying to defend pharma companies… they’re generally pretty scummy. But I’m guessing most companies spend more on marketing than R&D. Beyond that, the theory that they need to recoup costs still holds true with this. In fact, a big marketing budget indicates that they’re aggressively trying to sell the drug (maybe to recoup costs?).
Generally in business marketing budgets should generate more sales than they cost (in ideal case), so big budgets doesn’t mean that they’re “wasting” that money that could go elsewhere. If the sale wouldn’t happen without an ad, then that’s a necessary ad.
> If the sale wouldn’t happen without an ad, then that’s a necessary ad
I think that's the core issue here, healthcare and pharmaceuticals have basically inelastic demand. The US is one of only two countries where it's even legal for pharma companies to advertise directly to consumers.
I know in practice they can create demand for products, but that doesn't necessarily seem like a good thing, so I think you could argue that it's still a waste of money even if it does create profit for the company.
There's always complaints about healthcare and probably no perfect system, but a bad one where everyone has coverage seems a lot better than a bad one where everyone doesn't
> if we're spending 20% of GDP on something other countries accomplish with 10%, then that's a huge waste
I don't think that money is necessarily a waste if it goes back into the economy one way or another. There are very few things that are actually a waste, one example is probably flying first class or private jet. If you literally burn money then it's clearly a waste. A part of me thinks the huge cost of healthcare is contributing to more R&D by the big pharma and possibly the reason we're seeing RDNA breakthroughs. Yes a lot of that money also ends up in the pocket of people running the show, but they most likely then invest it with a Blackrock which in turn pushes the money back into the economy in form of private equity, VC funds, etc. For the record I don't like the high healthcare prices and wish US was more similar to other countries in this regard.
Is it really the same standard of care? I would speculate that the standard of care in the US is at least marginally higher than many other developed countries based on my admittedly anecdotal experience, especially if you plug wait times for providers into the calculation (this seems to be the number one complaint that comes from people I've met that have immigrated to the US).
There's many things wrong with US healthcare. But somehow the USA comes up with the the first vaccines for the coronavirus and if you want any sort of complex procedure you travel to the US to get it.
So maybe thats where the extra cost goes? To drive research, and support the infrastructure that creates good to better health outcomes on average vs the rest of the world.
Saying something is a failure just because it costs so much is only looking at one side of the coin.
> But somehow the USA comes up with the the first vaccines for the coronavirus and if you want any sort of complex procedure you travel to the US to get it
This isn't true, the US didn't make the first vaccines, it was created by a German company that partnered with Pfizer for trials/production/distribution[0]. People more often travel outside the US for care, we actually have the second highest amount of people leaving their home country for medical care[1], with the top destinations being in South America [1a].
> creates good to better health outcomes on average vs the rest of the world
This would be nice if true, but we spend more and still have worse outcomes in almost every area[2]. The only things I could actually find that are better here is post-op sepsis and 30-day heart attack survival, but in just about every other area it's more dangerous to get care in the US.
I think the preceding comment was operating on the presumption of linear returns from medical research. Since higher medical costs in the US goes directly towards medical companies, and indirectly to medical research after taxes and dividends and stockbuybacks...
Of course the biggest issue of all time is that germs are evolving to survive our antibacterial soap, we may need to develop a large variety of antibiotics.
As with everything it touches, it's the intrinsic failure of capitalism (ofc success for the capitalists / bourgeoisie). It's the amount of capitalism that defines prices. In every other country the more healthcare is a public matter, the cheaper it is for the people.
Healthcare in the US is definitely not driven by the free market. It is probably one of the most regulated industries. Whatever disfunction you want to call out in US healthcare it is going to be difficult to pin that on the free market.
Free market? Capitalism. I know we're on HN but, say the word? Capitalists take a cut. Shareholders of big pharma, insurance companies and hospitals are why healthcare in the US is expensive. Public sector not being monopolistic is why healthcare in the US is expensive. In France, social security reimburses about 70% of most costs. Cheap private insurance reimburses the rest. About 75% of public hospitals and not for profit. Generic medicine being prescribed is the norm. The state naturally fixes healthcare prices because it's monopolistic on healthcare. Same as all public services.
Non free market? Communism. I know we’re on HN but, say the word?
Of course the US market is highly regulated and so the market is not free to lower prices. Of course the AMA is a racket. Of course needs of certificate are abhorrent.
Given the customer non—coerced access to his preferred provider, and not taking his money and slapping a bunch of regulations on him will of course lower prices and give him better care.
That's right, communism. Social security in France is literally a communist system, founded by a communist minister. Hence why neoliberals want to destroy it.
People are rarely satisfied with this answer but its demonstrably true and was proven time and time again at the facilities ClearHealth managed.
1) Feverent, almost religious, adherence to hand washing.
2) No neck ties or dangly sleves whatsoever in buildings that house patients.
3) Change from stainless steel hardware for doors and travel touch surfaces back to "brass/copper".
Those are simple, virtually free, things that have a very meaningful impact on outcomes. Some of the most viscous fights I've had with hospital boards were over what amounted to the "uglier look" of copper/brass.
It is an extremely unpopular topic in healthcare but the area that takes a lot of effort to solve but also has a tremendously out-weighted benefit is reducing preventable medical errors. My opinion after being in healthcare ~20 years is that preventable medical error is absolutely in the top 3 causes of death in the US. The easiest subset of it to resolve is prescription related errors, we have all the tools to resolve those but not the will.
>"3) Change from stainless steel hardware for doors and travel touch surfaces back to "brass/copper".
Because of the pandemic I started encountering doors that have a shoe pull, where you can use your foot to open the door instead of having to touch the handle. I really hope these catch on, but they are still quite rare.
Also stop getting rid of paper towels if you still have manual faucets. Nothing grosses me out more than going to a public restroom with only air dryers, but manually operated faucets that now require you use clean hands to turn off after you turned them on with presumably dirty hands.
Bathroom doors are usually off a small hallway, sometimes a busy one. By design, they don't have windows.
So you run the risk of hitting people with the door. Also, you will still need to interact with the door to open the lock. Having a door that unlocks if you push on it would be a bad thing for people who use the bathroom with their children.
> Change from stainless steel hardware for doors and travel touch surfaces back to "brass/copper".
I have never heard of this. I had to Google it to even understand your meaning. It's eye-opening to learn that different metal surfaces have an effect on the spread of germs.
> Those are simple, virtually free, things that have a very meaningful impact on outcomes. Some of the most viscous fights I've had with hospital boards were over what amounted to the "uglier look" of copper/brass.
Am I the only one who finds copper/brass much more aesthetically pleasing than plain and boring stainless steel?
What about the incentive for non profit hospitals to grow so that they can better compensate leadership, resulting in capital that must be spent on facilities and equipment to retain non profit status. Leading to a spiral?
It is hard to compare details of the systems and outcomes across countries, but surely we can find where the money apent ends up? Construction firms? Doctors? Equipment manufacturers? Hospital administration?
Is there a rule that says a certain percentage of revenue must be spent on a facility to retain non-profit status? It can be spent on equipment and salaries, both of which would benefit much more than upgrading the building to no patient care benefit.
This is anecdotal but the number one complaint I've heard from physicians about patient care is facilities being run and and managed by non-clinical MPH/MHA "business types" whose primary focus is almost invariably cutting costs, increasing physician workloads, and fighting salary increases tooth and nail.
No, but there is a "rule of thumb" that a hospital will prefer private insurance patients to medicaid patients (due to reimbursement), and private insurance patients will go to hospitals with newer and nicer facilities. If you want the elective hip replacement patient, then having a newly remodeled orthopedic ward / office building is critical. Patients probably can't tell one doctor or nurse from another, and hospitals don't advertise on actual quality measurements like staffing ratios...
I've been told credential easing is by far the easiest one to implement. Doctors often do 2-6 years of excess schooling residencies learning areas of medicine that will never be relevant to them. That's 10-20% more time working for existing doctors, and who knows how many more people would enter the profession. Nurses could be empowered to make doctor lite decisions very easily.
> Doctors often do 2-6 years of excess schooling residencies learning areas of medicine that will never be relevant to them.
Where in the world did you hear this? Don't trust anything else that person told you.
Aside from some low-income clinic hours for certain specialties (which is objectively a societal good, not to mention typically specific to a given specialty, e.g. OBs have an OB clinic not primary care) no doctor is spending 6 years of "excess schooling residencies" learning anything.
Med school is 2 years of classes then 2 years of rotations where the students - who aren't yet doctors - do 4-12 weeks of rotations through various core and elective specialties. After they graduate they're now doctors but have 3-6 years of specialty-based residency training where for 80-100 hours a week, 50+ weeks a year, they do nothing but their specialty. ACGME limits weekly hours to 80 (I think over a 2-3 week average), but 90% of the doctors I know said they regularly broke that and just didn't log the extra time.
Especially in surgical residencies, all you're doing is your specialty-specific stuff during that period of those.
You just said what he said, but with emphasis on 100 hour weeks for years on end being good instead of bad. Why did you disagree with me, then go on to list how much doctors work before the get to practice on their own? His point was they get too much training, with much of it being irrelevant (not all). If you're this angry and reactive, you really shouldn't be a doctor.
People here seem to love the NHS. In the UK, doctors are not forced to study something irrelevant for four years in college, then do med school, then do a 4 year residency (i.e. age 30). They are often done by age 24, and ready to help.
One thing that makes conversations with doctors about regulations around board certifications easier to understand is that anybody who is currently a doctor in the US is heavily disincentivized from improving or changing the system in any way. The absurdly onerous restrictions on becoming a doctor work to the benefit of current doctors by artificially restricting supply and thus keeping wages high. Why would doctors want to get rid of those very regulations?
I'll be more clear - doctors have almost no extra or unnecessary training during their residencies. It's all very specialty-driven, or at the very least is specialty-specific public service (e.g. low-income clinics). If anything, the doctors I've spoken to said they should all probably be a year or two longer across the board if only to allow for better work-life balance, but none of them would want to have to go through that obviously.
The closest thing to "extra training" they get is fellowship-related rotations, but even this is all things they'll see in practice so they need to know how to handle it initially, if for no other reason than so they know when to offload it to a specialist.
> They are often done by age 24, and ready to help.
I'm sure this is fine for whatever the equivalent is to an urgent care doctor in the UK (bottom of the barrel family med in the US, probably not board certified - e.g. failed the exam or not qualified to take it - or doing transitional residency because they didn't match anywhere), but I'm not really interested in my orthopedic surgeon or neurologist just getting through their training as quickly as possible.
There are lots of ways the US could increase the pool of doctors, and most doctors are probably paid way too much (paradoxically, probably most egregiously at the low end of skill), but "cut out a bunch of training" is a dumb way to do it.
You realize that 4 years of residency isn't the magical number of the perfect amount? Two could very well be sufficient, and the other two "extra or unnecessary training." Also, you keep ignoring the college requirement, which makes you seem very disingenuous, and if you are a doctor, makes me worry for your patients.
I'm not a doctor, but why would you want a doctor who didn't go to college?
I was pre-med in college and quickly changed after I realized I didn't actually like biochem all that much. Imagine what that would have looked like had I been attending a medical school instead of a "normal" college.
It sounds like what you actually want is an NP or something like that. Which is fine, there are plenty of those around.
What? Why should I care whether or not they went to college? I want them to be able to do their job, and I don't care about prestige whoring over competency.
Maybe if they hadn't required classes that are irrelevant to 95% of doctors (orgo, biochem and pchem), you would have been able to pursue the career you wanted. This is yet more support for the idea my friend who "I should never ever listen to" said about requiring far too much credentialing.
NPs would be fine... if they were allowed to give medical advise. Unfortunately, we still have to pay for someone with 6 years of excess schooling to come in to weigh in officially and to pay a huge premium for it.
Taking this discussion at face value, it sounds like US physicians go through substantially more training than their UK counterparts. If true, does that manifest itself in substantially better outcomes for their patients?
It's more it's like 38%. Hollywood accounting all the way. In particular the deadweight cost of the the AMA monopoly on licensing is like 50% of GDP, which doesn't sound possible but if you know what a deadweight cost is it's a part of the nation's income that can't exist because it isn't there. It's how much better life would be if a doctor were as cheap as an uber driver, or if people healing others didn't get medium security prison as a result. It's the greatest threat to National Security, more than Russia China and the Middle East combined. And it's a military problem, wounded soldiers yeah get help from the Veteran's Association but they have to compete for those doctors and it ends up...out of pocket if they really want good care. Military gets fucked paying for doctors. America spends more money on obesity, than it does on defense. AMA is treason.
Standard Oil had a lot of different shareholders, John D. Rockefeller was never majority owner, that was an organization AMA is an organization there's now medical families. Common heirs. Medical students from a medical family get little hazing compared to the rest. All the maneuvers they make to avoid the words "monopoly" and "cornered market" are of no help and mitigate nothing. So they know people get fucked off with those words, like bad, they're afraid of those words.
I've worked in the medical device industry for 20 years and have a similar takeaway. I often describe it as the "hospital insurance company industrial complex".
I might be joining a large EHR company in the near future as a VP and am wondering if I can send you a question from time to time as they come up via email? Would love to connect but don't see anything on your profile here.
I work in data at https://www.carrumhealth.com/, and I've been parsing this data for weeks. The transparency prices allow us to meaningfully negotiate with providers, and make tangible, incremental progress toward cheaper health care. Providers and existing insurance carriers leverage information asymmetry to control the market otherwise.
For context, we bundle the 100's of itemized costs into a single, static bill per surgery type. In doing so, we've built a custom virtual-network with the most efficient surgeons. These surgeons are able to meet the volume and quality requirements to allow for lower margins. We're able to get negotiated rates that are 10-40% cheaper than traditional insurance contracts when we have data that we trust.
Unfortunately, this data alone isn't enough to properly determine prices because organizations will spread costs across procedure and billing codes that often occur in aggregate groups. For example, in a joint replacement surgery, some organizations may dump the cost into the billing for the implant itself, while others may put it under the procedure code. You have to gather billing data en masse to see which charges occur together, then combine this pricing data to determine what costs will actually look like for someone experiencing a procedure.
It's very expensive, carriers have an economic incentive to simplify it and this is still where they end up. There are a long tail of provider circumstances that the single-payer model will need to figure out. Some examples:
* Small hospitals in low-density, underserved areas have to make up for underutilized equipment and personnel costs. They raise prices on unrelated, common procedures to break even (This is very common)
* CMS (medicare/medicaid) sets a low price for a procedure that's overly common in a particular facility, now that facility loses money for each occurrence. They choose other procedures to raise the price to try to break even.
* Larger hospitals have higher administrative and operations costs (for things like training and research) that benefit society, but need to be averaged out across all procedure costs. This differs from hospital to hospital.
* Smaller professional facilities or physicians groups (like Ambulatory Surgery Centers) have much lower administrative costs and a smaller staff, so they have lower overhead per procedure. They are designed to be efficient, and can handle lower prices. However if there are any major complications, they won't be able to service the patient, and have to send to a hospital. This then pushes all the highest-cost, ICU-type procedures into hospitals, where there is already a higher overhead, causing hospitals to need separate pricing to cover more complex patients.
A large single payer price set will probably force efficiencies into the healthcare system. It'll be great for folk's costs, but we may see many facilities close, and lines of care will be consolidated into specialty centers. (more travel to get imaging, procedures, or to see a specialist)
What do you think about how Kaiser has handled the whole thing? The insurance company employing the doctors and just paying them a standard salary seems to create all the right incentives.
My experience in talking to people with chronic conditions that aren’t easily treatable is that Kaiser’s model works great until anything that’s slightly out of the ordinary happens, and then it falls apart. If you’re a zebra (as in “when you hear hooves, think horses, not zebras”), their model is pretty horrible.
Yes but if you're at Kaiser in San Francisco and have a zebra there may only be one doc (or a small group) at UCSF that can treat your zebra, and they are not in the Kaiser network, so you go to Los Angles where Kaiser's specialist is, get treated by a lesser doc with a virtual visit assist from LA, or pay cash out of network.
>>* CMS (medicare/medicaid) sets a low price for a procedure that's overly common in a particular facility, now that facility loses money for each occurrence. They choose other procedures to raise the price to try to break even.
This is precisely why most Doctors I speak with are abhorrently against a single payer system.
Most doctors I talk to vaguely run around the answer before mumbling that a huge way to cut costs (which will surely happen) is to cut doctors salaries.
Source: once engaged to a doctor who had doctor friends and doctor parents/family.
And there is a reason why we shouldn't go off of anecdotal evidence. It's blatantly false.
Doctors’ salaries account for only about 8% of U.S. healthcare costs. A 40% cut in these salaries would reduce healthcare spending by only about 3% [1][2].
Doctor salaries are not a huge way to cut costs. If anything this would make the problem worse.
It’s not false that doctors worry that. Doctors worry that single payer system will reduce their salaries. They’re an easy political target. They’re rich and (in this hypothetical case) their salary would come from the taxpayers. Taxpayers don’t like expensive salaries.
It’s irrelevant how much of the budget it is. It’s about perception and power. If you try to cut soap in the operating room or other supplies, you’ll look bad for endangering the patient. If you try to cut procedures you’ll look bad. If you try to cut doctor salaries, those “overpaid” doctors look bad for complaining.
Doctors have a reputation in america for being extremely well paid. If you tel people making $60k a year that their tax bill for medical costs could be lower if you reduce it by taking $50k from a doctor making $500k (taxpayer dollars!) they’ll support that. Even if it’s not a big amount.
Reducing healthcare spending 3% without any systemic change in medical treatments or equipment or negotiation with pharmaceutical companies is a huge and easy win.
PBS put out a documentary ages ago comparing America to other countries. At the time our administrative overhead was 25% while Taiwan's overhead was 2%.
Is the data unique or has it been duplicated for multiple formats? In other words is there a CSV file right alongside a Json file and an XML file that contains the exact same data, just in different formats?
Is the data partitioned at all (e.g. by state) so that you can just download the data for California without downloading all the data; loading it into a huge database table; and then querying it (e.g. SELECT * from <table> WHERE state = 'California')?
There is some duplication, where different networks under the same carrier could benefit from normalization, but in-general duplication isn't the primary issue.
The data is partitioned for some carriers at the network level, but unless that carrier has networks that are unique to a given state it's difficult to partition by location.
The majority of the data is lumped into very large, single JSON (not newline delimited), so an initial parsing step is required to break out substructures for parallel processing via warehousing technologies. I think Aetna has a 300Gb compressed (single) json file.
After breaking the json to a single array entry per provider/network, parsing is still a bit tricky because there are some very "hot" keys. Some provider array entries may only have 1000 code and cost entries, others may have 100k. We've seen array entries >50Mb for a single provider/network/carrier.
Sounds like an application for ML, to determine which codes frequently coincide per-patient at each provider and then assign those groupings to cross-provider "Treatment XYZ" buckets to enable apples-to-apples comparisons.
How do you get info on bills for historic procedures? Do patients opt in, or will the hospitals provide that information as part of their cooperation with insurers?
Someone with the right connections should call up Google Cloud and ask them to ingest the data into BigQuery as an example dataset like the NY taxi trips.
It would be a great way for them to show off the capabilities of the engine and helpful for everyone wanting to do analysis on it.
Sometimes people do a thing where they see certain keywords in combination and reflexively respond without regard to the meaning those words are expressing. For example, it's what happens if I use a word like "welfare" near that one uncle at Thanksgiving. The signature feature is a very strong negative reaction but with content that doesn't seem related to what the previous person was saying, except that it involves certain keywords.
I think that's maybe what happened here. You saw "Google," "data," and "ingest," and your sentiment analysis report came back positive, and it triggered a response.
For comparison, here's the first bit of an Anthem file (which contains some of the data that's just another row in the Humana record), along with the first record
"reporting_entity_name": "Excellus BlueCross BlueShield",
"reporting_entity_type": "Health Insurance Issuer",
"last_updated_on": "2022-06-14",
"version": "1.0.0",
"provider_references": [
{
"provider_group_id": 302.1518360704,
"location": "https://mrf.healthsparq.com/exc-egress.nophi.kyruushsq.com/prd/mrf/EXC_I/EXC/providerReference/Providers/S-000000001063.json"
}
],
"in_network": [
{
"negotiation_arrangement": "ffs",
"name": "Brief (20 minutes) care management home visit for an existing patient. for use only in a medicare-approved cmmi model. (services must be furnished within a benefi",
"billing_code_type": "HCPCS",
"billing_code_type_version": "2022",
"billing_code": "G0081",
"description": "Brief (20 minutes) care management home visit for an existing patient. for use only in a medicare-approved cmmi model. (services must be furnished within a benefi",
"negotiated_rates": [
{
"negotiated_prices": [
{
"negotiated_type": "fee schedule",
"negotiated_rate": 51.1,
"expiration_date": "9999-12-31",
"service_code": [
"11"
],
"billing_class": "professional"
}
],
"provider_references": [
302.1518360704
]
}
]
},
Anyone else get the feeling this is malicious compliance on behalf of the insurance companies?
"Oh, they're going to force us to publish our prices are they? Well we'll publish so much data it'll take a herculean effort to make it readable to anyone that doesn't work in data engineering"
Not only was this probably a best effort, but I would bet that at least a few people were busting their asses trying to pull this data together and clean it up.
The article mentioned CSV files, it seems more like a reflection of what a huge bureaucracy the US healthcare system is. I liked their suggestion that the government should have created the database as part of the law, done the processing on the raw data, and made it more accessible.
meh, I'd prefer the raw data. We can always create DBs out of the raw data, we can always link data. Handling this after the fact would be impossible.
Linking a few trillion records doesn't seem that difficult. It should be doable with a good data warehouse and a reasonable entity linking model. I suspect that we'll find more than a few instances of fraudulent behavior once the data is linked.
My father was nearly pushed into ~2 Million dollars worth of brain surgery that was unnecessary. Not only was the procedure unnecessary, the price for it was >5X what a top-3 hospital would have charged. I only became privy to this once I pushed him to come to Mass General Hospital (MGH) for a second opinion. The surgeon we saw at MGH also believed the suggested procedure to be dangerous.
I wonder if it's possible to cross-reference mortality/complication rates with prices...
with open(filepath) as fd:
first_line = fd.readline()
cols = []
for col in first_line.strip().split(','):
col2 = f'''"{col.strip('"')}" text'''
cols.append(col2)
cols2 = ','.join(cols)
print(f"create table {table_name} ({cols2});")
print(f"\copy {table_name} from '{filepath}' csv header;")
this variant will ingest whatever trash is in your CSV fields as-is (cast & cleanup later)
run the output in a psql instance connected to your db
(important note: \copy is a psql client command and it is critical to use \copy instead of COPY in many cases where the server process may not have the permission to read your CSV file. with \copy you can read any file the user that launched psql client has permission to read. to make things more confusing it is indeed possible to stream stdin through psql but you use the regular COPY for that instead of \copy)
This is obviously a quick and dirty hack. It only splits the header by commas. Postgres has its own read in logic you can adjust by passing arguments to \copy or copy. If you have anything more complex you need to handle it by parsing the header more intelligently than splitting on commas and adding appropriate arguments to copy or \copy
In their defense, if it was anything but CSV files, they would be accused of making it too complicated/locking into proprietary formats and so on. I can't say CSV would be my first choice, but I don't really want to think what the alternative would be.
csv is many times over again a simpler format than sqlite and easier to understand by anyone across the world.
Never heard of ndjson, can’t see one publishing this data in a format that isn’t nearly as common as something like csv (or regular json which some of the data is published in).
CSV only seems simple. Lots of parsing edge cases. Sqlite isn't readable by hand but it's basically bulletproof. NDJSON is literally just newline seperated JSON, it's just easier to process as a stream without a special parser.
Unless you involve MS Excel or, worse, MS Excel on macOS. OTOH, pitfalls are: UTF-8 BOM, comma vs semicolon, single vs double quote, multiline cell content and escaping, escaping in general...
Then you’d have to get your hands on a parser. Who wants to write up the business case and spend months going through change control (they’d probably just say no).
JSON and NDJSON can also be much larger than CSV files if the wrong structure is used.
I got a little excited, that's all I can say about the flow.
But there is an undercurrent of betrayal, there absolutely is, undermining everything, only visible if you get trapped in it, if not it looks like a meaningless whoopsie.
Perhaps you'll think my comments are unthinkable. My only response to that is that they were legibly written, not by a machine, but by a writer with a soul.
Yeah, it's unclear to me. A lot of the more personal things that are mentioned throughout the account's posts seem to match up with some of the quickly-googleable details that can be found just via their username. I suppose that it could be baked into the AI, but... /shrug
Came here to say the same thing. I would expect their internal systems to calculate certain pricing components on the fly so it feels like they’ve deliberately built all possible permutations and data dumped that.
Keep in mind these files are called "Machine Readable" for a reason. Yes, they dumped everything. Machines and 3rd parties are supposed to sort it out. Best be assured each carrier now has full visibility of each other leading to "transparency".
It might be a little exciting to be an underwriter right now :D
They should be forced to provide the costs of procedures up front. As in the whole procedures not just provide prices for a million sub-procedures that the _might_ bill you for. This is how it works in basically every industry. Healthcare tries to argue that it's impossible for them to do this, but is just a load of bullshit.
The problem is that the health care costs situation results in many deaths and very severe economic consequences for much of the country.
Until lying, cheating, and scheming, and screwing over the public have consequences like prison time, you can expect executives to do everything possible to avoid complying with the spirit of laws like this.
There probably was an effort to create a more useful and sanely worded law that would provide a uniform format for rules that could reduce dataset sizes by a factor of 100, but was killed by the healthcare industry because it would require some implementation costs on their end and make the data files actually useful.
Oh totally. It also probably is the formats they already had -- so they just dumped them into a file -- versus making something more orthogonal and ergonomic.
I'm not sure what format they store their records in, but I have a hunch it's a lot more structured than what we see in the CSV files. The data dumps have to comply with some CMS guidelines set out here: https://github.com/CMSgov/price-transparency-guide
I wonder what percentage of work in the US healthcare system is completely unnecessary from a general perspective but made necessary deliberately to justify the unethical system that allows millions to die unnecessarily.
Why that article points out the US spends $12k/capita on healthcare the singles out administrative costs at $1k/capita while ignoring all the other relevant factors is beyond me. They then use the misleading infant mortality stat, ignoring that the US considers vastly more babies viable than any other country, meaning we try to save infants that other countries write off, thus they count against the US when it fails, but not against the other countries that don't count them as viable. It's a really poor article ignoring important nuance in what it presents.
The US pays about twice per nurse or doctor in the system, and part of that is because the US pays nearly twice for most skilled work. So, to get prices like most other developed nations, we would be forced to cut nurse and doctor salaries, which would likely lower quality of workers as future workers went to more lucrative fields, which would likely lower outcomes.
The US can have higher cost or lower quality. How would you make this tradeoff?
> So, to get prices like most other developed nations, we would be forced to cut nurse and doctor salaries, which would likely lower quality of workers as future workers went to more lucrative fields, which would likely lower outcomes.
Why are you ignoring all of the costs that go to people besides nurses and doctors? I know very rich people whose entire careers are built around selling overpriced products to hospitals. These people are leeches that provide no value other than profiting off of dumb compliance laws. If you can buy the same product at any store for 1/10 the price, there is no benefit to requiring it be gatekept by people whose sole incentive is squeezing blood from a stone.
Get rid of graft. The problem is the system and the incentives it creates. US healthcare is dictated primarily by insurance companies who care more about maximizing profit than providing healthcare.
To fix the system you start with increased transparency, then you focus on accountability. Why do we allow such blatant corruption? Let's get rid of all the leeches first, since they provide no actual value while jacking up prices. There are so many areas we can improve results and cut costs before we address the salaries of doctors and nurses.
> The US pays about twice per nurse or doctor in the system, and part of that is because the US pays nearly twice for most skilled work.
which is in turn because in the US an average GP comes out of medical school with $200k-300k of student debt that has to have interest serviced and paid off within some 10-20 year timespan. That cost ultimately ends up being borne by the patient and their insurance.
unfortunately the US is very resistant to the idea of education reform in general, very very resistant to student debt relief, and very very very resistant to student debt relief for "high earners" like doctors and lawyers, even when a huge chunk of that earn is going to debt service. But there is a shortage of doctors and we're doing everything in our power to make the path unattractive for new students. And this time the problem isn't even the AMA - the AMA agrees there is a problem and is onboard with expanding the pipeline... it's just not all that attractive a profession anymore when you can make equal/higher compensation (after considering the debt) in software or other fields.
doctors are still extremely well-paid professionals in other countries, but if we tackle the cost of education we can get our numbers down much closer to theirs. conversely if you push salaries too low then servicing $200-300k of student debt won't be realistic and the path becomes even less attractive.
medical care is probably the single most complex political problem in the US because it's basically at the nexus of every single social and political problem we have. doctors are too expensive... because they're trucking around a quarter million of student loan debt from our shitty education system. we spend way too much on end-of-life care and not enough on earlier care... because seniors vote. we have way too much overhead due to the multi-payer insurance system and the market-driven pricing system's overheads... and all those insurance companies are huge lobbyists too. Drug and device costs are out of control... because the US doesn't allow conditioning of regulatory approval on price negotiations, or reimportation from other countries, etc. It's just every single political problem in the US in a single field all at once and every hand is dipping into the till as much as they can get away with, and it's politically infeasible to slap the hands that are necessary to slap to actually get costs reduced.
A debt of 1 to 1.5 years salary does not go very far to explain why US doctors are paid double what they would be in other countries.
The US brought this problem upon itself by cutting medical school funding in the 1980s to reduce the number of doctors and keep salaries high. That situation remained until 2005. Now we have too few doctors, too few schools, and a generation that grabbed all the money for themselves and is now retiring.
My understanding is that the largest portion of the discrepancy with other nations is the price of facilities. And that those facilities cost much more at non-profit hospitals (which is most of them). The hypothesis is that administration compensation is proportional to procedures performed, but money taken in can't be kept, so it is pumped back into facilities perpetuating the cycle.
Probably because the people pushing for free college etc tend to be ones studying the humanities, which gives people a bad impression of the whole thing. People make fun of Fox News touting lesbian dance studies majors, but the reality is most people don't support funding your bullshit gender studies degree with their tax dollars.
> the AMA agrees there is a problem and is onboard with expanding the pipeline
Oh come on, they could start by accepting everyone with a 3.5+ and decent MCAT instead of requiring that you have a 3.95, volunteering experience, clinical experience, and near perfect MCATs. The path is unattractive because it's filled with bullshit requirements that don't matter.
On top of this most med schools discriminate against their largest pool of potential top students: Asians. It's well known if you're Asian you need much higher MCATs and GPA to get into med school. How many people have been pushed out of considering medicine because of this?
>>Oh come on, they could start by accepting everyone with a 3.5+ and decent MCAT instead of requiring that you have a 3.95, volunteering experience, clinical experience, and near perfect MCATs. The path is unattractive because it's filled with bullshit requirements that don't matter.
I for one am glad that it s hard for doctors to get into med school - allowing less qualified people to practice medicine sure doesn't sound like a recipe for good outcomes.
As far as the non-academic 'bullshit requirements' as you put it, they matter - last thing you want is someone going to med school because they were above average smart, and there parents told them to goto med school (it happens) - much better to have people that have been in the trenches dealing with medical issues at some level who know what they are getting into - i.e. people who perhaps were a nurse first, or EMT or paramedic, or even a non-skilled person who provided personal care to dementia patients in nursing home - just being smart isn't enough to be a good doctor - doctors deal with a lot of things that most of society would find distasteful - better to weed out those folks before they ever set foot on campus taking up the slot of someone else that is more well rounded and proven they are not choosing medicine just because it pays well and their parents pushed them to it.
I would take a smart doctor motivated by money over a dumber one motivated by caring for people.
The reality is all of the top surgeons, cardiologists, etc. didn't become those professions just because they were "well rounded", they were smart and they wanted prestige/money. It would be good for medicine to have more smart and ambitious people.
Well rounded is just a euphemism to discriminate against Asians through affirmative action.
>>I would take a smart doctor motivated by money over a dumber one motivated by caring for people.
Luckily we don't have to make that choice, we can have the best of the best - the smartest people who want to go into medicine for the right reason - thats why its hard to get into medical school, as it should be.
>>"The reality is all of the top surgeons, cardiologists, etc...."
I assume since you are making such a sweeping statement that presume to know what motivates 100% of MD's, that you have a link or reference to back up that unequivocal statement? I thought so.
>>Well rounded is just a euphemism to discriminate against Asians through affirmative action.
Are you one of those people that assumes if you simply mentioned race in your argument, you win by default? Pathetic.
> thats why its hard to get into medical school, as it should be
We literally have a shortage of doctors and this is your attitude? Someone who wants to make money is an equally good or better doctor than the person who wants to help people. Medicine is scientific: you either do the operation successfully or you don't. You diagnose the patient successfully or you don't. A person's motivations for becoming a doctor doesn't play a factor in their skill.
We don't ask McDonald's workers why they want to work at McDonald's, why do we need to do it for doctors?
> Are you one of those people that assumes if you simply mentioned race in your argument, you win by default? Pathetic.
Do you deny that terms like "well rounded" and "holistic" are used to discriminate against Asians? Or should I point you to SFFA vs. Harvard, which showed that your coveted "well-rounded" personality traits can be and are used as tools of discrimination? I'll remind you that Harvard intentionally reduced the personality scores of Asians to make them seem less "well-rounded".
Med school admissions would be fairer without requiring such things as "well rounded" candidates. Personality scores are subjective and subject to bias and foul play, MCAT scores are not.
>the US is very resistant to the idea of education reform in general,
I suspect almost all Americans are interested in education reform, but are split between two opposing directions: getting rid of nondischargeable, subsidized federal loans; or making the whole thing run on federal money.
That price pays for the parking deck, security, janitors, nurses to administer the bag, needle disposal, IT, admin salaries, the hospital building itself, etc etc.
An urgent care can probably administer an IV. If that’s all you need, go there. They are far cheaper and not as lavish (or equipped) as hospitals.
The US has a mortality rate due to preventable causes comparable to Poland/Slovakia despite having nearly a 4x higher GDP per capita. Even poorer countries with better systems do far better w.r.t preventable mortality. This amounts to an extra 100,000 deaths per year approx vs countries like Italy, Germany, Switzerland, Sweden, etc.
There are lot of people who don't go to a doctor when they should. Even taking an ambulance after an accident is a gamble a lot of people can't afford.
>>There are lot of people who don't go to a doctor when they should.
and there are an awful lot of people, even if it is free or paid for by a great health insurance plan that ALSO don't go to the doctor when they should - thats just human nature
and on the other hand, lots of people running up ten's of thousands of dollars in unnecessary tests because they are hypochondriacs and run to the doctor every time they have a sniffle and demand every treatment and test under the sun.
It's pretty bad probably. It's basically a system that is incentivizing companies to make lots of money where a lot of the checks and balances are misaligned (in some cases intentionally so) which further enables this. So, you have pharmaceutical companies charging extortion rates for products that they sell for next to nothing elsewhere. Insurers that squeeze their patients hard. And hospitals that blindly prescribe medication because the patient demands it because they pay so much money for their insurance. Hospitals don't care because it's not their money and they get to bill the insurer for all sorts of bullshit. Insurers simply raise the prices for their customers. And they can actually cut loose patients over all sorts of technicalities so patients don't complain about this.
Aligning the incentives is pretty hard but not impossible. The Dutch system was facing rising cost a few decades ago. It was split in a private insurance and public insurance system. Privately insured people enjoyed all sorts of perks (like private hospital rooms, less waiting time, etc.). The same system still exists in Germany (I live there currently).
To improve financial efficiency the Dutch government decided to get rid of public insurance and empower people to switch insurance. Everybody has to have insurance, all the insurers are private, and they have to compete to keep people as they can choose to jump to another insurer and they don't get to reject people. They all have to offer the same base package of care to everyone but can choose to diversify on top of that. This results in people shopping around and being treated like customers by insurers.
The second thing they then did was empower insurance companies to make deals with care providers. After all, they are paying the bills and if some hospital is being inefficient, they have to pay for it. They can't reject patients. But they can make deals with certain care providers or refuse to do business with others. This incentives care providers to align with insurers.
Likewise, pharmacies that supply medication are incentivized to look for cheaper alternatives. So, pharmaceuticals end up competing with each other for some things and pharmacies will pick what's cost effective rather than what doctors prescribe (in case of compatible alternatives).
It's not a perfect system but it has resulted in hospitals and insurers improving their game and getting rid of inefficiencies or bad service. Bad insurers lose their customers, inefficient hospitals result in insurers taking their business elsewhere and they suffer financially. Smart hospitals and insurers align what they are doing and avoid needless treatment. Patients and employers shop around for the best insurers based on the needs and means and to get the best rate and care or access to their preferred care providers.
I actually live in Germany which has a system that resembles what things used to look like in the Netherlands. It's a bloated, inefficient system. There's stupid bureaucracy left right and center, endless referrals and waiting lists, and you are treated like cattle. I have private insurance so I get to jump the queue but I also get to deal with doctors that are a bit too trigger happy with treatments and needless appointments that they can squeeze the insurer for. The insurance is super expensive for me. And I can't easily switch insurer so they can squeeze me hard and up their rates. The hospitals are pretty bad and miserable compared to Dutch hospitals.
Here is a golden opportunity for the info/data visualization community to show how their tools can handle big datasets to make them comprehensible to the public.
Seems like every week there's a new massive scale DB project or company getting announced on HN.
If they're looking for projects that create public value and demonstrate the power of their products at scale, digitizing this and making it searchable may be a good marketing project that's appealing to certain kinds of customers.
It would appear us SQLite zealots have encountered the final boss.
Petabytes uncompressed would be tricky if you need to slice those columns. SQLite caps out at ~281 terabytes of storage before it can't track any additional pages.
None of this is to say you couldn't partition the data across a lot of SQLite instances in varying ways. I will probably take a shot at it this weekend. Looking to see just how unlimited my AT&T fiber connection is anyways.
> It would appear us SQLite zealots have encountered the final boss.
That's cute. :)
There isn't much value in feeding it all into a conventional RDBMS. OLAPs and columnar stores are what is needed here. But first it will need a great deal of grooming and ETL work.
Yeah.. It would be much easier to copy the data to S3/any object storage (better to convert it into a columnar format like parquet) and query it directly using a SQL on lake engine like Dremio or Athena or S3Select would work too.
>It would appear us SQLite zealots have encountered the final boss.
Just wait. It's actually a multi-boss fight, since you have to wrangle the Pharmacy Benefits Management datasets, plus Medispan, plus Medicare, plus all the MedicAid datasets, plus VA.
Are you and all your mightiest boxen bad enough dudes to make sense of the entire U.S. Healthcare industry?
Figuring out the size of this data was part of the research phase for doing just that: building out that database. I'm curious to know if other people are already working on it (maybe Turquoise Health?)
It's cool seeing that Turqoise Health exists. One of my first programming projects back in the day (when I was trying to get a jr role in 2014) involved building a simple version based on data.gov medicare data. The inputs were terrible and tiny (e.g. chest pain at hospital X costs ~$60k on average across 5 patients), so I was always curious what a real world version might look like.
edit: As I reflect, I'm amused to recall that this was early enough in my path that I didn't know about DB indexes, so I was very proud that I figured out how to basically roll my own indexes by pre-sorting the columns by lat and lon. I don't remember whether my solution actually prevented a full-table scan, but it felt like a major breakthrough at the time.
Very cool. Who do you see as the likely users of that database? Is it primarily for researchers/data journalists, or is there a commercial value to it?
I'd be very curious to read more about the data cleaning phase when you get there. Specifically, how hard it is to combine this data and construct good schemas.
As someone who's worked on the provider side in different capacities, I can tell you that there could be tremendous value on the provider side.
It's entirely possible that two surgeons with offices next to each other could be getting reimbursed at wildly different rates for their most common procedures for their most common procedures by the same provider.
If you're that provider, you ABSOLUTELY want to know what the surgeon next door is getting paid the next time your group is negotiating with the insurance provider.
Interesting. I'm kinda surprised this is handled by the doctors themselves. I'd expect there to be professional negotiators who parse this data themselves and then use it to negotiate on their behalf.
