I just don't get it. NHS needs a big data platform so they go to a... spy tech company? Why are they so insistent that it must be Palantir that builds this, instead of choosing from a dozen of other consulting companies that don't have a dodgy track record?
Ideally the NHS would just build it in house, but sadly it's a slow and bloated organisation unable to innovate (as most government managed things usually become).
> Ideally the NHS would just build it in house, but sadly it's a slow and bloated organisation unable to innovate (as most government managed things usually become).
This is patently false, a narrative often parroted by people who don't understand or have never worked in the NHS.
There are many intelligent people in technical/digital roles in the NHS that innovate on a daily basis - heck, many build systems internally at a hugely reduced cost compared to outsourcing yet decisions come from parliament/gov agencies that overrule internal decision making and waste insane amounts of cash on vanity projects, or scrapping internal work to be redone by the likes of Accenture.
Given the NHS' costs continue to explode, and most of the cash goes straight to padding middle management, it seems easy to see that it's a slow and bloated organization.
After all, the UK's health spending is increasing at a rate 2,000% higher than close neighbor France's is -- indicating massive fraud and waste.
That "increasing at a rate" vague phrasing makes me instinctively suspicious of abused statistics.
For example, suppose in 24 hours France's spending goes up +0% and the UK's spending goes up +0.001%: "Oh my god! The UK's spending is increasing at a rate infinity times more!!11"
The time period was 2010 to 2019. Even since 2019, the NHS has several hundred billion in planned budget increases.
My favorite thing about dysfunctional UK political rhetoric is when someone will say "the Conservatives are slashing the NHS!" when what they really mean is that they've slightly decreased the rate of increase, so by every metric it's still skyrocketing (but not enough to those who want to dump infinite cash in the middle management growth machine).
Another fun factoid: the pro-Brexit crew were chided for claiming they'd increase NHS funding by 350m Euros a week. Since Brexit, NHS funding has actually increased by more than double that.
Nothing can stop the NHS cash burning. Expand middle management at all costs. Nevermind that we're outspending comparable countries with comparable health pressures.
Dude, I'm gonna have to ask you to back up your statements with some actual data, because what you said doesn't match the what I found at all.
To be specific, I mean these charts of those countries' total spending [0] and govt/compulsory spending [1] during that time period. (Using the OECD stats visualization tool [2].)
For total spending [0] UK's spend-growth was less than France's (not greater) and for govt/compulsory spending [1] the UK's larger growth over that period was still in the same ballpark of ~1.31x versus France's ~1.28.
In other words, the nearest number I can derive for your "increasing at a rate X% higher than France's" is around 1.31/1.28 = ~2.4%. Yet you said 2000%! That's a gap of three entire orders of magnitude which desperately need explanation.
I'm using World Bank data (sourced from the World Health Organization Global Health Expenditure database) on a per capita basis. As you can see, UK spending is skyrocketing. The rate of increase between 2010 and 2019 is 2,000% higher than France's, as originally claimed.
I don't think your data source correctly accounts for inflation, making it essentially useless to see true costs.
As you know, France is a similar country with similar health concerns and similar hospital pressures.
Yet the UK lights taxpayer money on fire on NHS middle management salaries (care certainly isn't improving!) and France has no such problem.
> As you can see, UK spending is skyrocketing. The rate of increase between 2010 and 2019 is 2,000% higher than France's, as originally claimed.
I'm still not seeing it, what are the actual digits you are you math-ing in order to get 2000%? Are you sure you aren't comparing two different time ranges?
Because putting both country-lines on on the same 2010-2019 graph doesn't show anything too shocking:
Do you have a source to say most of the money goes to middle management? My understanding was management employment numbers and costs have reduced over the last decade, but I’m happy to be shown to be wrong via up to date evidence.
I've been at three F500 companies, ones you've heard of and probably own products from, and most had Server 2003 and aging HP-UX (or AIX, or Solaris) boxes in the mix.
That the NHS has a few old systems doesn't mean jack shit.
Like, there are still mainframes in use in many places, and more than you'd think.
We currently have a Conservative government, and they hate the NHS. Anything they can do to sink the NHS or destroy its reputation and push people toward private healthcare is a win to them.
I wouldn't be surprised if this Palantir stuff involves an extremely onerous contract, and probably terms that the NHS cannot even afford long term, and the Tories knowingly entered into it any way.
their primary voting bloc is aging boomers. who are fatter than ever, and with healthcare demands to match.
W Bush did the same thing with the Medicare expansion -- somehow the anti-Socialist party expanded social welfare benefits... to the one group that would vote for them.
I am afraid you can't simply blame the conservative for this. This is a total societal breakdown, and I am not being dramatic if you consider other simultaneous scandals such as PPE.
By that I mean it is impressive how this could happen in one of the foremost liberal society with a large well-funded media representing the whole political spectrum, fiercely adversarial two-party politics, respect for institutional whistleblowers (so long as they don't involve foreign policy) and finally an independent civil service. All of those layers failed to put a stop to this or even to shed an intense light on it for the public. Far more than the conservatives had failed to let this get through.
In fact what is amazing is the public barely aware of its existence as a news story. Try asking a few of your non-tech friends about it. Even the twitter mob failed on this.
I mean that sets the precedent for kick-backs as well, so I wouldn't be surprised if, after losing the next election, a bunch of sitting Conservative MPs go and mysteriously work for Palantir (Greensill all over again and - oh and hey, David Cameron's reputation is being rehabilitated right now by having him in government again, funny that)
> Ideally the NHS would just build it in house, but sadly it's a slow and bloated organisation unable to innovate (as most government managed things usually become).
Several government departments have robust in-house data platforms already. The Ministry of Justice does, the Department for International Trade as well. I've personally worked on the former and have pulled code from the latter.
I've heard some horror stories from HMCTS (part of MOJ) and the Home Office.
While there might be some good intentions, buying a vendor product is often better for large organisations rather than running their warehouses, ETL and Databricks. Just too much overhead and specialist skills required.
I cannot comment specifically on CHEETAS, but what I can say is that USG developing in house software solutions almost always produces a disastrous product that goes over budget and has extreme maintenance overhead.
To see why, you can simply ask yourself: do you think that the unelected officials overseeing government agencies that embark on enterprise software development projects have sufficient expertise and enterprise software project management experience to be able to do this well?
Furthermore, do you think that the quality of engineers that the NHS or DoD can attract with less than half of the compensation of an actual software company stands a chance at developing something good in house?
It’s unfortunately almost impossible for these projects to go right.
CHEETAS isn't really developed in house though; it's mainly developed by Dell. Certainly the leadership is USG-associated, but I think the leadership is actually really good. Unfortunately I seem to be unable to get _real_ access to CHEETAS and finding anyone who has worked with it is a challenge.
I suspect underneath it's mostly Hadoop but it's impossible to separate the roadmap from the implementation without getting my hands on it.
That experience speaks more to the perils of in-housing, not to why Palantir is the best COTS for specific needs here. Are there specific leading COTS here you view it so far ahead of for such a contract?
Closer to our own practice.. Modern LLMs have basically reset the field for SOTA in this space, with Palantir, by definition, being behind OpenAI in the most basic tasks, and thus being in the same race as everyone else to retool. Speaking from our own USG experience, we are deep tech leads in some other intelligence areas (graph, ...), and before OpenAI, often chose to adopt prev-gen leading LLM models (BERT, ...) for tasks closer to the NLP side as we recognized that wasn't where our deep tech had an inhouse advantage. We basically had to start over on some of those projects there as soon as GPT4 came out because it just changed so much that the incumbent advantages of already being delivering on a contract were a dead end for core functionality, and almost a year later, it's now obvious that it was the right choice when we get compared to companies that haven't been. Palantir has been publicly resetting as well for using GenAI era tech, which suggests the same situation.
It seems like you don’t know what Palantir is. Nothing OpenAI does is competitive with what Palantir does. Palantir, like every other software company out there, is exploring what “my product + AI” means.
That's a fair surface-level view, but worth thinking through a bit.
Palantir is multiple main things, and a whole ton of custom software projects on top, and a good chunk of them rely on the quality of their NLP & vision systems for being competitive with others. My question relates to the notion that they are inherently the best when, by all public AI benchmarks, they don't make the best components and, in the context of air-gapped / self-hosted government work, don't even have access to them. Separately, I'm curious how they relate to their COTS competitors (vs gov inhouse) given the claims here. For example, their ability to essentially privatize and resell the government's data to itself and make that into a network effects near-monopoly is incredible, but doesn't mean the technology is the best.
I've seen cool things with them, and on the flip side, highly frustrated users who have thrown them out (or are being forbidden to.) It's been a fascinating company to track over the years. I'm asking for any concrete details or comparisons as, so far, there is zero in the claims above, which is more consistent with their successful gov marketing & lobbying efforts than technical competitiveness.
AI leadership seems existential to being a top data management company and providing top data management capabilities:
* Databricks data management innovations, now that basics are in, are half on the AI side, like adding vector & LLM indexing for any data stored in it, moving their data catalog to be LLM-driven, adding genAI interfaces to accessing data stored in it, ...
* Data pipelines spanning ingestion, correction, wrangling, indexing, integration, and feature & model management, and especially of the tricky unstructured text, photo, and video nature, and wide nature of event/log/transaction recordings important to a lot of the government, are all moving or have already moved to AI. Whether it is monitoring video, investigating satellite photos, mining social media & news, entity resolution & linking on documents & logs, linking datasets, or OCR+translation of foreign documents, these are all about the intelligence tier. Tools like ontology management and knowledge graphs are especially being reset due to the ability of modern LLMs to drastically improve their quality and improve their scalability & usability through automation.
* Data protection has long been layering on AI methods for alerting (UEBA, ...), classification, policy synthesis, configuration management, ...
Databricks is a pretty good example of a company here. They don't preconfigure government datasets on the governments behalf and sell that back to them, but we do see architects using it as a way to build their own data platforms, and especially for AI-era workloads. Likewise, they have grown an ecosystem of data management providers on top vs single-sourcing, eg, it's been cool to see Altana bring supply chain data as basically a Databricks implementation. For core parts, Databricks keeps adding more of the data management stack to their system, such as examining how a high-grade entity resolution pipeline would break down between their stack and ecosystem providers.
They are using palantir foundry, which is palantir's big data platform, or how they call it: "The Ontology-Powered Operating System for the Modern Enterprise"
It varies by agency — either something built in house (very bad) or built by a company that knows how to acquire government contracts, of which there are few - the set of which frankly always has worse tech than Palantir. If product efficacy is not absolutely critical, the acquisition process will be driven by nepotism or other forms of corruption.
As an example for the second case in DoD space, there’s Advana.
Because to build a dodgy, possibly unethical system. Usually the most unethical company (and one that won’t cost a shit ton of money) will be the first ones that politicians or government will pick. Sure gov can build it in house but the system itself is a PR nightmare. Any unethical use and the current administration will be blamed.
By subcontracting it out, gov has an easy out.
- “oh there was a bad subcontractor misusing the system. It WaSnT uS!1!!”
- “oh it was just one bad actor within government. We sent person to black site. ItS NoT a SyStEmIc iSSuE”
The UK government's phobia of doing anything in-house almost certainly stems from the enormous amount of money that the political class makes from government contracts.
The NHS could be as slow and bloated as a whale corpse, but doing everything in-house would probably still be more successful than contracting fraudster companies.
Palantir products gets very little public exposure aside from mockup-looking advertisement materials, I'm sure it would have been cloned as an OSS if we were in 90s so long it amounts to anything substantial, yet there is none. That feels weird.
I honestly don't understand what's dodgy about Palantir. They build a platform to analyze big data and they have clients like government agencies, immigration control and law enforcement. What's the big deal here? Please educate me
Mostly its founding history. Funded by CIA’s venture arm, was basically built bespoke for the CIA and rest of the USIC. These days, yeah, at least a large portion of the company (+product) is a pretty normal enterprise software company that happens to build data platforms.
Probably because the quality of the final end-user software produced by universities is less than what a company like Palantir would produce.
It is kind of amazing what you can create if you have hundreds (if not thousands) of really well paid SWEs with industry experience of working on real products used by people who pay money for it. As opposed to a bunch of really smart (and really underpaid, looking at the stipend amounts) grad students who know a lot about their specific domain, but lack the industry experience and proper product management.
Add on top of that that Palantir will dedicate engineers just for their big customers, just so that they can assist with integrating and customizing the product the way the customer wants. Palantir literally has a role for that called Forward-Deployed Engineer, and they will fly out to wherever the customer is to address whatever issues they have. Even if all else was equal, there is no way a university would be able to do that.
P.S. Don’t get me wrong, universities can produce groundbreaking and amazing software that will live on outside of the university walls. But it would be usually rough cutting edge prototypes with zero customer support of the type that companies and governments typically look for.
Ideally the NHS would just build it in house, but sadly it's a slow and bloated organisation unable to innovate (as most government managed things usually become).