Part of the problem is that techy people, or justice-minded people, love the detail of just how awful something is. I've been boring my friends to death for years over this scandal.
By comparison, this drama was only a few hours long, was aired while people were still in a holiday/relaxing mood after Xmas and New Year, and spun a compelling narrative about the people and communities affected. It had much clearer "goodies" and "baddies" (without being unfair). So it deserves credit in reigniting interest in the scandal.
Another thing that I fear people don't get: they see the name "Fujitsu" and thing "oh, Japanese people". Not in this case, no.
Fujitsu is just the current name of who owns ICL, International Computers Limited. ICL are British through and through. This entire scandal is "Fujitsu's UK subsiduary", i.e. ICL
ICL were established and got into UK Government IT procurement in the 1960s, because Harold Wilson and Tony Benn didn't want to be beholden to the Americans (IBM) for these new-fangled computer calculator thingies.
ICL would supply both the _computers_ and the _software_ to British government. It still runs mainframes for the Inland Revenue and Department of Work and Pensions.
By the 1980s, they were still running but they were using Fujitsu components and entire PC systems, and by the late 1990s, Fujitsu had acquired ICL... but ICL is still ICL (with a new name) and is still incredibly chummy with the UK government and still wins contracts for government IT procurement. None of this scandal happened in Japan. It happened in Bracknell.
Fujitsu (and to a certain extent, the Japanese government) is free to stop lending its name to incompetent & criminal scumbags if they don't want to be associated with it.
The drama was brilliant. I knew that this scandal was brewing; really, everyone in Britain that kept up with any news at all knew it was coming. The drama brought out just how much suffering was caused by the Post Office's lies; and just how much CYA bullshit the Post Office got up to.
I'm waiting to see a bunch of Post Office senior management in court on criminal charges.
> Although parts of the EPOSS code are well written, significant sections are a combination of poor technical design, bad programming and ill-though out bug fixes. The negative impact of these factors will continue and spread as long as the PINICL fixing culture continues. This is partly due to the nature/size of the bug-fixing task and partly due to the quality and professionalism of certain individuals within the team. The problem is probably best illustrated [by] examples:
> Example 1: This extract from EPOSSCore.dll has been written to reverse the sign of a number and is equivalent to the command d=-d
Public Function ReverseSign(d)
If d < 0 Then
d = Abs(d)
Else
d = d - (d * 2)
End If
ReverseSign = d
End Function
> Whoever wrote this code clearly has no understanding of elementary mathematics or the most basic rules of programming.
2. Here are the juicy details about the overall architecture:
> The Fujitsu-designed Epos software on the PCs was written onto an off-the-shelf system called Riposte.
> Our source said the big flaw in Horizon was the way data was being written to Riposte.“Riposte wasn’t really a database, it was a messaging system based on an XML structure where you write messages down into the message store, and then Riposte took care of replicating them,” he said. “The first thing that you should always do with a system like that is design and agree a data dictionary and a message library repository, basically to say: these are the messages that are allowed to be written to the message store and they all provide the following function.
> “You should also have a layer of software that lies on top of the message store that checks that any application above it which is trying to write a message, conforms to the agreed data dictionary. Otherwise, you can just write freestyle to the message store, which is what they were doing. There was no application interface in there, no agreed data catalogue or anything.”
> Computer Weekly also spoke to a former Fujitsu employee who worked in the Horizon Service Support Centre from 2001. The support engineer – who also wants to remain anonymous – recognised this new description of Horizon’s badly built message store, adding: “Our job was to fix these problems as they arose. We all knew the code wasn’t fit for purpose and needed rewriting. The data dictionary was still being added to when I got there.”
> For the first 10 years of Horizon’s existence, transaction and account data was stored on terminals in each branch before being uploaded to a central database via ISDN. Our source says this part of the system simply did not work.
> “The cash account was a piece of software that sat on the counter NT box, asleep all day,” he said. “At the end of the day, or a particular point in the day, it came to life, and it ran through the message store from the point it last finished. It started at a watermark from yesterday and combed through every transaction in the message store, up until the next watermark.
> “A lot of the messages in there were nonsense, because there was no data dictionary, there was no API that enforced message integrity. The contents of the message were freehand, you could write whatever you wanted in the code, and everybody did it differently. And then, when you came back three weeks later, you could write it differently again.”
He gave an example of a message stored previously when a customer bought a stamp. It was feasible that a new message for buying a stamp weeks later could be slightly different.
> “When the cash count came along, it found a message it was not expecting and either ignored it, tripped up, or added something it shouldn’t be adding,” he said.
> “You should also have a layer of software that lies on top of the message store that checks that any application above it which is trying to write a message, conforms to the agreed data dictionary. Otherwise, you can just write freestyle to the message store, which is what they were doing. There was no application interface in there, no agreed data catalogue or anything.”
I think we can see that this sentiment has wider support in the software community - I would put many NoSQL database adaptions into this category. Another interesting development that shares this sentiment is frontend developers desire to bypass backend layer and get direct access to the data to "easy" and "speedup" their development.
But nothing in your quotes shows anything especially incompetent - there's no smoking gun.
The world runs on badly written software. I have even written some highly incompetent code myself for essential services. Finding terrible bugs and then looking for who caused them has almost humbled me once or twice.
>But nothing in your quotes shows anything especially incompetent
They did not care about data integrity and schema migration. It's probably the most incompetent thing one can do when working with data in such a system.
That's why we define database schemas, or write protobuf schemas, or avro schemas, and parse JSON data in the frontend with zod.
As a rule of thumb you don't even just change the datatype of a field and hope it works. You introduce a new field and migrate over the course step by step from the old field to the new field, i.e. you write the data into two fields, and read the data from the new field or fallback to the old field.
Actually their use of the Riposte message system in 1999 was not that different in what we use today in many cloud-based systems with microservices.
Yes, but at the end of the day nobody wanted to lose face and admit the system was flawed and people ended up in jail/going bankrupt/committing suicide or all three because £10,000s of discrepancies were assumed to be fraud and not bugs.
Yes. Flipping the sign of a number can typically be done in one line as `d = -d`. Or if that syntax isn't available in that programming language, then `d = 0 - d` or `d = -1 * d`.
What's wrong with code above is 1) it does a conditional branch (meaning if.. then... else...) which is unneeded. 2) The "then clause" does an absolute value which is a waste of a function call. 3) The "else clause" does `d = d - (2 * d)` which is a waste of a multiplication 4) possibly with an overflow risk and/or floating point estimation risk.
That said, it's admittedly possible that the programming language above, or the number representation in that code above, have some highly atypical needs that we here on HN don't understand.
They had teams with software engineers with different levels of competence and experience [1]. There were good, mediocre, and members that were not so capable. Which is true for many teams. When at least the good members have the time and review the code of the others, than this code would probably not go into production.
Even when these code snippets are cherry-picked they tell enough about their review practices.
>have some highly atypical needs that we here on HN don't understand.
Somebody else checked in another thread. You can use unary minus in VBA.
[1]
>One member of the development team, David McDonnell, who had worked on the Epos system side of the project, told the inquiry that “of eight [people] in the development team, two were very good, another two were mediocre but we could work with them, and then there were probably three or four who just weren’t up to it and weren’t capable of producing professional code”.
it appears correct but over-complicated -- it should be a single line of code. the over-complicated solution may have underflow/overflow problems with very large numbers.
By comparison, this drama was only a few hours long, was aired while people were still in a holiday/relaxing mood after Xmas and New Year, and spun a compelling narrative about the people and communities affected. It had much clearer "goodies" and "baddies" (without being unfair). So it deserves credit in reigniting interest in the scandal.