> that was actually what the outcry about Apple's version was about. The more the hash algorithm tried to produce the same hash for different variants of a photo, the more likely it was that someone could get their hands on a flagged hash and theoretically send you an innocuous looking photo that registered as CSAM.
That was totally infeasible. There were two separate hashes, a public one and a private one, and there needed to be multiple false positives for the system to trigger. So not only would you need to generate collisions for two separate hashes simultaneously, including one for which there are no public details, you would need to do it for several images.
People made a lot of assumptions about how it would work without actually reading the papers Apple published on how it would work. So there’s this caricature of the system in people’s minds that is a lot simpler and easier to fool than the reality. That’s what Apple was forced to react to.
and as a minor point of clarification: The structure of the publish perceptual hash (and presumably the non-public one as well) were vulnerable to essentially arbitrary second preimage attacks, not just collisions.
This means that I can take an image and can usually adjust it to have an arbitrary hash, even a hash of an image I've never seen.
It's much more powerful than a collision attack (where the attacker must modify both images).
I doubt have any serious doubt that I would be unable to generate second preimages for two such hashes, but given the second hash was never published that just remains speculation. As AFAIK the first person to develop and demonstrate the second preimage attack against it, I'd like to think that my speculation on this is at least somewhat better than chance. :)
The users privacy is compromised at the point that the public hash had too many hits. An attacker that can implant one hit (e.g. by giving you an altered image that matched genuine illicit material or by using stable diffusion to generate fake illicit material and altering it to match the unaltered hash of an image you possess then submitting it to NCMEC) can obviously also implant multiple.
At that juncture the cryptographic keys are leaked to apple, and all further security depends on apple telling the truth about their process, not being compelled by administrative subpoena, and not ever being unwittingly compromised by hackers or intelligence operatives.
The extra steps of a second perceptual hash and human review are thus not all that relevant, and nor were they clearly enough defined for any analysis of their security properties. Particularly the second perceptual hash's security is apparently at least partially dependent on its obscurity, but you have no reason to believe that it won't be obtained by hackers, rogue employees, intelligence operatives, etc. (And if its obscurity isn't relevant, then why not publish it?).
Even if the hashes were flawless however, the system would be relatively straight forward to attack through less sophisticated means and would retain the overarching philosphical flaw:
Your computing device is your trusted agent-- you share with it material more confidential than your doctor, your lawyer, or your priest. You paid to purchase it. You pay to power it. Increasingly you cannot communicate with family, business partners, or carry out essential and mandatory interactions with you government without using it. Your computing device mediates almost every aspect of your life. As a trusted agent it has absolutely no business scanning your files against unaccountable secret databases, encrypted against your inspection, and undetectably phoning home matches like a KGB spy that you're forced to confide in and house. To do so is a gross betrayal, one that shouldn't just be a bad idea-- it ought to be unlawful.
Service providers scanning content is morally fraught itself, but in our unfortunate current legal standard you have little to no expectation of privacy in information to provide to a third party. And that against-your-own-interests scanning is done on computers owned and operated by the scanners, rather than you. And it's done using access to your information that they already have, so it's a realization of the consequences of existent poor privacy rather than an a new invasion.
The transition to your own devices scanning against you is a bridge to far, no matter how much technical obfuscation is layered onto it.
As someone who has developed privacy technology I found the entire presentation additionally offensive because apple misrepresented the PSI components as protecting the users privacy, when in reality the only purpose for their existence was concealing the list of hashes from the users and thus protecting it from review and criticism. It's one thing for a security scheme to provide insufficient protections, it's quite another to fraudulently present technology which is weaponized against the user as somehow being for them.
That was totally infeasible. There were two separate hashes, a public one and a private one, and there needed to be multiple false positives for the system to trigger. So not only would you need to generate collisions for two separate hashes simultaneously, including one for which there are no public details, you would need to do it for several images.
People made a lot of assumptions about how it would work without actually reading the papers Apple published on how it would work. So there’s this caricature of the system in people’s minds that is a lot simpler and easier to fool than the reality. That’s what Apple was forced to react to.