Who would've thought only releasing pseudo-code isn't good enough...glad to see the scientific immune system fighting back against closed-source science. Your move Google.
I mean it shouldn't be enough to publish in nature. The whole point of science is that it can be validated. It's totally fine that they're hosting their models for free on closed servers with limits, even though it's not exactly the most ergonomic.
It was already validated by winning CASP and the paper by Paul Adams (https://www.nature.com/articles/s41592-023-02087-4) which, although it reads like criticism is actually high praise. Everything the model can do, will be (or already has) replicated by the open community.
Also, for work of the highest art (of which AF3 is an example), publication in nature really is the fundamental unit of scientific currency because it ensures all their competitors will get hyped up and work extra-hard to disprove it.
My statement is correct; both AF papers were published in nature, and both won casp. AF3 is superior to AF2 which means if adams wrote another paper, it would be on increasingly less interesting fine details.
To be clear, I don't think anyone distrusts the benchmarking work nor even the reported architecture, but also no one should need to operate on faith when it comes to work that presents itself as groundbreaking. Probably the first thing everyone did when they tested the model was run a sequence w/ a known cryo structure, but that's insufficient for how deepmind knows researchers will use the model.
> Also, for work of the highest art (of which AF3 is an example), publication in nature really is the fundamental unit of scientific currency because it ensures all their competitors will get hyped up and work extra-hard to disprove it.
IDK about disproving it, again nobody is distrusting the work, but let's also not pretend that a prestige journal is necessary to promote AF3. They could publish in the Columbia Undergraduate Science Journal and get the same amount of press. And to be clear the controversy has largely center on Nature for allowing AF3 to get away with more than they would most other projects, and the wasted time and effort it's taking to reimplement the work so people can add to it. FWIW an author did state that they're attempting to release the code but that's not like a binding vow.
Finally, AF3 strictly speaking didn't win CASP (it almost certainly would) but again this isn't necessarily the point when people talk about validation. The diffusion process does seem to result in notable edge cases (most obviously in IDPs and IDRs but also non-existent self-interactions), it's not a straight improvement in that respect.
