Currently I remain skeptical, 130 pages in 7 months plus meaningful experiments is quite some going. A list of the papers (so at least the authors can defend themselves), the source code and data used (because some of these methods require social media inputs) would definitely help.
After doing so much work though, why wouldn't you go the extra small few steps to publish? That way the work can be peer reviewed and the Scientific community has a chance to learn from it.
Yeah when I got near the end of the post, I was quite surprised. You spent a boatload of time on this and all you do with it is make a PSA on reddit going "hey lol I looked at all these papers and they're all overfit". Someone in the comments asked about sharing the work and was brushed off with "nah, the code is a mess".
This reeks of unscientific work, exactly what s/he describes the papers to be. While the results are in line with everyone's expectations (hence it getting upvotes despite offering zero proof, people already believed it and like it being confirmed), and so I don't really doubt they did try to reproduce papers and failed, it also isn't a reliable source in the slightest. If they were serious about disproving the papers, they could just have written down the steps, even if it's just a few lines of notes with snapshots of the code for each paper.
Just a list of papers isn't going to help the authors defend themselves, the methodology (if it can be called that) is so vague that the authors would basically be starting a "he said she said" discussion.
> That way the work can be peer reviewed and the Scientific community has a chance to learn from it.
After trawling through so much peer reviewed work which in their view is utterly broken, I can understand why they'd be despondent/mistrustful of submitting it to a peer review process.
If you want to go that route, why not write a detailed blog post? Or a white paper in an archive? Or a GitHub repository? Not putting anything out there seems awfully unproductive.
After doing so much work though, why wouldn't you go the extra small few steps to publish? That way the work can be peer reviewed and the Scientific community has a chance to learn from it.