Oh no, I'm well aware of those -- but academic computer science is even worse! It's very new to have artifact evaluation at all, meaning almost any published result that represents an evaluation of a system (as opposed to something purely theoretical) is probably bunk. Almost no academic systems are released publicly for evaluation, so reproduction isn't just something that should be done, but isn't -- it's entirely impossible.
This is basically the same as publishing a psych paper with a Methods section including a line that says "and then magic occurs". It's like an unsafe cast in a programming language -- you totally escape the guarantees provided by the wider structure of the academic system.
Academia in general has serious problems. There's a fundamental problem of incentive, and until that's solved, nothing's really going to change. But psychology is definitely unfairly maligned.
(It doesn't help that plenty of people doing doing what's almost psuedoscience are still harbored in a lot of psych departments, but their methods are still more rigorous than HCI research in most cases.)
This is basically the same as publishing a psych paper with a Methods section including a line that says "and then magic occurs". It's like an unsafe cast in a programming language -- you totally escape the guarantees provided by the wider structure of the academic system.
Academia in general has serious problems. There's a fundamental problem of incentive, and until that's solved, nothing's really going to change. But psychology is definitely unfairly maligned.
(It doesn't help that plenty of people doing doing what's almost psuedoscience are still harbored in a lot of psych departments, but their methods are still more rigorous than HCI research in most cases.)