With a paper-based solution, you (or your organization) can also count the votes yourself. With the added advantage that it's harder to manipulate, less prone to failure, and easier for non-technical people to understand.
Multiple people can't get access to the paper documents at the same time. And when they do get access there's no guarantee that they are getting the actual papers that citizens submitted. Even if you got citizens to individually sign each one with their private key and write the signature on the paper, you still couldn't prove that votes weren't removed
It doesn't involve all the interested parties and there's no way it could. That's what I'm trying to say. Maybe it's feasible for one watchdog organization to audit one polling station using your method. But it's not possible for every citizen to independently audit every polling station across the country, like you would be able to with the blockchain approach.