My biggest worry isn't about software security. My worry about online remote voting is What is to stop physical coercion of the voter?
When you have to gather at a centralised polling point to anonymously vote you can be damn sure no one is standing over the voter's shoulder twisting their arm while they vote. If you are voting remotely via a computer screen who is to know?
Physical coercion is possible also with paper ballots. A modern approach is to ask voters to shoot a picture to their voted ballot. No picture, big trouble.
For that reason taking a picture of our own voted ballot is a crime in Italy. Obviously is a crime also to ask people to do that. Cameras are not allowed in polling places and smartphones should be left outside the polling booth. However nobody asked me to do that the last time I voted. I remember they did many years ago. There were many phones lined up on the desk of the president of the polling place.
I've seen the following idea implemented: You can cast your vote as often as you want with only the last one counting. Sure, in practice there are a lot of things to consider to make it foolproof and it might be impossible to do securely - I don't know. But it's worth considering.
It would help against forced votes. But it's useless if some lazy person wants to sell his vote. With regular voting, there's a minor barrier in place. The incentivized voter has to go to the polling station and bring back some kind of proof. With online voting, someone could just take his ID (or e-signature device) and vote himself. The voter selling his vote may even don't know whom he "voted" for. Furthermore, someone might buy those signature devices off homeless/addicts/etc and build a massive voting farm.
When you have to gather at a centralised polling point to anonymously vote you can be damn sure no one is standing over the voter's shoulder twisting their arm while they vote. If you are voting remotely via a computer screen who is to know?