To bring up just one thing that doesn't have to do with trustworthiness: If you let an employee onto a 3rd party VPN you open yourself up to a whole new vector of attacks that you can't prevent.
And then there's the problem with allowing unrestricted, unmonitored Internet access with regards to auditing and establishing a timeline of events if you ever need to do so.
You can visit sites that aren't blacklisted on the company's network which makes it easier to social engineer you. You have less control over what stupid things your employees can do.
You're right, this wouldn't be any more dangerous than being on a coffee shop's wifi but you already don't care about network security if that's how you're working.
And then there's the problem with allowing unrestricted, unmonitored Internet access with regards to auditing and establishing a timeline of events if you ever need to do so.