No, the browser is a very necessary layer; it provides very effective security isolation between the internet and the windowing system.
The mainframe days assumed that software was installed by the privileged admin and that users were the threat. It's now the other way round: most computers have only one user, but need to run all sorts of software that needs to be kept isolated from the system and each other.
Effective security isolation is a necessary layer. Browsers are one option to provide that, but there are others. They also provide ways to fetch, run, and interface with cross-platform, highly-efficient code from all over the world. That's both a blessing and a curse.
When I think about the volume of other people's software that we run while doing things online, I wonder how much of it is actually directly related to whatever I'm trying to do, and how much is incidental, unnecessary overhead.
Browser as it is now is a stupid and unnecessary abstraction. Even a humble Tcl interpreter running in a sandbox would have been many times better than all that thick client browsers.
This was one of the intended use-cases of rebol, IIRC. I know that code/program distribution over the network was, I believe isolation was a part of that (little time to research and confirm) or was intended to be at some stage.
Sort of interesting to think about this now, in the context of web apps and the recent Android instant apps.
The mainframe days assumed that software was installed by the privileged admin and that users were the threat. It's now the other way round: most computers have only one user, but need to run all sorts of software that needs to be kept isolated from the system and each other.