How about a spawning a separate process and using seccomp, or having the child process run as a different user perhaps?
There are a couple of advantages to doing it through the OS. For one thing, the implementation is shared across programming languages. For another thing, it's potentially more secure against an attacker trying to inject code.
I guess the disadvantage of doing it at the OS level is that you might have to write OS-specific code.
I think that mostly works (as an aside, last time I looked at this in Windows, we couldn't do something like sudo reliably because there was no secure way to provide the password without user interaction - it seems they've just released sudo for Windows in Feb this year). The OS can support this like how a mobile OS limits the capabilities of apps, which is more secure from a final standpoint.
But I was also thinking of the development and refactoring side of things. Guarantees and assurances. Sometimes I know the generic code I wrote is 'OK' but the C# compiler says no - you're not being specific enough, I can't cast that - so I have to do better. A while ago I was trying to track down programs that loaded data from certain network shares, and it was a bit rough because these calls were all very deep, and very occasional. Traces (or blocks) of system calls only work when the code fires, but my task was to find and replace them all, so we could delete the share (for various reasons string searches only got part of the way). If 'network access' was tagged at a high level, I could follow the trail to see what we actually access and report on that. We had a similar issue identifying jobs that called out to external APIs. We left a log running for a while to catch some, but some only run a few times a year and we didn't have that long. Adding a firewall exception later could take days and these jobs have turnaround times.
I don't know if this is at all feasible. It's just some thoughts I had back then.
There have been papers that limit the capabilities of a program within a certain context (i.e., within a function), some of which were implemented at the OS level and enforced with hardware protection (e.g., address space isolation).
The difficulty is that doing this sensibly requires new OS abstractions. It's one thing to put one in a research paper, but it's really tough to get this kind of thing into an OS nowadays. Another difficulty is OS level abstractions, with a few exceptions, cannot be used without a system call, which is cheaper than it used to be but much more expensive than just a function call.
A third problem is just that the programming language has a lot more semantic information, or at least it _can_ have a lot more semantic information, than can be fluently transmitted to the OS. There are approaches to deal with this (like having a richer OS / user land interface, almost impossible to get into an OS). In general, plugging into and extending the type system of some user land is probably going to be much easier route to take.
If the research world worked differently than it does, I'd have loved to continue previous explorations on OS / userland interfaces.
There are a couple of advantages to doing it through the OS. For one thing, the implementation is shared across programming languages. For another thing, it's potentially more secure against an attacker trying to inject code.
I guess the disadvantage of doing it at the OS level is that you might have to write OS-specific code.