Someone who would make a new OS, should define a completely new system call interface, as it is likely that now it is possible to conceive a better interface than 50 years ago and anyway if it would not be different there would be no reason to make a new OS, instead of modifying an existing one.
Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs.
Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on.
After having a usable environment, one can write whatever new program is desired, which would use the new native OS interface, but it would not be possible to replace everything at the same time.
Besides having a POSIX translation layer, which can be written using as a starting point one of the standard C libraries, where the system calls must be replaced with the translation layer, some method must be found for reusing device drivers made for other operating systems, e.g. either for Linux or for one of the *BSD systems.
Nobody would have time to also write all the needed device drivers. So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine.
The same as for user applications, if there is special interest in a certain device driver, it should be rewritten for the new OS, but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.
> Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on.
> So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine.
> ... but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.
I'd think most people making a hobby OS specifically want to do these things.
I also think most don't care about wide hardware compatibility.
Even if you do not want the new OS to run on anything else but your own laptop, that still needs a huge amount of drivers, for PCIe, USB, Ethernet, WiFi, Bluetooth, TCP/IP, NVME, keyboard / mouse / trackpad, sound, GPU, sensors, power management, ACPI and so on.
The volume of work for rewriting all these is many times larger than writing from scratch all the core of a new OS.
Rewriting them requires studying a huge amount of documentation and making experiments for the cases that are not clear. Most of this work is unlikely to present much interest for someone who wants to create an original OS, so avoiding most of it is the more likely way leading to a usable OS.
If you do not want those features, that means that the OS is not intended to be used on a personal computer, but only on an embedded computer.
For dedicated embedded computers, the purpose for an OS becomes completely different and compatibility with anything does not matter any more.
Not only personal computers cannot be used without a huge amount of device drivers, but even for a very simple server, e.g. an Internet gateway/router/firewall or a NAS server, the amount of work for writing the device drivers, the file systems and the networking part would be much more work than writing the core of a new OS.
Only for embedded computers the work needed for device drivers can be smaller than for the base operating system.
You hit it on the head: The point is purely for fun and learning. I want to learn as much as I can by rebuilding apps from scratch, etc. I had my first computer in 1977/78, a Commodore PET 2001, followed by a Vic-20, so if I can duplicate the bare system I had them and a PL to create apps, I am back where I started - having fun with computers!
> Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs.
I disagree. POSIX sucks. Build a hypervisor so people can run their applications in a VM and insist that native programs use the non-garbage API. It's the only way you'll ever unshackle yourself.
The point of writing a new OS is to use it, otherwise you do not get any of its supposed benefits.
If you do all your normal work in a virtual machine, what will you use your new OS for?
Writing any useful application program in a complete void, without standard libraries and utilities, would take a very long time and unless it is something extremely self contained it would not be as useful as when it can exchange data with other programs.
It is much easier to first write a new foundation, i.e. the new OS, with whatever novel ideas you might have for managing memory, threads, security and time, and then start to use the foundation with the existing programs, hopefully already having some benefit from whatever you thought you can improve in an OS (e.g. your new OS might be impossible to crash, which is not the case with any of the popular OSes), and then replace one by one the programs that happen to be important for you and that can benefit the most from whatever is different in the new OS.
For the vast majority of programs that you might need from time to time it is likely that it would never be worthwhile to rewrite them to use the native interfaces of the new OS, but nonetheless you will be able to use them directly, without having to use complicated ways to share the file systems, the clipboard, the displays and whatever else is needed with the programs run in a virtual machine.
Implementing some good methods for seamless sharing of data between 2 virtual machines, to be able to use together some programs for the new OS with some programs run e.g. in a Linux VM, is significantly more difficult than implementing a POSIX translation layer enabling the C standard library and other similar libraries to work on the new OS in the same way as on a POSIX system.
> If you do all your normal work in a virtual machine, what will you use your new OS for?
You write replacements for or properly port your every-day workflow to the new OS. You already wrote a whole new OS for some reason even though there are hundreds to choose from, presumably there is value in replacing your tools to take advantage of whatever you put all that effort into or else why bother? The VM is for things you haven't ported yet or less important workflows.
Besides, people run windows and do all their work in WSL all the time.
> Writing any useful application program in a complete void, without standard libraries and utilities, would take a very long time [...]
So does writing an OS and you've already decided that was worth the effort, yet you balk at rewriting some commandline utilities[0] and a standard library? Please.
> It is much easier to first write a new foundation [...] then start to use the foundation with the existing programs [...] and then replace one by one the programs that happen to be important for you and that can benefit the most from whatever is different in the new OS.
Any reason not to just do that with a VM? Forcing POSIX compatibility into your OS is going to constrain your choices (not to mention your thinking) to the point that you'd probably be better off just modifying an existing OS anyway.
> For the vast majority of programs that you might need from time to time it is likely that it would never be worthwhile to rewrite them to use the native interfaces of the new OS, but nonetheless you will be able to use them directly, without having to use complicated ways to share the file systems, the clipboard, the displays and whatever else is needed with the programs run in a virtual machine.
A: it isn't that complicated. B: if you can use them so directly without having to deal with the separation provided by a VM, it's likely you didn't improve their security situation anyway. Again, why not just modify an existing POSIX OS in this case?
> Implementing some good methods for seamless sharing of data between 2 virtual machines, to be able to use together some programs for the new OS with some programs run e.g. in a Linux VM, is significantly more difficult than implementing a POSIX translation layer enabling the C standard library and other similar libraries to work on the new OS in the same way as on a POSIX system.
I doubt it is as hard as, say, writing a brand new OS that's actually in some way useful. Why go through the effort of the latter only to throw away a bunch of potential by shackling yourself with a set of barely-followed standards from the 1970s?
[0] POSIX does nothing to help you with anything GUI.
> Someone who would make a new OS, should define a completely new system call interface, as it is likely that now it is possible to conceive a better interface than 50 years ago and anyway if it would not be different there would be no reason to make a new OS, instead of modifying an existing one.
For an example of how things like this can be done incrementally, you can look at io_uring on linux.
Nevertheless, the first thing after defining a new OS interface must be writing a POSIX API translation layer, to be able to use without modifications the huge number of already existing programs.
Writing a new OS is enough work, nobody would have time to also write file systems, compilers, a shell, a text editor, an Internet browser and so on.
After having a usable environment, one can write whatever new program is desired, which would use the new native OS interface, but it would not be possible to replace everything at the same time.
Besides having a POSIX translation layer, which can be written using as a starting point one of the standard C libraries, where the system calls must be replaced with the translation layer, some method must be found for reusing device drivers made for other operating systems, e.g. either for Linux or for one of the *BSD systems.
Nobody would have time to also write all the needed device drivers. So there must exist some translation layer also for device drivers, maybe by running them in a virtual machine.
The same as for user applications, if there is special interest in a certain device driver, it should be rewritten for the new OS, but rewriting all the device drivers that could be needed would take years, so it is important to implement a way to reuse the existing device drivers.