That teaches how to compile Linux kernel and software, not to create an OS out of thin air.
The go-to guide for anyone that wants to build their own operating system has always been the OSDev community wiki and forum: https://wiki.osdev.org/Main_Page
To be fair there is a lot you can do in userland. Graphics protocols, package management, isolation, security etc. And even more can be done by incrementally patching the kernel where needed.
It's just that Linux is kind of usable out of the box with just the kernel and /bin/sh. But for other OSes this isn't true, the kernel can expose a vastly different interface than the actual os which users interact with.
I've done every bit of the kernely/cgroupy/bpf/vfio/syscall stuff for a long time and I honestly don't think I could tell you pretty much anything about building an OS.
I actually came into this thread thinking, "damn, I have no idea specifically WHAT I would do/not do if I made an OS, or where to even start" not that I could at all. But I mean feature-wise, etc. Like, it's obviously a ton of apis that you call to invoke things at the system level but I have no idea what to do at the system level. Maybe you just communicate with APIs over there for everything on the motherboard. I know Nvidia has NVAPI I've used that a bunch through their SDK, not more low level. And you start with probably a super baby kernel..
There are also different styles of OS, like windows with the registry which I have broken and fixed thousands of times, and then nix which is pretty much file based. I don't know what else would be better than either of these, I hate the registry thing but maybe it has some huge benefit I haven't figured out.
When I look at C I see heiroglypics. Maybe me learning rust is helping with that. I do really need to learn it for security sake and (live)patching, etc. I write go and ruby so I don't touch memory much.
I'm the same about filesystems. I know a bunch of them and hve done tons of performance testing for SANs etc but couldn't tell you a thing about the lower level, making one, stuff.
edit: lol I guess a lot of this is installing drivers too
Don’t look at c specifically, look at syscalls. This is the real api of the operating system. It will also be confusing probably, since it is in some senses like thinking about how functions are implemented in assembly, but you don’t have to focus on the mechanics of how they work if that confuses you. One interesting exercise might be thinking about how syscalls are used to make up the standard library functions you are more familiar with.
I have always found it interesting how people who come from the top down (software -> hardware) get confused by the low level stuff, and myself who came from the bottom up (hardware -> software) gets so confused by non-low-level code. I.e. I have no idea how to write a program for windows, it's totally alien to me and I cannot make sense of the code.
Perfect! I've worked with a lot of C kernel security devs and I don't think I've ever seen one of them write something not Cish, like python/ruby/go/etc.
So, what exactly happens here - you need to write a kernel in some super fast low level language like C. I THINK The kernel that you write is probably talking to, for instance an Intel/AMD CPU, Motherboard, GPU, etc API to access the hardware.
Is this close/wrong?
If I made a completely new kernel how do I boot it? Does the kernel have its own API that the BIOS calls? Does the BIOS have anything to do with the kernel but instead something starts Windows and that loads the kernel? I think the kernel mounts/becomes aware of all of the hardware (apis?) potentially by the motherboard telling it what's on it. Does the BIOS just start any C app you point it at and they're just C apps that start the kernel?
Maybe it goes through the motherboard for most of it.. Maybe it accesses every piece of hardware directly without an API somehow?
This may be stupid questions but hopefully they make sense.. And thanks if you or anyone has input!
edit: There's a Boot processing of linux Wiki [1] thats pretty good
> After being loaded into RAM, bootloader (also called first-stage bootloader or primary bootloader) will execute to load the second-stage bootloader[2] (also called secondary bootloader).[6] The second-stage bootloader will load the kernel image into memory, decompress and initialize it then pass control to this kernel image.[2] Second-stage bootloader also performs several operation on the system such as system hardware check, mounting the root device, loading the necessary kernel modules,...[2] Finally, the very first user-space process (init process) starts, and other high-level system initializations are performed (which involve with startup scripts).[2]
> For each of these stages and components, there are different variations and approaches; for example, GRUB, coreboot or Das U-Boot can be used as bootloaders (historical examples are LILO, SYSLINUX or Loadlin), while the startup scripts can be either traditional init-style, or the system configuration can be performed through modern alternatives such as systemd or Upstart.
> The intermediate stage loader (stage1.5, usually core.img) is loaded and executed by the stage1 loader. The second-stage loader (stage2, the /boot/grub/ files) is loaded by the stage1.5 and displays the GRUB startup menu that allows the user to choose an operating system or examine and edit startup parameters. After a menu entry is chosen and optional parameters are given, GRUB loads the linux kernel into memory and passes control to it. GRUB 2 is also capable of chain-loading of another bootloader. In UEFI systems, the stage1 and stage1.5 usually are the same UEFI application file (such as grubx64.efi for x64 UEFI systems).
> Perfect! I've worked with a lot of C kernel security devs and I don't think I've ever seen one of them write something not Cish, like python/ruby/go/etc.
There are a couple of us true full-stack developers but you're very right that we're not common. Programming in low-level languages allow you to think about how, precisely, your memory will be laid out, but on the flip side, they require that you think about memory layouts. If you enjoy thinking about such things, you get used to planning for cache optimization and so on, and moving to python requires actively turning that part of your brain off and taking a „que cera cera" attitude to any performance consideration lower level than big-O notation.
Unfortunately I have never written low level code for a CPU. I have a background in electronics and mainly work with microcontrollers, where everything is C or assembly and you work directly with all the registers in binary/hex.
I know fundamentally it's all the same in regular computers, but I am unfamiliar with the structure of it.
Cool no worries. Embedded stuff is really cool I've always been curious about that stuff on like, assembly lines, etc. I updated my comment you're replying to with a bunch of details if you're interested.
Since OSDev is community project, so I think that says way more about the community than any strong bias on the part of OSDev. That said, a lot of "low level" is concerned with shoving bits around efficiently, not higher level abstractions useful for writing web browsers and such. C (for all it's issues) has a pretty good impedance match for that use case.
The reality is there have always been other languages[1], and not all of them codified the evil stream-of-bytes paradigm. We've been writing OSes in Ada since the 80s. Pascal variants were somewhat common back in the day (e.g. Charles River PERC, CDC Cyber NOS). Apple was on the Pascal train for a long time. Unisys Clearpath is still written (and sold) in their highly extended Algol variant (ESPOL)[2]. I've used OSes written in Modula-2 & Modula-3. There's a number of folks that are trying FreePascal as an OS language.
Of course, there was a healthy LispOS population at one brief, shining point in time (sigh...Genera). Tektronix and a few others had Smalltalk machines.
IBM still writes a lot of low-level stuff in PL/I dialects across it's proprietary product lines (i Series, z Series). Intel wrote it's first ones (e.g. iRMX, ISIS) in a PL/I dialect (PL/M). And CP/M is there too.
Then you can get to the really niche OS writing languages: JOVIAL, anyone?
[1] The source code for some of these legacy OSes is out there. Studying it might be instructive.
[2] Clearpath is the current incarnation of MCP from the Burroughs Large System mainframes (e.g. B5000, B6500, etc.). Stack based and very interesting; now runs as a VM on top of Xeon servers. Not only can you still buy Clearpath, you can download a hobbyist version to try out locally: https://www.app5.unisys.com/library/gmmail/emails/documents/...
The page you link to straight-up has a guide for evaluating if any given language would be suitable to implement a kernel in. How on earth is that unnecessarily married to the idea of "low level == C"?
For a long time, that was what everyone used. But there always were tutorials about other languages - C++, Pascal, even C#. Now Rust is gaining traction.
The go-to guide for anyone that wants to build their own operating system has always been the OSDev community wiki and forum: https://wiki.osdev.org/Main_Page