The forums are also excellent. They are a little bit like the Arch Linux forums: People can be a bit snarky, but pretty much everyone is extremely smart and pretty helpful. Noobs are tolerated if they are willing to put in some effort.
This is making me wonder: Thinking about Dan Luu's research on latency of modern systems (https://danluu.com/input-lag/), what is the minimum latency one could get in a modern system (e.g, USB keyboard, modern display hardware)?
144Hz refresh is rapidly growing more and more common as displays become easier to drive. I'm sure the market penetration isn't as high as 4K screens, but I'm confident in it being the 2nd most popular refresh rate after 60Hz. I really refuse to work on 60Hz at this point. I notice significantly more eyestrain on lower refresh rate displays. Switching form a 144Hz display to a 60Hz feels noticeably worse.
I hate to sound dismissive but I feel like a lot of these projects put too much emphasis on booting and too little on something like say process semantics. I've written bootloaders before and it was some of the most uninspired code I've ever written.
Process management is hard. And that's if you throw out all the important stuff like "I want to run my programs outside the kernel" and "It sure would be nice if programs didn't have to share address space".
I recommend the OSDev wiki for a deeper dive but ultimately, implementing processes involves a lot of different moving pieces that need to work.
The most basic process manager would be cooperative, otherwise you'd have to write drivers for some external driver to regularly drive an interrupt.
Cooperative means the program will either print A or B and then call an interrupt into the kernel. The kernel would save the registers and instruction pointer, restore the one of the other program (which prints the opposite of what the first printed; A or B) and do an interrupt return.
This doesn't scale beyond a few processes that you have written yourself very carefully.
Any more competent process management will need to rely on regular timer interrupts, will have to keep track of processes and how much CPU time they got, have a thread to run if nothing else can run, manage processes when they die and has to be able to switch in and out of ring3 for running the actual process. On top of that it has to be performant and efficient, you have only a few hundred opcodes before the latency of your process management becomes too large.
If anyone wants to try this, on x86 getting a regular interrupt for preemptive multitasking is pretty easy, you can write an interrupt handler for the [real-time clock](https://wiki.osdev.org/RTC). This will work on any PC and takes about 15 lines of code to set up. Every 1ms, whatever's running will be paused and your scheduling code can run. Actually writing the scheduler is the hard part ;)
Then you want virtual memory and before you know it you are twiddling the CR0/CR3 registers and knee deep in multi-level page table management. At least that's how I remember it from my own 32-bit kernel experiment some years back, and as far as I got before I was in well over my head.
I second that. My degree project was a 68K-based RTOS kernel with a simple preemptive task scheduler, and the scheduler was one of the easiest and smallest parts of the whole project.
Realtime Schedulers have it a bit more easy IMO than non-realtime.
For RT you can write a scheduler that at any point knows if all processes will meet their deadlines and you can easily design an algorithm to get to the most efficient scheduling order.
In non-RT it gets harder because a process might not need to run at all and is just eternally paused on reading some dead socket, it might have higher priority than other processes, you might have 2000 processes to run but only 1000 timeslots left for the current timeslice.
A process might begin to eat up CPU time and you have to somehow preserve the interactivity of the system, ie prevent other processes from starving without starving the big process in turn.
And once you enter multi-CPU it gets even harder; coordinating multiple concurrent schedulers to run from the same task queue, which cannot ever take a simple spinlock without the system suddenly becoming dead.
The parent comment simply noted that setting up the timer interrupt is fairly easy (though if you want something precise and faster than 1ms and multicore it might get more complicated), it's the stuff after that were you are forced to solve NP-hard problems in a fast way.
You know, once a machine is running, you can do everything else that's limited by your own imagination.
I've been trying to figure out how a powerpc machine is being booted so that I can insert my own 'hello world' OS. Sure there is a BIOS equivalent somewhere in there, but how to figure out how and when to talk to it?
Booting x86 might seem like nothing, but booting some old SPARC for instance seems like black magic to me.
I guess it all depends on the documentation. Sun's is excellent e.g.: http://www.bitsavers.org/pdf/sun/sun3/Sun-3_Customer_Mainten... - on the Sun 3/60 you have a rather nice boot ROM which does similar things to UEFI but without the drama. It will let you netboot it over tftp, for example.
Which sort of PPC machine is it? Have you already managed to get e.g. NetBSD running on it?
Powermac g5 and/or power6. I got a CLFS kernel version 3.16 to boot and run (insanely fast) on the g5, but power6 is completely mysterious to me. There is some sort of RHEL linux based firmware running on it, but it seems there is no way to make it boot anything other than what's officially supported.
I think it's because the boot sequence is always pretty much the same on a given architecture, but process management is where things start to depend less on the hardware and more on the OS design.
Finding the kernel is not as easy as it used to be. You need to read the partition table on the disk. You need to be able to understand the filesystem that the kernel is stored on. This adds a fair bit of complexity.
On a modern UEFI system you actually get provided with an API that can do all that for you, though only FAT32 partitions are supported for the FS out of the box (you can load your own drivers for more).
If your boot partition is FAT32, UEFI makes things much much simpler.
True. And I think that's more than adequate for your homebrew OS. (Which can just have a second stage if you really need to boot off of an encrypted RAID-5 partition connected to a custom storage controller.)
I just read one chapter, which I thought was OK. It felt more like a book on how the C programming language translates to operating system concepts, rather than a book about the operating system concepts themselves. For example, the chapter I read discusses malloc() and indicates that it allocates memory from the heap, but it doesn't mention sbrk().
It often splits practical vs. theoretical concepts. One chapter might go over the theoretical scheduling algorithms, and then another might address practical solutions to internal fragmentation.
Disclaimer: I'm definitely not an expert, I'm reading the book to learn.
These days sbrk is rarely used, in fact, on Apple systems I am under the impression that it is depracated (A quick search confirms this impression).
The correct approach is to use mmap with the MAP_ANON flag (Which interestingly isn't doesn't seem to be fully documented in the Linux documentation...)
x86 16-bit real mode! It's certainly easy, but only because all the real work (initialising the PCI bus to speak to the video card, USB host drivers for the keyboard) is being done by the BIOS somewhere.
Does anyone know these days if the 16-bit boot environment is still "bare metal", or is it inside an UEFI or ACPI hypervisor of some sort?
If you're on UEFI, the 16bit environment is very likely running inside a UEFI "hypervisor" (ie, it puts the CPU into 16bit mode after setting up the appropriate interrupt handlers and hardware drivers). If you boot Windows or Linux via UEFI and not BIOS methods, then you never leave 64bit mode during boot.
Real BIOS that boots from 16bit is quite a rarity these days.
"Running inside a hypervisor" is not equivalent to the UEFI setting interrupt vectors to point to its own code.
Not sure how UEFI's "Compatibility Service Module" works exactly but if it acts like the original BIOS did, it's just a chunk of code that can be called, either by a program or be set as the destination for interrupts.
But there is no "VM exit" mechanism like there is in hypervisors. One thing a hypervisor does is intercept IO address accesses and does them on behalf of the client code (without the code knowing), don't think the CSM is doing this.
To my knowledge, UEFI does sorta do hypervisor-y things when running a 16bit bootloader since the GPU needs to be operated from 64bit ( via it's UEFI driver) and keyboard and mouse need to be emulated via UEFI drivers, which are also 64bit. That is unless the device has a CSM module and can run natively 16bit code during CSM boot, which not all devices can do.
Does UEFI start the same way as BIOS (firmware on LPC bus, PC starts at 0xFFFFFFF0)? Or is there a different top level mechanism?
My understanding was that the CPU still starts in 16 bit real mode (with an evil hack to sign-extend IP) but that by the time UEFI hands off to code on a hard drive it is in long mode.
I've not watched the series, but the full stack (ie. Cpu and gpu design in verilog, hdmi, bootstrapped mutitasking os dev, not your average FullStack marketing-speak) for a platform is done by bitwise: https://m.youtube.com/#/user/pervognsen
EFI will kill your program if it runs too long unless it disables a lot of the drives by exiting the boot environment.
Additionally, a lot of the EFI structures aren't intuitive or straightforward since they cover a lot of functionality, you'd still write a lot of code, probably more than the non-EFI variant.
0: https://wiki.osdev.org/Main_Page