Very neat project! It appears to run on qemu. Can someone tell me at the 30,000' view what's required to get it running on actual hardware? I'm imagining something like signed boot manager, hard disk drivers, etc. They seem to have a ton of USB stuff working which seems amazing to me, but I've been out of the low-level PC loop for a couple decades and don't know what prevents working on bare metal in 2024.
I've got a hobby OS that runs on real hardware (sometimes, on some hardware). Mostly it's just patience and debugging effort to get things going. qemu defaults to doing some things quickly instead of accurately, so it's easy to do things wrong as an OS and if you don't test regularly, you can end up with a lot of breakage.
I don't have a boot manager, my OS is multiboot compatible, so I can boot with qemu's multiboot support or PXElinux, or grub or whatever. IMHO, with the multiboot standard, there's no need to write a boot loader if you want to build an OS, and no need to write an OS if you want to build a boot loader.
Looks like this OS has a good selection of storage drivers, so should be good to go there.
At 30,000' I'm not seeing anything that would prevent this from booting on hardware. It seems to have a reasonable complement of modern, real-world drivers. On the other hand, physical hardware tends to have bugs, quirks and diversity that VMs don't have, so hobby operating systems may have trouble with it if they aren't tested on it.
I have managarm installed on a partition on my testing ThinkPad. The steps needed to get it set up were creating the partition, copying over the sysroot, and adding an entry to my grub configuration.
Theoretically, since it has a few real world drivers, it may be possible to run on real world hardware, but in practice, Managarm tends to get hung up very low level, at Eir or Thor (the two components of the microkernel). Since basically all the testing is currently done on QEMU, real world testing isn't really a thing for Managarm
Is this a hobby project or intended for real applications? How does it compare performance-wise to Minix 3 or the L4 family (one of which, Hazelnut, was also written in C++)?
people always say that C++ is not a good fit for kernels, but so far this is the only language I know where small teams or individuals are regularly able to create non-trivial hobby OSes from scratch that go from zero to GUI:
Hi, author of axle here - thank you for the shout out! It’s been a wonderfully fun and enriching project to work on over the years. I’m now working on XNU at Apple, so won’t be working further on axle for the foreseeable future.
Is the kernel really the differentiator there? seL4 is a Proven microkernel that is some 9000 lines of C. This ancient SO post[0] claims Linux is 140k lines. The kernel is just a tiny component of the many things required to get an OS up and running. I suspect most projects just peter out as the enormity of the complexity becomes apparent.
I’d love to know what the smallest OS is that can boot on an old MacBook, get to network (wired or wireless), and includes a basic browser (tui or gui) even if it doesn’t drive all the onboard hardware. That feels like it would be an excellent learning platform.
DOS just isn't much of an operating system, it's more like a bunch of helper functions preloaded into RAM, and you're free to do whatever you want with or without them.
You are seeing a creation date bias. And it doesn't help that there just aren't that many systems programming languages.
Until Rust in 2010, there basically was almost no motion in the "system programming space". Even afterward, there needs to be enough uptake to actually have critical mass. Only then can people start using it for projects.
Side note: D actually predates a lot of this by being from 2001, but, sadly, never seemed to get any traction. It seems like it had the misfortune of being about 10 years too early and that programmers just weren't ready for a new systems programming language at that point.
> Side note: D actually predates a lot of this by being from 2001, but, sadly, never seemed to get any traction. It seems like it had the misfortune of being about 10 years too early and that programmers just weren't ready for a new systems programming language at that point.
I was only a kid starting my university degree at that time, but I had been coding since I was 9 and I was very excited about D. What I remember is that getting it up and running was a cumbersome, fully manual process. The compiler was closed-source, delivered as a tarball without any kind of installation script, ditto for the libraries. I wrote some little programs and I liked the language, but in the end I gave up.
On the other hand, Rust has rustup and Cargo which are just amazing. I am sure that a big part of Rust's popularity comes from Cargo.
There have been a number of OS-type systems in Ada/SPARK, though at least mainly in the embedded space. E.g. the Muen verified separation kernel: https://muen.sk/
The two counter-examples I can think of with plain C:
* GTK+, though honestly speaking I think quality has dipped over the decades where C++ equivalents have flourished.
* Old school Win32 style. Counter to the bad reputation, I find it easy to be productive with in plain C once you adjust your mental model to its expectations. Though it's probably better from C++ than C, for a few convenience features to reduce boiler plate.
Old Win32 style received less attention than it deserves, I think. It's an object system in its own right—a Smalltalkish one, at that, with an (admittedly muddled) E-style separation of synchronous and asynchronous calls!—but one rarely sees it mentioned in the same sentence as Objective-C, C++, and GObject/Vala. Part of the blame for that undoubtedly rests on Microsoft’s incompetence at documenting concepts, but whatever the causes I’d definitely like to see that corner of the object-system design space explored more, with or without GUIs.
When win32 was dominant they didn't need to document it, people had the motivation to figure it out.
Successive waves of Microsoft people then tried to make it easier to use or replace it entirely, which, they never did so with the same quality as the original.
Unfortunately some of those misadventures became synonymous with building a windows UI.
Kind of. He stops somewhat short of admitting that Win16/32 windows are objects and their graphical representation is mostly incidental. That’s not a bad thing—the book is introductory on many topics, in a good way, and just dropping such an idea somewhere in the beginning portions would be more confusing than helpful for the intended audience. That audience would almost certainly not be helped by the observation that they are dealing with a sort of Smalltalkish/Actorish system with badly bolted-on multithreading—most of it wouldn’t have knowm what that meant, at the time.
Still, this is kind of a common theme. Perhaps “you had to have been there” worked as an approach to documentation for the first ten to fifteen years, but afterwards any newcomer is just guaranteed to get lost.
If you want to learn Win32, you need to read Petzold. If you’re interested specifically in how the dialog manager works, you need to invent a time machine and read a series of blog posts Raymond Chen wrote in 2005[1]. If you want the particulars on message queueing, you need to invent a time machine and read the chapter about then in Raymond Chen’s blogobook[2]. If you want to learn about STRICT and the very useful message cracker macros in WINDOWSX.H, first, you were probably reading Raymond Chen, because the official documentation is basically mum about it, but second, you need to track down that one KB article about porting to Win32 that accompanied one particular Windows 3.1-era Microsoft compiler[3].
If you want to learn DCOM95-vintage COM, you need to read Box and Brockschmidt and that COM Programmer’s Cookbook[4] thing that was somebody’s own private initiative inside Microsoft and is buried somewhere in the technotes and then the COM Specification[5] will be accessible to you, and only after that will the MSDN specification make some sort of sense. If you want to learn the how or why of COM marshalling in particular, you will perhaps be helped by some of the above, but really you need to invent a time machine and read a series of blog posts Raymond Chen wrote in 2022[6]. Only then will the MSDN reference be even slightly helpful (and even then not that much over the IntelliSense hints).
If you want to learn ActiveX ... I have no idea what you need to read! And that itself is indicative of a problem (took me five years to stumble upon Brockschmidt).
If you want to learn MSMQ/MTS/COM+/whatever, you need to read that one book[7] that’s still going to leave you with more questions than answers, and then maybe track down some MSJ articles from after COM+ was only a vague marketing term for something internal, but before .NET completely overtook all Microsoft communications channels. If you want to learn about using COM contexts, first, my condolences, second, AFAIK your only source for this Win2K-era, and I quote, “backdoor into the very low-level COM infrastructure” is Raymond Chen’s post from two decades later[8]. There’s no motivating spec for any of this, even one as scarce on details as the later parts of the COM one.
If you want to learn about WinRT internals, well, there are some blog posts random people on the Internet have written[9], and maybe you can spelunk some in the Win8-era legacy MSDN docs that are somewhat more explicit about what’s happening. If you want to learn about WinRT’s headline feature of “metadata-driven marshalling”, fuck you, it’s not explained anywhere, nor is an up-to-date MIDL grammar that would include the ways to adjust it a “priority” for that team—or maybe it’s just that I don’t have a time machine, but Raymond Chen is mortal too, and it’s been over a decade.
And it’s all like that. (Eric Lippert copied some of the articles on Active Scripting / WSH from his now-deleted MSDN blogs, but not all of them. Michael Kaplan’s blog, one of the best resources on Windows i18n, was speed-deleted from MSDN after he published a Vista leak, and the man himself died shortly afterwards, so for anything after that you’re out of luck. Etc., etc. Need I say there’s no type system spec for TypeScript?)
Again, if you were there at the time and you subscribed to MSDN, until 2005 or so I think you would’ve gotten basically all of the things I listed on the CDs, and more besides (Dr. GUI and Dr. International anyone?). But if you only joined in 1995 or 1998 or 2000, God help you. (WinRT happened after 2005, though, so the sum total of in-depth, under-the-hood narrative stuff about it is fuck all.) The references from that era are excellent, but as far as, again, narrative docs are concerned— C’mon, guys, you had Ted Chiang at your disposal and that’s what you managed?..
(Then again, it probably had very little to do with the technical writers themselves and a lot to with the preposterously hectic development and a management culture that just did not prioritize this sort of thing. But still.)
[5] What, were you expecting a microsoft.com link? Hahaha, nope! Just imagine you’re listening to Rick Astley, I guess. Anyway, there are two versions in circulation (originally as DOC). The “Component Object Model specification”, version 0.9, is dated 1995 and is easier to find, e.g.: https://groups.csail.mit.edu/medg/ftp/emjordan/COM/THE%20COM.... The “COM core technology specification”, version 1.0, is dated 1998 and is, as far as I know, only available from this one Russian-language homepage, last updated 2004 (and you can certainly tell): https://thegercog.narod.ru/index.htm?u=https://thegercog.nar..., direct link: https://thegercog.narod.ru/Files/Docs/ds/com/com_spec.rar. A lot of the same material is in the DCOM Internet-Draft and the ActiveX Spec drafts on the Open Group website.
I guess I’m just a bit obsessed with Golden Age Microsoft, having grown up during that time without understanding much of what was going on. My original point, though, was that they had some very nice, perhaps thesis-chapter-worthy points that ended up buried in the specifics of (and on more than one occasion, in the same grave as) their products.
Windows’s, well, windows as a object/concurrency system are one (even if the Win32 transition to multithreading is IMO conceptually botched). The actor model admittedly already existed at that point, but Mark Miller et al.’s work on E (the only other source I know for having separate synchronous and asynchronous sends) hadn’t even started at that point. (There are innumerable papers on various versions of actors, though, so I might have missed it.)
COM aggregation is another. It would probably have been much more successful had they used fat pointers, and there are a couple of old papers pointing this approach is a viable way to do implementation (as opposed to interface) inheritance. But I know of no precedent in the literature for the inside-out way WinRT builds its allegedly more conventional inheritance mechanism on it. (I guess I would’ve referenced BETA if I were writing a paper on it?)
DCOM’s “causality IDs” for reentrancy control are yet another. You can basically have calls to your object guarded by a recursive mutex whose ownership is passed along with any remote calls that that object itself makes. I’m not sure they are a good idea (recursive mutexes are perilous in the simplest of situations, and this is most definitely not one of those), but I’m almost certain it’s an original one.
That’s it, for now, but I’m almost certain I could find more, if only I knew where to look.
Recursive mutexes are a terrible idea, except maybe in context. In a world where you can SendMessage a message to yourself or to another window in the same thread, process the message synchronously, and then return back to where you were, you have essentially uncontrolled reentrancy. If you want to throw mutexes into the mix, they pretty much need to be recursive. Then you get to acquire the mutex in a message handler to protect against other threads, and everything sort of works.
In my own code, I try fairly hard to avoid this sort of reentrancy.
OOP is especially good for making UI toolkit. If you look at GTK+ and Win32 closely, you’ll notice that they all sport some kind of a homegrown class system, complete with inheritance and polymorphism.
It’s popular to dunk on OOP and its concepts nowadays, but I think that languages that straight away shun them because “OOP sucks” are an example of their authors overreacting to OOP’s dominance back in 1990s-early 2000s and tendencies to shove OOP into every nook and cranny, with a notion that if you don’t do it, or do not enough of it, your solution is inferior.
Holy hell. Linux kernel is object-oriented. Because it’s damn convenient for a lot of things.
It's probably more accurate to say that C++ devs are more common than C devs, and they are more interested / motivated to create operating systems than programmers in other languages. It has little to do with C++ as a language.
It's interesting to consider "what next" once an OS project reaches this stage. There are soo many directions a team can take, but also, none of those direction lead to a clear path towards massive user adoption.
There are obvious holes/gaps in what mainstream OSes offer today, however it is not clear how a project goes from here to addressing those gaps, and even if those were to be addressed, it is not clear how it could displace mainstream OSes.
We are clearly better off having projects like this, that give us options in case something were to go horribly wrong with mainstream OSes. However, what is the incentive to keep projects like this alive when the path ahead is soo unclear?
How does one disrupt the operating systems market?
Motor OS (https://motor-os.org) attempts to do exactly that, by focusing on a rather narrow, from a "mainstream OS", point of view, niche. Kind of "do this one thing better" approach.
rather than displace mainstream os's what id rather have is some way to easily switch os depending on my task, gaming? windows, anything else? linux (for now), and have that run directly on the hardware with little interference... like a meta task switcher os.
someone is going to come on here and tell me we already have this (i hope).
ive been out of computing for so long i dont know if this reliably exists, but back in the day this is what i would have wished for.
and since i dont trust windows to host my work os securely, and performance of games would be abysmal with the hosting roles reversed, what i really need is 2 boxes and a kvm switch!. the question now becomes, are there physical systems that turn on and off at the same time... i.e. share a psu but host different os's!
suppose i could build my own
am i looking at building a rack mounted system? heh, think i just found a project for myself.
1 psu, 1 gpu (for gaming), 1 kb/monitor, 2 hdd, 2 motherboards, 1 kvm switch, build my own rack out of wood, done! so the only extra cost really should be the 2nd motherboard and the kvm switch.
I was actually starting to think about how I could make a KVM due to the costs but this link looks really interesting. Just have a PC and a work laptop and switch between the two. Thank you!
You're talking about a type 1 hypervisor from what I've just read. Linux KVM/QEMU looks interesting. Do you think win 11 would work at near native speeds under this?
But this "not necessarily purely microkernel-y if very difficult" interpretation essentially means that Windows NT is pragmatic.
Because this is pretty much exactly how NT came to be the mess it now is.
I wonder if the parent commenter's point is that really, almost any OS design can be called "pragmatic", and therefore stressing it is particularly prideful.
"Like VMS,[24] Windows NT's kernel mode code distinguishes between the "kernel", whose primary purpose is to implement processor- and architecture-dependent functions, and the "executive". This was designed as a modified microkernel, as the Windows NT kernel was influenced by the Mach microkernel developed by Richard Rashid at Carnegie Mellon University,[26] but does not meet all of the criteria of a pure microkernel."
[...] the strict distinction between Executive and Kernel is the most prominent remnant of the original microkernel design, and historical design documentation consistently refers to the kernel component as "the microkernel".
[0] which I am old enough to remember as an adult and a graduate -- I remember particular criticism from academics in OS design around the time of NT 4.0, which as I (admittedly hazily) recall relaxed some of the distinctive design because 3.5's graphics performance was too poor and the graphics subsystem had to be moved essentially into the kernel.
I don't think it's hubristic to describe what philosophy it hews to. The opposite of "pragmatic" here would be "without compromise"/"principled". They know they are willing to make compromises in the name of expediency, so calling it pragmatic is just a declaration of intent, not a boast in front of the Olympian gods.
