Ask HN: Do you understand your operating system?

[s2r2]

Dear HN readers,

this is something that bugs me for quite some time, therefore I decided to ask around:

Do you feel like you understand your OS? Have you really have acquired the feeling of trust and mastering its (inner) complexities? And, most obvious: do you care about this?

This question affects free OSes a lot more than proprietary ones, obviously, so if you care, it'd be great to hear from your experienced feelings corresponding to the OS you use or used, as well as the depth your technical understanding had to reach in order to gain the experience of 'groking it'.

I hope that's not too vague, maybe it helps to tell you that this question popped into my mind when I thought about switching from GNU/Linux to *BSD once again, this time hoping to leave some mysteries behind.

[pstevensza]

I thought I knew FreeBSD until I joined a couple of the lists and realised that I've hardly scratched the surface. It used to bother me that I didn't know. These days, I can accept it, and enjoy my occasional dip into the unknown when I need to change something.

[ovi256 → pstevensza]

  These days, I can accept it

And one day, you buy a Mac. Trust me. You want to.

[radu_floricica]

There is at least one cognitive bias involved when answering this questions. The less you know about something, the simpler you believe it to be. I'd expect Torvalds to say he has a "fair" understanding of linux.

[synnik]

Or, at an even lower level... do you understand your hardware? Not just disk space and memory, as most programmers need to know those... But do you know how the keyboard works? USB? How the processor receives/manages all the signals?

I did know all this, 10 years ago... I haven't kept up with newer systems, but even that basic understanding helps in many subtle ways.

[ErrantX → synnik]

Slightly as an aside but one source of amusement for me is programmers or sysadmins who cant tell you the difference between a serial and parallel comms link (beyond the obvious) and how one might work.

"Explain how a USB keyboard interacts with a computer" is one of my favourite interview questions at the moment. A surprising number of people can't tell you an in depth answer and you either hear:

- it's serial err no idea beyond that

- serial connection, I guess.. <creative attempt at answer>

- random bullshit about bits and serial (they can waffle for hours)

Really good for seeing them come up with creative solutions. Couple it with a whiteboard and work through the electronics with them and you can have a great interview.

(I started to do the whiteboard thing after one interviewee asked for a pen/paper to try and sketch out what he was thinking)

[ern → ErrantX]

Do the sysadmins and programmers you are interviewing need to understand the innards of a USB keyboard as part of their job function? If it's not part of their job description, then I am not surprised at all.

[ErrantX → ern]

nope, and their answer is fairly irrelevant.

If they have the balls to say "no idea, why do I need to know" that is someone we like. If they say that and then attempt to work out an answer extra bonus points. If they stumble they get no set back - it is a random question. If they bullshit they are marked down a fair way - not the kind of person we want.

(we design it to be a question you have to actively fuck up to be marked down on)

We sit and work through an answer together and you get a real insight into them outside their area of expertise (where you would expect them to be slick)

[aurora72]

You miss something the author's talked about and it's the transition from GNU/Linux to BSD

I assume he/she wants to have a more understandable OS; and because the BSD's excel at that area, I guess she/he's referring to the BSD's.

Windows --> Great but I still cannot understand some of its peculiarities such as unexpected crashes, sudden explorer.exe restarts.

MAC OS --> Great but I still cannot understand just why its USB connection speeds are interestingly low or why it has such an inadequate Unicode support for different FileSystems like NTFS.

Linux --> All great, but I cannot figure out why there are so many different package managers out there?

As for BSD, because it's structure is so well-founded there's nothing which is not understandable. Everything's got its place where you'd expect them: Ports collection under /usr/ports, and package manager is just the pkg_add command. There are no window-manager or desktop-environments pre-loaded and configured, you just chose what to do. And the kernel structure is extremely simple: You can add extra functionality to kernel easily with the Loadable Kernel Module (KLM) concept. That's it, everthing's under control.

[pmjordan]

I'm currently working on a block device driver, which means I'm digging around in kernels. I've found the O'Reilly books on the Linux kernel ("Understanding the Linux Kernel, 3rd ed" and "Linux Device Drivers, 3rd ed") as well as "Mac OS X Internals" by Amit Singh extremely helpful. The latter doesn't only cover the darwin/xnu kernel but also a lot of the OS X user space. All of them are quite technical and can be a bit dry, but they're certainly densely packed with information - and you can safely skip/skim most sections if you like. Linux Device Drivers is probably only useful if you're actually trying to write a kernel driver, I can recommend the others for general interest as well, though.

I've also got Windows Internals from MS Press, but I've only looked at it briefly so far. It seems higher-level than even the OSX book.

In general, I've found it extremely interesting to see how it all fits together, but I've got a general soft-spot for low-level hackery. (until recently that mainly involved game consoles)

[kbob]

There are some things I understand. I've written parts of Unix/Linux virtual memory, network stacks, process management, threads, interrupt handlers, synchronization primitives, TTY line disciplines, various kinds of device drivers. I've debugged parts of block I/O systems, file systems, NFS client and server, CPU schedulers. I feel like I understand those.

There are things I don't understand. I don't know the I/O architecture of a PC nor how the OS integrates with the BIOS. I don't know how the subdirectories of linux/drivers/ are organized, even at the first level. I've never designed a filesystem, and suspect there are subtleties I don't appreciate.

I never bought Bill Gates' testimony during US vs Microsoft that an operating system includes its web browser. In my arrogant opinion, an OS starts and ends with the kernel. So the next question is, how much do I understand about the rest of the platform? It varies.

I do know how the X11 protocol works, how the classic X server worked, how Xt intrinsics work. Do understand the classic Unix utilities and libc and their modern GNU reimplementations.

Don't know how an RDBMS works inside. Haven't kept up to date with compiler optimizations. Don't know how debug info is stored in object file and executables or even exactly what it is. Don't have any idea how that huge pile of libraries in GNOME (or KDE) is organized. Have only a vague idea of how the current X server is organized.

Even though the last (and first) time I implemented a page fault handler was 1986, I feel like I understand what they have to do, why, and what data structures they need. I'm sure the specifics are very different in any of today's OSes than they were in 4.3BSD, but I don't think that matters. OTOH, I think most of my graphics knowledge is obsolete.

So, in sum, I partly understand my OS. (-:

[rbanffy]

I care and I think I pretty much get what's going on under the hood of my Linux boxes as they seldom surprises me. OTOH, it's a complex system, with stack upon stack of hardware, firmware and software that makes really groking it (as I did with my Apple IIs in the early 80's) next to impossible. Even before the OS loads, an x86 PC is a very complicated animal.

As for the Windows notebook I am typing this in, it never ceases to amaze me and surprise me in almost always frustrating ways. It's indeed a box full of magic, but I am not sure its genies always work for me rather than against me.

I have tried a couple times to move to FreeBSD, but I always got frustrated with, what I perceived, a purist view of what a Unix machine should be - clean, simple and somewhat bare. I kind of like the magical things that happen in Linux, particularly package management. I utterly dislike the way it handles wireless networking, but I guess this is a feature nobody got right.

[starev]

It's hard to say what it is to "understand" an operating system, though. Knowing how it works in a fairly broad sense, or even knowing how specific parts of the kernel work doesn't mean you completely grok the system.

I mean, understand in the broad sense of "stuff all OSes do": certainly. I realize that there is memory management and etc. and etc. that is going on, and I have a basic idea of how that works, at least on Windows. (I use Windows, Linux, and Mac regularly, and can reach each from where I'm sitting.)

I have a decent enough understanding of Windows system internals and how it actually handles memory allocation, permissions, etc., but not down to the lowest level, just sort of a vague idea of what system files do what.

I assume Linux and OSX handle things somewhat similarly aside from implementation details, but I don't really know.

I haven't had the time to poke around Linux kernel internals (I have a pretty good understanding of the system down to that level, though), and I doubt I'll ever mess with OSX internals beyond what is necessary to have it 'just work'.

I figure I will master what I need to get things to do what I want. I find it fascinating, and at some point I'll probably have a project which requires me to hack on the Linux kernel in some way, shape, or form (unless someone else already has). In the meantime, though, I remain knowledgeably ignorant (I know that I do not know), which I think will have to do for now.

BIOS-level stuff and all of that is black magic to me. :P

It's hard to say what is and isn't necessary just from a development standpoint. From a curiosity standpoint, assuming unlimited time, complete knowledge of a system is better than limited knowledge. From a practical standpoint, such knowledge may make you a 'better' coder, but it may also tie you more to one OS rather than another. It cuts both ways there, though, because it could be that by understanding what does and doesn't change from one system to another, writing portable code is much easier.

So do I care that I don't know? Yes. Do I think it's the end of the world and must be remedied immediately? No. I will have plenty of opportunity to remedy my desire to know more about OS internals if I ever take on a project that requires me to know them exceptionally well. At some point that will inevitably happen, considering my interest in them. In the meantime, I'm content to know that I do not know and that in an ideal future, someday I will know.

[jrockway]

I do understand my OS, but it's probably better to not think about it. Knowing too much about UNIX leads you to want to throw it away and write something sane... but that is time consuming. Better to close your eyes and not think about the details.

[ErrantX]

Windows: yes Linux: pretty much (90%) Mac: nope

When I started my current job in computer forensics and security it shocked me how much I didnt know. There is a LOT of stuff going on in the internals that is pretty smart.

Believe me: you have no idea how bloody complicated a discussion of the Windows file system (a fairly simple high level subject right? :P) can become when you put a couple of forensic guys in a room :)

EDIT: I like a lot of the ways the Linux device/file system is organised. It makes a little more logical sense (though it might just be that it is more accessible) than windows.

[hdx]

Getting there with a little help from my friends: http://inst.eecs.berkeley.edu/~cs162/sp10/

[dirtbox]

I've got a pretty indepth knowledge of every OS I've used for any length of time. My first instinct with all things is to unscrew the lid and find out what makes it tick and for me, operating systems practically beg to be tinkered with,

[javery]

I knew Windows inside and out, used it from 3.1 and could do just about anything I wanted to do and fix just about any issue. 3 years ago I moved to Mac OS and I am still working on getting myself to the same place.

[ARR]

Windows: never tried to Linux: I know quite a bit but the important thing is that I know that I will understand everything I don't know. Learn by experience.

[simonw]

I don't at all, but I reall wish I did. The more experienced I get as a web developer the more the low level stuff matters to me.

[zackattack]

i run mac os x. no i don't understand its internals. i got it cuz it "just works"!