One of the first things I do when launching htop on a new box is to disable threads. The current way of showing threads by default is untenable because some programs launch hundreds of threads, making it hard to get an overview of a system unless you make the terminal size very large.
I appreciate the author wanting users to discover a program running amok and producing a bunch of threads (anecdote: I've never seen it happen), but there's a bunch of ways to solve it from a UX perspective:
* make threads collapsed by default
* add a column showing the number of threads a process has
* add the threads toggle to the bottom bar for easy discovery
* add a prefix to all threads, e.g. [THREAD]
All of these are a lot more obvious than the color green.
The authors story of having users ask him why processes are green is simply an indication that the user interface is hard to understand.
I feel like that's the crux of it. If I'm reading htop right I'm at ~140 tasks* and ~1000 threads.
The itch.io app that is hanging out in my system tray is currently sitting at ~200 threads by itself.
It's not helpful or educational to have pages of identically named threads especially when everything is only getting more multithreaded.
*and that's for my Cinnamon desktop, three firefox windows, thunderbird, steam, itch, a terminal and htop, lord knows if I was actually doing something with my computer.
We really need better tooling for this. It is possible but you have to parse snaps which I think is expensive ish. Would be great to get something useful info commonly used tooling though.
Me too, I really hate that! I wish they'd stop making that very obvious and simple mistake in reasoning!
...just so that _I_ can know that you know what you're talking about, can you elaborate on what that obvious mistake (which I clearly know about) is?
Dropping the joke - I assume it's that threads share memory in some way (so that the total consumed is not the sum of each consumption), but a. how and why, and b. why isn't it trivial for htop to represent that "correctly"?
> There's no way to know particular thread memory usage
This sounds like the sort of statement that is self-evident to folks with a CS education (I studied math, and all my CS/programming knowledge is self-taught). I'll add that to my _long_ backlog of things that I should research to get over my imposter syndrome :)
For most operating systems, threads each have their own stack space, and (if supported) thread-local storage, but share heap, constants (.bss), and static variables (.data). It's impossible to say exactly what resources a given thread is using from those shared resources, so htop just includes them all, for every thread.
I'm not aware of any other user scenario where launching htop and threads showing makes sense. Threads are details, while most people watch for OS processes first. That they are green doesn't help anyone if there's nobody to ask why or colours don't show properly. This is misguided newbie fixation that alienates those users who just need a reliable tool with sane defaults. Just another user feedback.
This is a lesson to all of us, that we might have very good reasons, but there's still a chance to be wrong about things.
While the author's reasoning mostly resonates, I agree with you and disagree with them on this particular point. For a typical user of htop, what matters is not "a process" or "a thread", but a killable unit. A piece of software that, should it misbehave, I can kill and possibly restart later. Killable units are almost always individual processes. Almost never threads - I can't think of a piece of software that can recover from having its random thread killed. Sure, I like to see threads - particularly in tree view - but I think the default should still be to hide them (with a more discoverable toggle than just Settings screen).
Yea, the reasoning isn't bad and try to be user-focused. It's just that the end result might not be real-life optimal, for most users or most use cases. You see this alot with UX. Sometimes it breaks new ground (iPhone), but it can also make things really hard to utilize properly (mobile interface for everything).
While it doesn't happen nowadays, I did some really weird things to selfcompiled/patched Firefox in the past, including killing tabs via killing threads in htop. But it wasn't obvious which thread was a tab, or if it was a tab at all.
Second this, killing processes is one of my main uses of htop. Having the interface be optimized for that by collapsing threads by default would be nice.
The bikeshedding in this thread is astonishing. Everybody in the world seems to have a strong opinion about how threads are shown in htop, about the shortcut for them, the default for showing them and their color.
Seems like that's easier to do than a thought about the actual substance of the blog post. Or being grateful that there is such a fantastic tool around, that there is a profound philosophy behind it, or that the original author shared it with us with some pretty good arguments.
But sure, let's keep discussing why shift-H needs to be made into a more prominent entry in the htop option screen.
I've only recently started using htop, and the last task I had to do in it was try to figure out a similar thread problem.
The default UI did not tell me if the threads had 'run amok'. There aren't enough lines on the display to tell whether a process with 10 threads on a normal day is now running 50.
So like you said, I had to turn off threads, then turn on the thread count column (which I never would have found without a search engine/stack overflow).
> * add a column showing the number of threads a process has
FYI the column exists already, but it's not enabled by deault. It's called NLWP, which I assume is short for Number of LightWeight Processes.
As for my own UI suggestions, I would hide threads by default, and stop displaying anything in all the columns of a thread where it's impossible for the cell to be different to its parent.
Maybe in tree mode, have all of a process's threads grouped under an expandable [threads] pseudo-process as well.
And everytime one toggles any setting that's is retained over htop restarts like tree view, the file gets edited. And you get diverging files everywhere. Annoyed me very much. Had to write a custom patch to disable some of it.
Indeed, why oh why is there no way to provide configuration preferences apart from doing so interactively? I would gladly wrap the command to provide it using flags, but no. I would equally gladly edit a configuration file (even if the syntax was wonky), but no. Instead I am left having to remember to set (and reset!) my defaults across users and machines.
For the benefit of someone like me who only uses Linux and bash on behalf of myself and the companies I work for and haven't studied it extensively, can anyone here tell me what the i is for?
I wasn't able to find it in man chmod on my machine, DDG failed me, as did Google.
(positive feedback to employees of DDG and Google: whenever I get assigned to the experiment where you show the snippet you thing matches my query it brightens my day, - even if the snippet shows that the result was irrelevant like here "<something about> chmod. I <something about the author>". I think Google used to nail this back in 2007 but anyways, seing the snippet in the front page helps a lot.)
> program running amok and producing a bunch of threads (anecdote: I've never seen it happen)
I've seen it in production. But the simple solution is to cap the number of threads to a reasonable value (ulimits, Kubernetes Pod Security Policy) and add observability for thread count.
A design decision that was perfectly appropriate in 2005 might not make so much sense in 2020. That's okay.
Threading is much more prolific today than it was 15 years ago. Multicore processors are the norm today and there are plenty of great libraries out there which make threading easier, and less error prone than it used to be. It is common today for language libraries to keep thread pools available for executing small concurrent operations. In such a case, knowing what thread was consuming resources probably wouldn't help much in tracking down the issue.
So yeah, I agree with your suggestions. I think they make a lot of sense today. In fact, I bet of the original author were creating htop in 2020, they would make these changes.
> You have no idea how many times I was asked “hey, why are some processes green?” over these 15+ years — and no, it wasn’t annoying: each of these times it was an opportunity to teach someone about threads!
That's because there isn't a easy way in htop to discover that.
htop is great, I've used it for a decade, but I didn't know you could hide the threads (Shift+H) until I read this today.
A "View options" / "Advanced" menu item on the bottom bar would help discover these option exists.
I also pressed some key, and some things turned purple, and there's no indication what view is shown, what the purple colour means, and what to do to turn it off.
I didn't know you could click in htop until today; it even works through tmux in mosh without any special configuration.
I think there's a lot to learn in the design philosophy of giving the power to users by default, but if you don't give your users the tools to learn this power they're going to have to ask questions if they want to yield it
This was my biggest take-away from the article too. It never occurred to me to even try interacting with a CLI application with my mouse.
I'd appreciate it if more developers considered 'discoverability' as a core part of their design philosophy. I think the author is close, but when you need to ask someone/search 'why is this green?' that's enough of a barrier that most people won't bother asking (or it'll be a small enough issue that they don't consciously notice it).
There's even a generally accepted solution for explaining semantic colours, a legend. Might be nice to have one down with the F-keys.
There is "F1 Help" in the bottom bar which lists these options, but not in much detail, so it's probably not enough for someone not experience with the tool.
For example, I missed that they use capital and non-capital letters: H (shift+h) toggles the threads, but i sat there opening/closing the help screen by pressing just h. Careful reading of the help screen would have prevented that. As none of the same lowercase/capital letters are next to each other, it is easy to miss.
"View options" are in Setup menu, which is mentioned in the article.
To be quite honest, I don't know any of the keys not mentioned in F-list at the bottom other than space and I think k, everything was always through Setup Menu.
On the other hand I casually looked through settings and found it immediately looking for something entirely different. Settings have a not overwhelming amount of lines each page so a quick glance led me to set whatever I wanted in the first place and things I found out by opportunity
Thanks for sharing that, now I know that I'm not the only one who has been using htop for years and didn't know that you could hide the threads with Shift+H. It's not even on the help screen, for crying out loud :D
Plus, after looking through all the options, I couldn't find an option to hide threads by default. EDIT: later I realized that pressing Shift+H automatically sets the option "Hide userland process threads", and I overlooked it because I was looking for a disabled option. So htop was actually smarter than I was giving it credit for. However, it definitely wouldn't hurt to mention Shift+H on the help screen :)
Ah, I swear I looked at the list three times until I finally found it. It's not listed under "Shift+H", it's under "H" (capital H = Shift+H). If they had a shortcut "Shift+1", they would probably rather look up the keyboard mapping to find out what character that is rather than writing "Shift+1". I still think htop is a great little tool, but sometimes it's really too clever for its own good...
'Purple' sounds like the 'Display options>Merge exe, comm and cmdline in Command' setting, hotkey-able with `m`. Yeah, echo'ing to the UI which hotkey was just activated could be helpful.
I wonder if the author's remark suggests a culture where the developer and their users are talking to each other more, and on more equal footing. Not always realistic, not always welcome by the grumpier ones, but certainly more hackerlike.
This is what quality software looks like: It has power features around the corner, which can be turned on by experts. But is nevertheless accessible by novices.
CLI is trending amongst dev's again, and we should take a lesson by htop how to design a good user interface.
I don't know if that's as true as "GUIs are now unfashionable to devs". It is hard to blame them since GUIs have regressed so significantly ever since the iPhone.
No, but it became a bit of a victim of being unfashionable. I think partly because of an influx of new devs who weren't in the position of having little choice but to learn at least some of the non-GUI way, particularly in the web sphere. As things have matured (arguably) and become more complex, and those devs have gained experience, they've started seeing the benefits of other methodologies.
I just hope it doesn't swing back too far the other way as it has before (I've been around long enough to have seen the cycle previously!). I'm a big fan for command line, scriptable, tools and plain text pipe-able output, and have been since being introduced to pipes & redirection all those decades ago, but we need to avoid the holier-than-thou thing that seems to develop when our old timer ways get the lime-light for a while. It only serves to put people off learning that they are great tools for many jobs and definitively the right ones for a some.
CLI has always been in vogue. It is TUI (text user interface – ncurses kind) has fallen out of fashion. Long time back, I used to use mutt, lynx, midnight commander etc on a daily basis. In mid/late 2000s, it fell off for me. Other than top, htop, dstat and a few others, there aren't that many that come to mind these days.
Check out newsboat, nmtui, ranger, w3m, mikmod, slack-term/weechat/irssi/finch, sc, mps-youtube, micro (text editor) just to name a few... Seems to be very popular among people who use tiling window managers.
In fact, with a good terminal multiplexer, mpv and fbv, all you need X for is stuff like web apps and Gimp.
They never stopped being useful, but at some point CLI was hidden by a lot of popular tools, like Visual Studio, Delphi, Xamarin, Xcode, Java IDEs, etc. So it was possible to program and rarely see a console.
Today, even some of those tools are embracing CLIs.
There seems to be a trend last few years where many of even the most traditional CLIs are getting questioned by modern replacements that provide a richer experience without compromising on the old values of simplicity. For example ripgrep, bat or exa instead of grep, cat and ls.
Lets call them what they are: a workaround to display graphical content in a terminal windows. We need them because SSH, because we linux UI is a mess, because UI is hard, in embedded, etc.
In an ideal world the concept of TUI would not exist.
Sometimes less is more. Having access to more features sometimes leads devs to bloat their applications and make them subjectively less usable.
A pathological example for me would be gvim. I tried using it for a while but I kept running into weird performance issues and general lagginess. Meanwhile from a terminal it Just Works. So I don't bother with it anymore, I always run Vim from a terminal and I never looked back.
In general I find that the terminal imposes two generally good constraints to application makers: everything must exist within a single window and mouse interaction is very limited (and not supported everywhere) so it's important that everything must be achievable with keyboard inputs.
These constraints disincentivize pathological mouse-driven "click click" pop up dialog fests which are the bane of my existence.
I do think that we are missing a good gui library that would direct developers to create lean applications (both from ux and resource consumption points of view) somewhat similarly as ncurses.
> In an ideal world the concept of TUI would not exist.
I have a hard time imagining that ideal world. TUIs have their own advantages that can't really be met with GUIs without restricting them and giving up some features of GUIs.
1. Being able to use it through SSH and other sorts of connections like a serial connection through the headphone jack on the PinePhone.
2. All text can be copy-pasted, guaranteed. Devs don't have the option to prevent that.
3. Its use is very simple to automate.
3.1. Don't need random sleeps while the interface is updated, just read and write to the terminal and let those calls hang if they need to.
3.2. No need to guess pixel coordinates of inputs to emulate clicks on them because the interface wasn't thought for keyboard use.
3.3. Sometimes you can even encode interactions in a stream of bytes that you can then paste into the terminal to execute on the TUI.
3.4. It's very simple for a script to pull text data from a TUI, but it's near impossible to pull it from a graphical window.
4. Terminal features like rectangular selection or keyboard-navigated copy-paste can be used for all TUIs.
5. Encourages more efficient use of screen real-estate.
6. Simpler themes, controlled by terminal settings. Simple to set the use of favorite font for everything.
7. More consistent look across TUIs because of terminal restrictiveness.
8. Can call a subprocess to take control of the terminal to do something and just come back when it finishes. This allows for simple ad-hoc multitasking through the terminal connection without special support of something like termux which requires foresight.
9. The fact that the application doesn't need to handle frivolous things like re-rendering after the window is moved or handle every mouse movement over it, makes figuring out what it actually is doing that matters quite simpler. I'm referring to the output of strace. It's largely more readable on TUIs than GUIs because TUIs don't need to do the frivolous things GUIs do.
TUI is just another way of writing portable UIs. Cross platform UI libraries exist, each with its own look and benefits, web-based UIs are another solution.
TUI also tends to have a certain aesthetic. I cut my teeth on computers where it was common for the UI to be made from ASCII text.
This is like saying that books made out of text are a workaround for the movable type printing press, and with high-DPI laser printing in an ideal world the concept of text books would not exist.
For me the design philosophy worked quite well. I learned so much about my system just by using it and trying to understand all of its output. And I am still learning after years of using it.
I actually like to just sit and watch it sometimes, it's fascinating to me how much a computer does at all times, even when it is idle.
When I first started using htop, I had the opposite misunderstanding to what the author describes. I spent a long time trying to understand why there were over a dozen instances of the same process, when I thought I launched just one! When I discovered that htop was showing threads instead of processes, I hit ctrl-h to hide threads and committed .htoprc to my dotfiles so I wouldn’t have to see them again.
The only complaint about htop is that it starts numbering CPUs from 1, while in real life CPU numbers start at 0. When I see core 18 is pegged with system(red), I instinctively want to ask perf -C 18 what's up... but that's not right, is it 17 or 19 again? If you're going to do that, don't show CPU numbers at all, just the collection of bars.
Indeed, but why is starting from 1 the default? Literally no other software does this except for htop, and the numbers are meaningless because they're off-by-one when you try to compare to anything else.
I actually agree with the issue reporter. I think threads should be disabled by default. Usually I’m hunting for misbehaving processes, I care about misbehaving threads second. The default view should be as concise as possible.
htop is one of the best tools available in your linux toolbox. Thanks, author.
While thinking about it, I'd like to inject htop in misbehaving kubernetes containers. Does anyone know if there is a static compile available somewhere or something.
You can use the kubectl debug command to add ephemeral containers to a running Pod. It's an alpha feature introduced in v1.18.
The ephemeral debug container can contain htop while the application container doesn't. This way minimal application containers aren't complicating debugging when something goes wrong :)
This philosophy is exactly what I had with PC Tools and XTree Gold.
The utilities which made computers interesting for me.
Having the ability to ruin my OS (all three floppy disks of it) made learning easier. Being protected from root level commands doesn't suit me on my personal machines.
"Mistakes are proof you're trying".
How can you ever learn any responsibility without any power, great or otherwise?
I'm 100% onboard with the author's design philosophy: "A good design leads a user to more powerful tools." I have not used htop but the screenshots show some things that I very much appreciate in a UI design such as the menu bar at the bottom with hot keys clearly displayed. There's no reason to make a user hunt for the hot keys or to guess at the possible function space.
Software, as I think of it, is a way to expose the "general purpose" nature of computers to users. Good software teaches users how to access that nature. Like a good teacher, that leading should be subtle and gentle and have the end goal of user empowerment.
>“stripped-down tools for mere mortals”, the latter being a design paradigm made popular by certain companies and unfortunately copied by so many OSS GUI projects
I appreciate the author wanting users to discover a program running amok and producing a bunch of threads (anecdote: I've never seen it happen), but there's a bunch of ways to solve it from a UX perspective:
* make threads collapsed by default
* add a column showing the number of threads a process has
* add the threads toggle to the bottom bar for easy discovery
* add a prefix to all threads, e.g. [THREAD]
All of these are a lot more obvious than the color green.
The authors story of having users ask him why processes are green is simply an indication that the user interface is hard to understand.