> As long as the posting of subscriber links in places like this is occasional, I believe it serves as good marketing for LWN - indeed, every now and then, I even do it myself. We just hope that people realize that we run nine feature articles every week, all of which are instantly accessible to LWN subscribers.
> Where is it appropriate to post a subscriber link?
> Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared.
I think posting this on HN does not defeat the purpose of gaining subscribers, but supports them. At least for me it worked. After reading some LWN articles on HN I decided to become a paying subscriber.
Telling people they are not allowed to post such links here actually IMHO hurts LWN.
Is "SubscriberLink" a way to share paywalled LWN stuff with non-subscribers? Damn, I had no idea. I just presumed they un-paywalled stuff every now and again (like after N days or so).
> I just presumed they un-paywalled stuff every now and again (like after N days or so).
All of LWN's paywalled articles become free for everyone to view after a week. During the week that articles are paywalled any subscriber can share currently paywalled content by creating a SubscriberLink.
LWN produces some of the best highly technical Linux and Linux adjacent content on the web.
I can honestly say the resources at LWN have helped my career immensely. The in depth series on topics like namespaces and io_uring directly allowed me to tackle some hard projects - without those articles I would have steered clear of "magic i don't understand" and the relevant projects would have suffered.
It's definitely worth subscribing if you have the means (and the low cost of subscriptions suggests a lot of readers here have the means).
Just to show support for corbet and his team. He could keep the articles paywalled forever, but is making them free after a week. Becoming a paying member is our way to say thank you for his fair attitude (and keeping ads at minimum level).
I work at totaly different field in IT, most articles are just "wow, that's interesting" for me and - I still keep LWN's subscriptionm and it is the only one I have.
Indeed. They even provide a mailing list you can subscribe to which will notify you whenever an article goes free. I think you need to have an account with them firstly though.
If they wanted to keep it subscriber only, they could achieve that by requiring users to be logged in.
It's clear that the intention is to allow for sharing (although there is the question of "at what scale?").
Then there's the page itself:
> Welcome to LWN.net
> The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net!
The people who run the site seem to have less of a problem with sharing its content than you do.
I only found out about LEB via links shared on HN. I haven’t subscribed yet but will consider it once I landed a job. I don’t think that sharing their most interesting articles is hurting them, more like the opposite.
Thanks for the reminder. I was a subscriber for a few years and then I stopped when I hit a long time crunch with no time to read much news. I just subscribed now that I feel I have some leisure time again for this and HN in general.
It should be pointed out that Corbet himself has come out in support of subscriber links posted here and at other places. That's why they provided that capability. That said it shouldn't be abused and I suspect if it ends up becoming too much they might remove that ability.
I agree with one of the other posters, we should support quality journalism, and lwn definitely is.
It's ok to post stories from sites with paywalls that have workarounds.
In comments, it's ok to ask how to read an article and to help other users do so. But please don't post complaints about paywalls. Those are off topic. More here.
I recommend browsing their index pages, and pause and ponder where else you'd find such high quality, curated, approachable content, in such concentration. You could suggest the kernel's own documentation, but that's more developer-facing.
I'm a huge fan of Rust (I write in it for a living), but I still can't help wonder how they are going to tolerate the compile times. A couple of drivers...no problem, but what happens once they get a large number of drivers in Rust? Compile times will definitely start going up considerably. I wonder how much this has been considered, or possibly they have no plans to do anything this extensive?
Given the disposition of kernel developers and the bespoke needs of drivers, I assume that Rust-written drivers will be neither pulling in dependencies willy-nilly nor will they be using generics all that heavily (but feel free to correct me if I assume wrongly). In that instance I don't think the Rust compilatiom process would impose any undue burden, especially since Rust can only be used for optional components of the kernel since it doesn't compile for all the targets that the kernel supports.
If you watch the actual talk and check code at https://github.com/Rust-for-Linux you will see that it’s actually pretty heavy on generics and macros. Eg for the trait that defines file system operations (https://github.com/Rust-for-Linux/linux/blob/459035ab65c0ebb...) and other bindings to C code. I was actually surprised of seeing those, since they certainly don’t help with telling people a story that Rust is an easy language. But it’s probably the best one can do when having to interface with predefined APIs.
But You are certainly right in that external dependencies won’t be an issue.
The trait there doesn't look concerning to me (e.g. none of its methods are generic (its methods do use associated types, but those are pretty trivial to resolve or generate code for; there's only ever one set of associated types per impl, whereas with generic functions there's the potential for combinatorial code explosion depending on usage)).
However, I do see a custom `vtable` macro in there, and indeed macros can be a concern for compilation speed if they're not implemented with compiler-friendliness in mind. But other than that, I'd say that file shouldn't represent a problem for the compiler.
The Asahi GPU driver which is currently being developed uses a proc macro to deal with the different versions of firmware that the driver must support (i.e. fields are added/removed in firmware updates, and the driver must support both).
That probably will never compile especially quickly. I'm not sure how sensitive kernel devs are to clean build times. Presumably doing a non-incremental build is relatively rare?
> That probably will never compile especially quickly.
Can you link the code? Proc macros are not inherently slow to compile, but most proc macros pull in the `syn` and `quote` crates for convenience, which are fairly heavyweight (and then most people experience proc macros most commonly via Serde, which is doing a ton of work on its own). In simple cases you can forego most of that work, and some libraries like https://github.com/Manishearth/absolution exist as lightweight alternatives in these cases. Depending on what the Asahi driver needs it could also benefit from something like this (although without seeing the code I'm not sure why it would need proc macros when it sounds like conditional compilation via `cfg` would suffice there).
For the professional work I have done in Rust the compile times are workable when using cargo workspaces and incremental compilation. Making sure to when possible work directly in a member crate compiling only that for the iteration cycle. That requires a DAG structure for all dependencies so does not work everywhere.
Getting CI up to speed is a completely different (awful) topic.
It might not be a big hurdle. OS developers aren’t as sensitive to compile times as app devs. An hour of futzing to bring up on a hardware dev board isn’t unheard of (although 5-10 mins would be more typical).
There was a big regression in compile times in the linux 2.2 to 2.4 time frame (the kernel grew a lot of functionality) but i don’t recall it getting much more than a little grumbling.
That was in the time of the Linux from Scratch book and Gentoo stage 0 builds. Some people don’t mind overnight compiles if it means they can specify just exactly the right combo of compiler flags to eeek out the max performance of their cpu.
Kernel compiles are a classic benchmark by which Linus has judged performance-related patches. Kernel developers do care(, mostly about incremental builds).
I worked on a (legacy now) larg-ish code base (mostly C++ '03 with some self-imposed restrictions, but extensive usage of templates) for an 'embedded' system (enterprise NAS). A full build took north of 6h on the reasonable beefy desk-side workstations available to us (we considered distributed builds, but never got to test it afair). Every once in a while there was some grumbling about the build times, but it never really improved (every three years or so we got better build machines, but by that time the code base had grown as well), partly, I suspect, due to https://xkcd.com/303/ and partly because we learned to work-around that by finding the Makefile in the tree which (likely) gave us a proper build in the shortest possible time for the files we were actually working on (not being helped by the most complex Makefile tree I've ever seen).
I very much doubt that such large build times would go over well in the Linux kernel community.
increased compile times are a reasonable tradeoff for correctness, no? hardware can always be thrown at the problem. as soon as someone does a c oopsie you're going to lose whatever time you saved to debugging
When tightly controlled, Rust compilation times are quite good. Without attention it can bloat tremendously with proc macros and excessive generics (turn them into dyns!), but it's totally manageable. Depending on your corners of the ecosystem, your default experience could be a few orders of magnitude better or worse than someone else's.
I haven't done much experimentation, but I've heard the same. Currently, for new projects I try to design with as many crates as possible in order to a) keep them small b) make as many 'spokes' (aka not dependent on other crates) as possible. This helps a ton and something I wish I'd have known before starting some of my large projects. I also wish I had used more trait objects and less generics - the overhead rarely matters in practice.
Traits vs generics: yeah, personally I hope to one day see a language that allows application developers to decide what gets monomorphized, rather than library developers.
It seems like you could usually just synthesize the boxing needed. So why would you ever leave that choice in the library's hands, which has no idea how it will be used?
I believe they mean spokes as opposed to "wheels with other spokes", i.e. leaves vs trees.
The best dependency trees are as short and wide as possible. It means fewer changes with each upgrade, fewer chances of conflicts between them, and generally more-parallel compilation and better build caching.
Which is true in every language, but every community seems to need to relearn it from scratch.
I did not expect Plan 9, the Linux kernel, and Rust to intersect like that! Would adding this module mean that I could finally mount a 9P filesystem on my Linux machine without the need for third-party software and FUSE?
You already could, actually! zgrep CONFIG_9P_FS /proc/config.gz to see if your kernel's configured for it, modprobe 9p if it's built as a module. The article's talking about doing an in-kernel 9P _server_ instead; I think this is just so that it can act as an abstraction of an SMB server, because you ought to be able to do a 9P server in userspace with no loss of capabilities...
> I think this is just so that it can act as an abstraction of an SMB server, because you ought to be able to do a 9P server in userspace with no loss of capabilities...
Correct, 9p file servers can live in user-space. At some point you have to mount the fs which on lunix/unix is done in-kernel.
In plan 9 the kernel is just a VFS that lets you build arbitrary file trees from collections of mounts and binds. On-disk file systems like FAT and EXT are served via user-space programs which open the disk partition (served as a file by sd(3)) and translates the 9p messages to EXT, FAT, etc operations. File servers can also provide synthetic files, e.g. an arduino can serve its io bits and/or internal variables over 9p (ninepea on github written by echoine.)
Since its a VFS all the way down and data moves via two way pipes mapped to fd's you can boot and pull root from ANY 9p server. Don't care if its CWFS, HJFS, FAT, EXT, SMB, SSHFS, NFS, etc. Kernel don't care as long as it sees files through a 9p connection over whatever 2-way medium you are using including rs232. You cant touch that level of flexibility with other operating systems.
Question: The author of the 9P server module "created an executor that can run async code in a kernel workqueue"; is this made a lot easier by the fact that Rust doesn't include an async executor as part of the core/stdlib?
The work on Rust in the Linux kernel doesn't include the standard library, and even if it did, having an executor around doesn't prevent you from creating and using a different one.
I think they are pointing out that because the Rust design has decoupled the functionality from the implementation, these kind of replacements are even possible in the first place.
Of course, if you use an alternative executor there are large swaths of the (non-std) async ecosystem that are not available to you (thinking of anything like relying on a tokio::timer::Delay as an example). But as you point out Rust in the Linux kernel already doesn't rely on the std, so they wouldn't even begin to imagine to use an off the shelf crate and expect it to work unmodified for their use case. I just think it is worth pointing out that this is a constraint we currently have.
Right, yes: if we had a design that built a single executor into the language or we had a runtime that included it, you wouldn't be able to replace that as easily. Because executors are, at most, a library mechanism, you can always replace them or run more than one.
Thank you, you're too kind. I don't know how much that experience of "make the compiler try to talk to a human in a way they can understand it" translates to other parts of life, though :)
It doesn't in rust but in many other languages there would have been a single blessed async library and trying to use anything else would have been very painful. The fact that rust doesn't particularly force the use of it's standard runtime is key to why rust is useful for kernel work.
Yeah, but I think what they're getting at is that Rust could provide a standard executor in the library and also be modular enough that you can use whichever one you want. The questions are orthogonal
How do these Rust kernel modules handle out of bounds access in an array?
In a C module it would be undefined behaviour leading to kernel panic or weird bugs, in a Rust userspace binary it would be a panic that terminates the process with an error message, what happens here? Would it be a kernel panic as well? or is there a possibility of handling it?
The Rust for Linux implementation converts a Rust panic into a Linux kernel BUG macro call. I believe this will expand to an invalid CPU instruction (at least on popular architectures), and if you're a kernel thread you die immediately with the kernel reporting the state where this happened. Obviously in some cases this is fatal and the report might only be seen via say, a serial port hooked up to a device in a test lab or whatever.
So, it's not a kernel panic, but it's extremely bad, which seems appropriate because your code is definitely wrong. If you're not sure whether the index is correct you can use get() or get_mut() to have an Option, which will be None if your index was wrong (or of course you could ask about the length of the array since Rust remembers how long arrays are).
I guess that's quite drastic for a checked out of bound access, when there's no actual memory safety issue and the compiler can simply return an error from the function, or do something else less drastic.
In Rust code, if you're not able to locally reason that an array index is valid, it should be written with .get() and the None case handled appropriately.
It's impossible to claim there's "no actual memory safety issue" when a program's invariants have been broken: all bets are off at that point.
> It's impossible to claim there's "no actual memory safety issue" when a program's invariants have been broken: all bets are off at that point.
When the underlying _runtime's_ invariants have been broken, not when the program invariants have been broken. i.e. you can recover from almost everything save for a VM error in a VM language like Java, since there's no way for the program to mess up the VM's data structures in a way that they cannot be brought back to a defined state.
Because in the common case you assume that, if your code is correct, all of your indexing will be in bounds, but for memory safety reasons we need a bug to be reported if memory safety would have been violated. So we allow direct indexing with a panic on out of bounds because it's the most ergonomic for that common case
I've come to believe ergonomics is a siren song here, mostly because recently I've been considering panics as forbidden as memory unsafety is... it's never okay for your embedded system or web server to panic, so don't act like that style is somehow preferable.
If you "know" the index is in bounds, you can get_unchecked. Otherwise you should get. Either would be a sane choice for the index operator.
Because it's convenient and familiar to most programmers. Not providing bounds-checked indexing makes some kinds of code very hard to write.
But note his problem also happens with integer division.
In Rust, a[x] on an array or vec is really a roughly a shortand for a.get(x).unwrap() (with a different error message)
Likewise, a / b on integers is a kind of a shortand for a.checked_div(b).unwrap()
The thing is, if the index ever is out of bounds, or if the denominator is zero, the program has a bug, 100% of time. And if you catch a bug using an assertion there is seldom anything better than interrupting the execution (the only thing I can think of is restarting the program or the subsystem). If you continue execution past a programming error, you may sometimes corrupt data structures or introduce bizarre, hard to debug situations.
Doing a pattern match on a.get(x) doesn't help because if it's ever None (and your program logic expects that x is in bounds) then you are kind of forced to bail.
The downside here is that we aren't catching this bug at compile time. And it's true that sometimes we can rewrite the program to not have an indexing operation, usually using iterators (eliding the bounds check will make the program run faster, too). But in general this is not possible, at least not without bringing formal methods. But that's what tests are for, to ensure the correctness of stuff type errors can't catch.
Now, there are some crates like https://github.com/dtolnay/no-panic or https://github.com/facebookexperimental/MIRAI that will check that your code is panic free. The first one is based on the fact that llvm optimizations can often remove dead code and thus remove the panic from a[x] or a / b - if it doesn't, then compilation fails. The second one employs formal methods to mathematically prove that there is no panic. I guess those techniques will eventually be ported to the kernel even if panics happen differently there (by hooking on the BUG mechanism or whatever)
Some people have argued that indexed access is a wart, but it would be quite heavyweight to always have to unwrap an option when accessing a known-good index:
let foo = [0_u8, 1, 2];
foo[0].unwrap(); // really?
Instead, indexing on arrays / vecs is (essentially) sugar for .get(index).unwrap(), if you don't want the unwrap behavior use get. This is very similar to Python, though Python throws an exception which obviously isn't available to Rust.
Would a kernel module be written as a normal Linux-targeting Rust program, or would it be more like a bare metal target with its own (user-provided) panic handler?
Worth noting that one doesn't need to use raw array accessing in Rust nearly so much as in C because you have things like iterators and for..in loops that will ensure correct access.
But I would assume it would be a kernel panic. It definitely won't be UB.
How does the Rust compiler handle, check bounds and ensures nothing bad can happen with a dynamic array that is passed to a Rust module from the Kernel, another C module or a C userspace program?
If an array and a length can be passed to a Rust module, I can just lie about it's size and, unless there is a runtime bound check (which can be slow), I guess bad things can happen too.
So accessing an array out of bound will have a runtime check that will call the panic handler, and that panic handler calls BUG() which means kernel panic.
I believe Rust in linux was made so that it never panics. Here's a patch that removes panicking allocations for example: https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@ke... (but I think all other instances of panicking were removed as well).
EDIT: Look at replies. "Linus considers panics acceptable in some cases".
Linus considers it acceptable to panic for some programming mistakes, since after all the C code also blows up if you make some programming mistakes.
One I ran into (in the sense of read about, not experienced) was if I flatten a Vec of arrays, it's theoretically possible that the flattened structure has too many items in it to represent as a machine word integer. If this happens the flatten operation will panic.
This can't happen in a language like C (or C++) because their smallest type has size 1, so all the arrays can't possibly be bigger in total size than the amount of memory, that's nonsense. But Rust has two smaller sizes than this. The relevant one here is the Zero Size Type, Rust has no problem with the idea of an array of empty tuples, such an array could have say, a billion empty tuples in it, yet on a 32-bit system it just needs 4 bytes (to remember how many empty tuples are in it).
We can see that flattening a Vec of arrays of empty tuples is a pretty wild thing to choose to do, and nevertheless even if we do it, it only panics when the total amount of empty tuples won't fit in the integers of our native word size. But the function could be asked to do this, and so it might panic.
[ You might be wondering how can there be two sizes smaller than C's single byte types in Rust. The answer is the Never type ! and its pseudonym Infallible. The Never type is Empty, no values of this type are possible, so not only does Rust never need to store this type, it doesn't even need to emit machine code to handle this type - the code could never run. This makes sense in Generic programming, we can write Generic error handling code, but it evaporates when the error type was Infallible ]
The 9P file protocol, he said, comes from the Plan 9 operating system. The kernel has a 9P client, but no 9P server. There is a 9P server in QEMU that can be used to export host filesystems into a guest. The protocol is simple, Almeida said, defining a set of only ten operations. His 9P server implementation works now, in a read-only mode, and required just over *1,000* lines of code.
... I wonder how many lines of code the Plan9 (C) server takes? (much less I suspect)
From previous example drivers, I'd expect that it's not dissimilar overall and mostly the cause will be style preferences e.g. maybe the C programmers loves one-line for loops and the Rust programmer chooses default Rust style which doesn't do that, or contrariwise the C programmer finds it helpful to write out complicated variable types across multiple lines and the Rust programmer was comfortable letting them be inferred instead.
If the C ends up hand rolling something Rust just has built-in, that adds up pretty quickly. For example Rust's arrays, strings etc. know how big they are, C's do not and so you need extra code to re-calculate or to explicitly store and retrieve those sizes. On the other hand it may be tempting to express a Rust type as having methods defined over it, rather than as you must in C using only free functions throughout, there's also a Rust discipline of always deriving common traits e.g. Clone, Debug and Eq, when they're appropriate, as a convenience, that's one line extra in typical style.
This Linux 9P server is a 9P-to-Linux-VFS translator. No such software component exists in Plan9.
You can think of the Plan9 kernel as a 9P multiplexer. 9P in, 9P out. The closest matching thing would be e.g. a 9P-to-ext4-in-a-file translator -- but it's always "9P server talking to $FOO" (or "9P server that makes files up on the fly").
Most of the complexity of this Linux 9P server would be in translating between the unrelated worlds of 9P and Linux VFS. A more apt comparison is the size of this 9P server vs an in-kernel NFS server -- they both translate something unrelated to Linux VFS.
(Oddly, grsec itself has expressed opposition to Rust in the kernel, I think on the grounds that their custom GCC plugins can't cross the language boundary. Now that there's work on supporting Rust in GCC itself, I'm not sure if that objection still applies.)
More seriously: many different security improvements are filtering into the kernel idea-by-idea, insofar as folks working on kernel security do the actual work of making them fit in with the kernel, and/or coming up with better alternatives.
Build up that absurd strawman so you can be angry at Rust continuing to eat the software world, yeah! Rah!
You won't use Linux, because it might have a bit of Rust in it, because some microcosm of Linux users will like that it has Rust and trust it more? That about right? Okay, you have fun.
I never said they shouldn't do it, or you shouldn't use it, they do what ever they want too, hence why i move away, to let them do what the f they want!
What junk am I subscribing to by thinking compiler checking of my memory access is neat, and by thinking it has uses in the kernel (which manages memory as one of its jobs IIRC)?
It's not so much trusting security because of language use as distrusting security because of language use. With the tools C gives you, it is impossible for multiple humans to write a large secure program. Rust doesn't guarantee security, but it gives you a fighting chance.
Im confused. You are going to change an OS because you think "trusting something based on use of a particular language" is a problem that might come up. Isn't that just you basing your trust level on the use of a particular language?
Religions generally don't phrase their actions as experiments, as this developer is doing.
Engineers test new techniques to see what works. Not all of them play out. The Linux kernel community is very good at sorting out the junk, and this is part of that process.
> Coding it in Rust does not, in fact, guarantee it is correct.
Of course.
> Coding C++, you need only choose known-correct primitives to get the same level of assurance.
C++ does not even have the means to express "known-correct" primitives. The compiler will not tell you if you accidentally access a non-thread-safe type from another thread. It will not tell you if you accidentally resize a vector while having an outstanding reference to one of its elements. These are not "bad primitives" you can blame - getting this stuff right requires higher-level reasoning which the C++ compiler cannot do for you, and the Rust compiler can.
> is exactly as safe as Rust eBPF, but likely less annoying to code.
Personally I find C++ far more annoying to code. I mean header files... In 2022? Text based macros? No standardised way to manage dependencies?
> Personally I find C++ far more annoying to code. I mean header files... In 2022? Text based macros? No standardised way to manage dependencies?
Yep. This is basically why I switched to Rust from C++, but I stayed for the coherent language design, the rapid pace of improvement, and the borrow checker.
> Hint: use of std::vector would not appear in kernel code, in any case.
They didn't ask for std::vector, just for a vector, which in this context is any extensible array type. So that seems like a disadvantage straight away. Rust for Linux provides Vec like your userspace Rust code.
Of course, the alloc::Vec in Rust for Linux doesn't have yolo-allocation, if you want space for 615 items in your Vec you will need Vec::try_with_capacity(615) rather than Vec::with_capacity(615) because you must decide what to do if the allocation fails, and if you want to push an item you'll need to try_push(item) not just push_item) for the same reason as the Vec might be full and unable to grow.
Rust's usual promises hold up though, if I try to Vec::truncate() a Vec I need to provide a mutable reference to the Vec (&mut self) and Rust won't give me one of those while I still have outstanding references, even immutable ones, to that Vec, my code won't compile.
Thanks! Could you point me to the C++ primitives that prevent cross-thread access of non-thread-safe data? I'd honestly love to learn about this.
If it's not asking too much, could you also point me to the C++ primitives that prevent dangling pointers as well?
EDIT: Also, the primitives that prevents the mutation of what a const reference refers to? (Much in the way that shared references and exclusive references can't co-exist in Rust)?
I'd hope the C++ primitives for the above all perform their job at compile time of course.
An extension to C++ that adds (pretty much) a type system on top of C++ to enable static analysis and prevention of race conditions. Widely used at Google.
Super interesting! That's exactly what Rust is doing too, except I think Rust's version (which pretty much relies on two "traits", `Send` and `Sync`) is much simpler, much more ergonomic, and much more complete.
I skimmed docs for std::atomic, and I believe what you're suggesting is a super-brief version of what was outlined here[0].
If so, thanks!
Through I don't yet "see it" that C++ has safety primitives that "equal" that of Rust's, I've learnt a lot about C++'s safety features from other folks' replies in this thread. So again, thanks!
By definition you can't safely access "non-thread-safe data" from another thread unless it was already safe to access despite being "non-thread-safe", but there are also many ways to make something thread safe.
You can prevent dangling pointers using strong/weak pointers or many other ways depending on the context, most examples are quite contrived though and the only "dangling pointers" I see are usually those relating to resources/handles to things from the OS where the lifetimes need to be managed more carefully, i.e. audio/video/gpu/etc, how would a rust program handle a handle to the currently playing audio device being unplugged and invalidating your handle? Every OS also has its own quirks, so how would it work cross platform?
Constexpr primatives/functions have compile time guarantees and can't be changed.
About the threading stuff, I think I wasn't clear initially. Maybe a rephrasing/simplification of my question will help: Does C++ have a compile-time way of ensuring, say, that mutex-protected data isn't accessed without acquiring a lock on the mutex?
> You can prevent dangling pointers using strong/weak pointers
I presume you're talking about unique_ptr/shared_ptr/weak_ptr right? If one avoids raw pointers and uses those, I think you cannot have a shared reference without ref counting right? (because shared_ptr uses ref-counting). Not that this invalidates your point, just thought it was useful to point out.
> how would a rust program handle a handle to the currently playing audio device being unplugged and invalidating your handle?
I've never had to deal with that situation, but I presume some mechanism similar to how the mutex in the stdlib has a .lock() method that returns an Option<MutexGuard<_>> (which forces the user to acknowledge/handle the case of a poisoned mutex).
> Constexpr primatives/functions have compile time guarantees and can't be changed.
Consider the c++ code here:
int a = 4;
const int &b = a;
a += 1;
Such a thing would be disallowed in Rust, which (though hard to see from this example) is the exact mechanism that prevents a Vec being modified while it is being iterated on. Does C++ have primitives that provide such guarantees?
Practically all valid Rust can be rewritten as valid C++. To make C++ reject most invalid thread-based code that Rust rejects, you can mark cross-thread-shared references as const&, atomic fields as mutable, and write a Mutex<T> class which wraps a mutable T, and only allows accessing a T& through a Mutex const& by locking the mutex. So implementing Rust-style threading in C++ is viable, but extra work to write a Mutex<T>, and the compiler lacks borrow checking to prevent you from making lifetime and overlapping &/&mut mistakes and committing data races. (http://www.drdobbs.com/cpp/volatile-the-multithreaded-progra... proposes using volatile& for this purpose, but many regard this as actually a bad idea.) Though in practice most C++ codebases sprinkle mutexes ad-hoc, relying on at best comments describing what data is protected by what mutex, with predictably disastrous results.
I feel Rust is a better language than C++ at implementing correct multithreaded code, but is so constraining that it makes correct single-threaded code hard to implement when it doesn't fit neatly into tree-shaped ownership (eg. graphs, GUI trees, and extending existing program architectures).
That would unnecessarily constraint `a` to be known at compile time though? I imagine that would be incredibly limiting.
E.g., if you're implementing an iterator that takes a reference to the length of the container being iterated, and you require that that reference is constexpr, that means the container will have a fixed, compile-time-known, size.
Since you didn't reply to the other parts of the comment, I'll presume that C++ doesn't have the mutex I'm talking about (apart from this attempt[0], which I just learnt about and commented on in a different reply), and that it's not possible to have non-ref-counted shared references while statically avoiding dangling pointers.
A lot of constants are known at compile time and constexpr solves a lot of problems. However, you do not need constexpr to solve your example- this is still a compile error:
There are many ways of forcing compile errors for mutexes that people have built, but they all come with their own trade-offs.
I've not encountered a problem where I specifically needed "non-ref-counted shared references while statically avoiding dangling pointers", so I'm not an expert on it.
What I'm referring to with regards containers is: preventing containers being modified while they are being iterated on (which is usually bad, because you will probably invalidate the iterator when modifying the container). That is what is prevented by Rust's references rules, and to my knowledge, isn't possible to protect against in C++ (statically).
> how would a rust program handle a handle to the currently playing audio device being unplugged and invalidating your handle?
You would either represent this as all the operations having a Result which can be Unplugged or whatever, or you might decide (as the designer of that library) that you'll eat the error and silently ignore operations when unplugged, offering an unplugged() predicate so that callers can check whether they got unplugged if they care.
If you mean, what if the system is allowed to just invalidate handles we've got for some reason, without telling us about that, and then after invalidation they just mustn't be used, then you're screwed, regardless of programming language, that's a pretty bad design and there's nothing to be done about it.
A more common design (e.g. for OS file handles) has an explicit call where you give back the handle (close in the case of file handles), promising you won't use it again. If the handle is meanwhile broken for some reason anyway (e.g. user pulled out the USB stick with the file on it) then the OS will tell you about the problem, but you still need to acknowledge that by closing the broken handle, you don't just suddenly find there's a different file behind your existing handle now. It's no trouble to represent this properly in Rust.
Let's explain a bit more about why Rust cares about thread safety. Rust types can have marker traits named Send and Sync. Send means "You can safely give this type to another thread" and Sync means "You can safely give references to this type to another thread".
For example lets say I have a Goose, which is three 32-bit signed integers named x, y, z, plus three booleans flapping, honking and hissing. All six of those elements are Send, so Goose is Send. I can give a Goose to another thread no problem.
But the type Rc<Goose> isn't Send. Rc is a reference counted smart pointer, like shared_ptr<Goose> from C++ except it's not for threaded software. It's a little bit faster, especially on some CPUs, but you can't safely use it across threads. Rust has another reference counted smart point Arc, and Arc<Goose> is Send for the same reason shared_ptr<Goose> is thread safe in C++ -- it uses atomic integers for reference counting.
Because Rust tracks the marker traits for me, I don't need to carefully read documentation for a new type I'm using FunkyTractor to check whether it's thread safe. If it's Send then I can give it to another thread, and if it isn't then that program doesn't compile and I go "Aww" and use say a Mutex to wrap my FunkyTractor so that multiple threads can access it safely, then it will compile.
Only the people actually innovating tricky thread safety stuff (e.g. building your own spinlock) need to care about this, and make decisions like "Should this type I'm creating say it is Send?" for everybody else it's automatic.
It is being worked on (at the build system level). Even so, I don't think Linux would support modules because it'd require some…unfortunate behaviors to support in `make`, namely at least one recursive make level.
Though if `gmake` is assumed, there is `libcody` support that's in a patch that can do it without recursive `make` at the expense of having an unknown number of outstanding compilation processes open waiting to figure out what their topological order is at build time. I do not know what the behavior of such a solution is in the presence of import cycles or unsatisfied imports, but that's also why I prefer the "scanning" solution.
FD: CMake developer working on C++ modules support.
I use C++ on a daily basis. I am extremely proficient in it.
It is an observable and downright undeniable fact that developers, even expert developers, cannot consistently write correct C++ code. This is true even if you have a greenfield project that uses all of the modern best practices and you don't have the keyword "new" anywhere in your codebase. People fuck up lifetimes. People fuck up iterators. People fuck up integer arithmetic. People fuck up concurrency. Using a subset of the language and relying on modern tooling like sanitizers will not save you.
It has been literally years since I shipped a memory usage bug. It just doesn't come up. There is no temptation to make memory usage bugs, because they would be extra work to code.
I understand that there is a sub-literature devoted to making up stories about how the C++ experience is exactly identical to C's, by people insisting they are active, competent C++ coders, but there are almost always clues that this is false.
You are free to collect data and publish a revolutionary result in ICSE or whatever. Until then I'll go with the mountains of data showing the existence of these bugs in codebases despite adoption of smart pointers, etc. You can even see the pain in the adoption of string_view as a clear example of how you end up with non-owning references in real modern C++ code that fucks up lifetimes and ends up with UAFs. You don't need a "new" anywhere in your entire codebase to still end up with UAF.
I also spoke about way more than memory errors. C++ has a million other footguns that people regularly fire beyond accessing deallocated memory or accessing memory out of bounds of an allocation.
eBPF doesn't prevent UB, just keeps that code from hurting the rest of the kernel right?
I have written c++ for 28 years and I can only recall letting something rust would have protected me from make it out of test once. That I know of...
A coworker found some UB in a rather large c++98 codebase recently. It wasn't exactly idiomatic modern c++. But the same person writing the same code in c++20 would just have had explicit and auto in more places. It was a use after free on the stack that wasn't terribly subtle but I only found it because I knew something was there. Cppcheck and clang-tidy were no help. Debug builds didn't exhibit the UB with g++ or clang, clang didn't exhibit it with -O3 (optimized out), but g++ did. Asan would have caught it, if they used asan (they don't) and if they had unit test coverage for that branch (also no).
So experienced c++ developers with years of experience (no novices on that codebase) made an error that can't be caught by static analysis, and requires 100% branch coverage for asan to catch. It eventually popped up in production.
I find modern c++ much more pleasant to write than rust. I like that it is easy to interface with the rest of the world.
I hate spending eons to get unit tests to find that last path theough the code in a meaningful way. I hate waiting for pipelines to run unit tests and coverage to find a mistake.
Modern C++ built with static analysis (including modernization checks) takes as long as a rust builds, but still doesn't catch as much stuff.
I'm not using rust yet, but I'm planning to start. I'm tired of this stuff, it's been almost 30 years.
Going from C++ to Rust, keep in mind that Rust does not mainly solve a technical problem, but a human one. It's not so much about making programs that run fast and don't fail than about making such code writable and maintainable by a wide variety of people over a long period of time. The initial Rust learning curve is steep but the competence curve is actually much flatter than C++, IMO. It's worth reiterating this because this is not as apparent to a seasoned systems programmer.
I do find that surprising. I have never used rust for more than toy programs, and I don't use a fraction of what is available, how can a language with so many features have a flat competency curve? I can easily write something that doesn't use traits or iterators, panics all the time, and calls clone all over the place to satiate the BC and call it a day right? I could create very unidiomatic code that was still correct, but not performant or easy to read/use. I suppose the fact I was able to make that list without ever having used the language in anger maybe makes your point... But with all those features surely I'm missing as many pitfalls as I know about?
I agree with your parent's assessment. And I think it may be because Rust's multi-paradigm approach allows you to use what you need when you need it.
> I can easily write something that doesn't use traits or iterators, panics all the time, and calls clone all over the place to satiate the BC and call it a day right?
You're basically not going to write your own traits until you've spent the time required to get something working. You'll copy and paste and modify to create another function.
If you're unfamiliar with iterators, then you'll write for loops. I was unfamiliar with iterators until I said, "I'm going to rewrite every for loop in this personal project as an iterator." Rust forces you to use lots of patterns re: memory, but forcing you locally to implement as a for loop or a iterator is not one of them.
I think a fear of clone() is wrongheaded. Most of the time it just doesn't matter.
When you think, ugh, this isn't the best I can do, it's time to learn iterators/traits, then you can modify the code to work then.
> But with all those features surely I'm missing as many pitfalls as I know about?
I guess my question would be: compared to what? I think the pitfalls are much, much, much more explicit in Rust, than C, like this won't even compile because this code has a condition which might be a null which I have not handled.
For app development, Rust really feels like a smallish language once you get over the hump.
>I can easily write something that doesn't use traits or iterators
Do you mean that you aren't using them at all or you aren't defining your own? Because I would expect any nontrivial rust program (outside of some obscure embedded context maybe) to make use of the traits that are in the standard library/your dependencies. Similarly, you can't even write a for loop without using iterators.
>panics all the time
That might be fine, depending on your usecase. Presumably if you were going to put it into production you would go through and handle the errors. Crucially, by grepping for `unwrap` and `expect` you can find the panics very easily.
>calls clone all over the place to satiate the BC
If it's not a performance issue then this might be fine.
>I could create very unidiomatic code that was still correct, but not performant or easy to read/use
This is true in every language. The value proposition of rust is that if you do write idiomatic code then it will likely be reasonably performant and easy to maintain.
>But with all those features surely I'm missing as many pitfalls as I know about?
Maybe, but not every program will need to use every feature. Quite a few of the more advanced features are mainly useful for libraries, for example.
> Crucially, by grepping for `unwrap` and `expect` you can find the panics very easily.
I write Rust code up and down the stack, and this is the thing that ticks me off more than anything else. You can get most of them looking for this stuff (and a handful of others), but there are just so many places in vanilla Rust that exhibit the Wrong Behavior By Default, e.g.
a[i] = x + 1;
Which looks so innocent but can panic in two places! Ugh.
If x's Add doesn't have panic on overflow then you get rid of the panic risk on the right hand side.
For example if x is Wrapping<u8> then it can't panic. Having chosen Wrapping<u8> you will presumably not be startled when 255 + 1 == 0
Or if x is Saturating<i16> again, no panic, but 32767 + 1 == 32767
Wrapping and Saturating really exist (well Saturating isn't stable, but it's in nightly) and if you're in a space where that's what you want, you can express it cleanly in Rust. Most programmers just mean the integers they learned in school which don't have a "maximum" or "minimum" they go on forever, and so overflow means the programmer's model of the world mapping those integers to a type like usize was wrong and this code is faulty.
[Edited to add: This doesn't quite work because in the end neither Wrapping nor Saturating implement Add over the normal integers, since it's not certain what you intend if you wrote that, better to have an error asking you to explain. AddAssign works though]
Now, there aren't any built in types whose IndexMut can't panic AFAIK, but we could build one, and if a was such a type then a[i] doesn't panic either.
For example we can imagine a is a (hypothetical) PythonDict while x is a Saturating<i16> and now our code just compiles to add 1 to x, saturating if necessary, and then shove the result in our PythonDict under the value of i.
Rust's slices (the most obvious thing you would index) have three ways to index them, if you are out of bounds the consequences are Undefined Behaviour; panic; and return None. Today those are provided by get_unchecked(), the Index operator and get() respectively. [[ Calling get_unchecked() is unsafe of course, but it's potentially faster because it has no bounds check ]]
I'm confident we sometimes want each of the three options, so if we're having the Index operator return None then get() presumably panics, or at least some new function is provided with that behaviour.
I reckon this just drives annoyance that [k] has to be unwrapped, while get(k) does what people actually wanted [k] to do. I could be wrong, but that's my sense.
I don't think I agree that we need three different things to handle them. We already have a general mechanism to convert an Option<T> to a T and panic on None, a method which shall not be named. So we just need get and get_unchecked, really.
If the index operator is offered as sugar over one of the basic operations, great. But why add sugar for a method call that we actually don't ever want to see in production code?
One can conceive of tooling that can find these panics for you so you don't need to grep, but I'm not aware of anything that's been written at this time.
I meant "flatter" as in "linearly increasing" competence. Once one knows the basics, it's easy to write "correct" programs in Rust that will not exhibit unexpected behavior. Of course, learning more of the language, one will write more expressive and performant constructs. The progression is made easier by the language and lib having relatively few pitfalls. Also, much care has been given to the developer's cognitive load through proper tooling (cargo) and informative error messages from the compiler.
"Fighting the borrow checker" has become a Rust meme but it's actually a good part of the Rust experience to have a "conversation" with the compiler. One is forced to reflect upon the implications of the app or lib's model, often leading to a better design or at least an understanding of the limits of the code. This is possibly similar to what LISP people describe when talking about their REPL-based workflow.
You‘re also missing that there is a chance of UB in your code base for which your compiler just assumes your original intention and will then go on to correct it
Correct. And eBPF is cool, but it's not a replacement for in-tree drivers. It also goes through a strict verification process in the kernel before being run which among other things, imposes limits on looping.
Error handling with exceptions mostly perform better on the happy path than error codes and explicit checking. Granted, on the sad path they perform much worse.
Of course returning an error code, then not checking it performs better than exceptions, but that's comparing apples to oranges. The same applies for comparing the same program compiled with and without -fno-exceptions.
exceptions opens the door to your control flow being rug pulled in exchange for... an error message? realistically all you can do is wrap chunks of code in a `try { } catch (const std::exception &e)` block and hope they've implemented `std::exception::what`. an enum based approach gives you both better error messages and certainty of what errors can occur and when.
I replied to the overhead claim, you are moving the goal post.
I don't think we will ever have consensus about the control flow aspects of exceptions. For example once you employ RAII it's easier to write error-agnostic functions with exceptions than with error codes. With error codes you have to explicitly propagate the error up, while with exceptions it is automatic. Sometimes this is desirable. It's arguable that the function isn't really agnostic to exceptions, as you have to employ RAII to qualify, but in modern C++ codebases that should be the default, really.
Having said that you probably don't want to introduce exceptions to a large code base that was designed with no exceptions in mind, and this definitely applies to the Linux kernel.
edit: enum based approach giving better error messages, really? You get a fixed error message per enumeration, but you can have a custom error message with exceptions that include context.
You wrote that if you were ever to code with exceptions, you would do a useless, disruptive thing no one recommends, and get undesirable results. Trolling.
Vaguely descriptive attacks that serve no purpose other than religious wars. When I see statements like that I see another person who thinks it’s simply fashionable to hate popular languages because they heard someone like Torvalds, Thompson, or Stallman say it. If you have a fundamental explanation like they do, then it's at least reasonable, disregarding the irrelevant context of a Linux kernel forum thread. But I rarely see that, instead I find mutter like "cpp is bloated, unstable, overcomplicated crap with ugly syntax, that's why I use rust, or nim, or any other pretty new thing that was conceived 35+ years later".
For instance:
> c++ is so brittle that a gentle breeze could cause a cacophony of soundness issues
Okay, how so, anything about C++ or just C in general?
There's a design principle here, rather than just some sort of hubris.
In languages as powerful as Rust and C++ you can express programs which fall into three categories, two of these aren't very interesting, the programs which are obviously valid, and the programs which are obviously nonsense. We know what to do with these programs, the former should result in emitting correct machine code, the latter should cause a diagnostic (error message). The problem is the last group, programs whose validity is difficult to discern. The bigger and more complicated your software the more likely it may end up in this last group.
In C++ the third category are treated as valid. In the ISO standard this is achieved by having clauses which declare that in certain cases the program is "Ill-formed, no diagnostic required" which means the standard can't tell you what happens, but you won't necessarily get an error message, your compiler may spit out a program that does... something. Maybe it does what you expected, and maybe it doesn't, the standard asks nothing more.
In Rust these are treated as invalid. If you try hard enough (or cheat and Google for one) you can write Rust programs which you can reason through why they should work but the compiler says no. You get an error message explaining why the compiler won't accept this program.
Now, if Rust's compiler doesn't like your program, you can rewrite it so that it's valid. Once you do that, which is often very easy - you're definitely in that first category of correct programs, hooray. The program might well do something you didn't intend, the compiler isn't a mind reader and has no idea that you meant to write "Fnord" in that text output and "Ford" is a typo, but it's definitely a valid Rust program.
In the C++ case we can't tell. Maybe our program is the ravings of a lunatic, the compiler isn't obliged to mention that and we are in blissful ignorance. This also provides little impetus for the standard's authors or compiler vendors to reduce the size of the third category of programs, after all they seem to compile just fine.
Obviously it'd be excellent in theory to completely eliminate the third category. Unfortunately Rice's Theorem says we cannot do that.
Inherently. Because of Rice's theorem you can't just ensure all the valid programs are correctly identified (basically you need to solve the halting problem to pull that off). But the ISO document doesn't allow you to do what Rust does and just reject some of the valid programs because you aren't sure.
Now of course you could build a compiler which rejects valid programs anyway and is thus not complying with the ISO document. Arguably some modes of popular C++ compilers are exactly that today. But now we're straying from the ISO C++ standard. And I'm pretty sure I didn't see a modern C++ compiler which reliably puts all the "No diagnostic required" stuff into the "We generate a diagnostic and reject the program" category even with a flag but if you know of one I'm interested.
So one person said creating the rust interface is very difficult to do correctly and the other said rust bare metal isn't complete enough for his use case
I called both of these (and compile times) when people first tried to get rust on the linux kernel. Maybe it's better to acknowledge these problems before you start instead of beating your head? I'm no psychic and I know I wasn't the only one who called these
> Maybe it's better to acknowledge these problems before you start instead of beating your head?
What's unacknowledged? They're doing exploratory work. That often involves developing under known and expected limitations to understand the problem space and have something to test with when progress is made towards addressing limitations.
The fact that writing unsafe rust code is harder than unsafe C due to more rules? The fact that Rust has little support of nostdlib which is what the kernel is using? There's also the fact that most of the time in drivers (in my limited experience) you almost always want unsafe things but I assumed noone would think or admit that one
This was a pretty weak article? All it basically says is that it can be done and adds some speculation that it might make things better. But not much testing have been carried out and the comparisons to existing code that have been made show the Rust implementation to perform worse.
Here in the Linux Plumbers Conference talk regarding the NVMe driver (the same one discussed in the OP), the benchmarks show nearly identical performance to the existing implementation. At the end of the talk the co-author of the NVMe spec (Matthew Wilcox) made a point to stand up and comment with how unexpectedly impressed he is with the performance: "I was not expecting to see these performance numbers, they are amazing." (at this timestamp: https://youtu.be/Xw9pKeJ-4Bw?t=9486 ).
Linux Weekly News focusses heavily on work going on in the Linux kernel, and often looks at features and patch sets which aren't quite ready for mainline yet, but could be an interesting shift in direction for the kernel in future.
The inclusion of Rust in the kernel is one of these ongoing bits of work which isn't ready for mainline yet, but has still generated a lot of interesting discussion in the past on whether Rust is suitable at all, or whether it's suitable in it's default form, or if it is suitable then what is the best way to start introducing it, or, or, etc...
And this is another step along that (long, convuluted) path. The proposed 9P implementation specifically - to me at least.
If you mostly care about the Linux kernel insofar as what features it can offer you right now then, sure, this might not seem that interesting. And that's entirely fair - there are plenty of other technologies out there that I have that level of interest in.
But for the people who take a deep interest in the upcoming work and future direction of Linux, I think it's quite interesting.
I'd agree that the article probably is a bit too niche, or forward-looking, for a generic tech website/magazine, and doesn't have a huge target audience. But for a focussed site like LWN, I think it fits in pretty well, and matches the audience that LWN targets. It's definitely the kind of article that I keep my subscription for.
This is the 5'th SubscriberLink over the last week on HN: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
Please, folks, subscribe, and let Corbet put the subscriber links here himself. The LWN weekly editions are definitely worth every penny.
Edit: I am paying LWN since 2012 and intend to keep paying for as long as there is LWN.