Hacker News new | past | comments | ask | show | jobs | submit login
Rust: Systems Programming for Everyone (infoq.com)
126 points by adamnemecek on Aug 20, 2016 | hide | past | favorite | 66 comments



The title makes it sound like Rust is easy to use or learn. It isn't. And I mean that in a nice way.

The whole point of Rust is to give you compile-time errors (as opposed to UB or runtime checking) when you are doing something unprovable and force you annotate your code so that things become provable. That involves understanding up-front a lot of things I learned the hard way while writing in C and C++.

If I just want to quickly get something done that involes hitting the bare metal, then C is probably easier than Rust. What Rust makes easier is getting up to a high standard of confidence in the coide. But easier is still hard.


We are very interested in making Rust more learnable, and are taking active steps to make it more so:

1. I am re-writing the book with the knowledge we've gained watching people try to learn Rust over the last 18 months. For example, compare https://doc.rust-lang.org/stable/book/ownership.html to http://rust-lang.github.io/book/ch04-00-understanding-owners... ; The former version takes a rules-based, theory-heavy approach. The new version uses a concrete example with Strings, a common thing that people new to Rust struggle with.

2. Error messages are being overhauled across several axes https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-com... IDE integration is also being worked on.

3. We created a dedicated "beginners" channel on IRC, it's been a rousing success so far.

and hopefully more things in the future.

  > If I just want to quickly get something done
I think one of the big differences between Rust and C is this: in C, you can quickly get _something_ done, but it quite possibly is not the right thing. In Rust, you have to get the right thing done. That's hard at first, but gets easier over time. By now, I get stuff done more quickly in Rust than I do in C, because I rarely fight with the borrow checker, and Rust's other productivity advantages come into play: numerics, cargo and crates.io, etc.


Re: 1. I tried learning Rust 8 months ago, and gave up because I couldn't quite get my head around ownership, borrowing, etc. I just read through the link you gave, and the explanation of ownership now is very good - far more approachable than it was, then, in my opinion. Awesome work. :)


Thank you so much! Glad to hear it.


I am very impressed with how well Rust tries to educate its user.

I especially like the error messages and the compiler's ability to tell you how to fix your error. And though I don't like to be dependent on IDEs, it would be great to see one which integrated that guidance into the editing.

But all of these are still palliatives around the fact that Rust is a complex and difficult language. Rightly so it's rules embody a lot of the practical wisdom of highly experienced programmers and lets them (and sometimes forces them) to express their advanced ideas.


  > numerics
It's too late to edit this now, but I meant "generics". Whoops.


Will rust have a canonical IDEs ?


There's no current plans to have our own special IDE. There's not a lot of agreement about this in the Rust community; many people don't use IDEs at all, and there's not a lot of coherence on either side: vim, emacs, VS Code, IntelliJ, Visual Studio, Eclipse, the list goes on and on. So we're interested in supporting the editors people want to use, rather than trying to get everyone to use the same one.


So the plan is for the compiler/tools system to expose an api that can be consume but everyone ? (a bit like C# new compiler ) ?


Yup!


I do not think that "provable" is the correct term to for the rust compiler. Type checking (safety) is only one subcategory of proofs. You can still have unproven behavior that will break the program at runtime that will not be prevented with a type checker.

You can find higher confidence in code being correct in other programming languages. "provable" is better used to describe interactive (Coq, Isabelle, etc.) and automatic (Z3) theorem proving languages. Interactive theorem provers generally require a large amount of work on the coders part. Automatic theorem provers are limited in the types of problems they can prove. I am sure they can still run into runtime issues.


I did not mean that the Rust compiler proves your code is correct. Or that it proves more than any other compiler.

But like all type-checkers, it does prove that your code obeys certain rules. In a trivial way this is even true of C -- but languages like Rust and Ada go further and prove that you are avoiding large classes of common bugs.


> I do not think that "provable" is the correct term to for the rust compiler.

If it was, in every sense, the right term, there would be a lot more fuss around that.

> Type checking (safety) is only one subcategory of proofs

I hardly know what I'm speaking of, but these lemmas on nLab [1,2] seem to imply more than that.

[1] https://ncatlab.org/nlab/show/propositions+as+types

[2] https://ncatlab.org/nlab/show/relation+between+type+theory+a...


You're right, I should amend this statement:

> Type checking (safety) is only one subcategory of proofs

Type checking as provided by Rust, Haskell, etc. only covers a subset of possible proofs. Liquid Haskell goes further to increase the range of possible proofs it covers through type checking by using automatic theorem provers.


I agree. I've been watching Rust for a long time, with some enthusiasm, like since it was Graydon's creation and safety wasn't the big focus.

I'm pretty pro-Rust, but I spent some time last night trying to write up some fairly trivial code in it. I hit the complexity wall very quickly. Because I don't use the language often, I have to relearn the core concepts of borrowing, etc. anew each time. It was quite frustrating and I kept twitching thinking how easy it would be to fire up CLion, write a CMakeList and write what I was doing much more quickly in C++11...


I gave Rust several trials in the last year, but I somehow ended up thinking that it's [maybe too much] hyped. Rust managed to get quite a bit of traction.

I've been programming in Haskell and Python in the last years, but I've done more of a decade with C++ only and lately begun some MCU work where I had to use C++ almost exclusively again. I don't have a strict preference anymore, although I edge in Python for new projects as it strikes a good balance.

I'm quite familiar with runtime type checking (hs), and hey, I've actually wrote a couple of programs in ATS (whose syntax is absolutely horrid, and second only to the compiler warnings), because ATS essentially translates to C without an extra compiler, allowing you to use it for any MCU supported by your toolchain.

I came out thinking that doing seemingly simple stuff in Rust, such a mutable linked list, is too complicated for what you get. Let's be honest here, writing a safe linked list in C++ (even more so with C++14) is _trivial_, despite C++ being anything but simple to understand as a whole.

This is aggravating if you think that in systems programming, mutable state is almost central to performance (caches and locality). Most of the efficient structures you end up writing are ad-hoc, mutable and with manual memory layout. Complicated code is a double edged sword: if I need to write complicated code to get compile-time insurance, it means that it's a trade-off to just jumping into unsafe territory where the payoff is small.

I somehow like the syntax of rust, but it's quite verbose. Almost too much in places. And I compare this to C++, which is not too skinny either. But C++ got much better lately. I also really don't like how rust inherited the JS style of wrapping closures directly as arguments of functions (think unwrap). It really reads horribly. I'd have vouched for dedicated syntax for such a common idiom. C-like languages (and C++ included) do not read as nicely as homoiconic languages in these cases.

If writing simpler code leads to less bugs, then Nim is actually a much better language. I wish it had more mindshare, as the core and language itself is far from being polished and regular as rust currently is. The current premise of compile-time safety of rust is good, but not a dealbreaker if you use a modern C++ compiler IMHO. It's pretty easy to wrap unsafe behavior in a class. It's actually easier than wrapping unsafe code in rust.


> I came out thinking that doing seemingly simple stuff in Rust, such a mutable linked list, is too complicated ... writing a safe linked list in C++ ... is _trivial_

This is really interesting to me as a curious outside observer of Rust. I conclude that either writing a safe mutable linked list in C++ is really not easy or Rust's type system makes it too complicated to capture simple invariants and properties.

Which is it?


They're not strictly the same, of course.

The trivial "safe" C++ linked list prevents you from dereferencing uninitialized memory, leaking resources and performing incorrect type conversions, which are the typical issues you'll find in any C linked-list implementation. But there's no compile-time check for the actual lifetime of the objects. This means you can manually destruct the same object twice, for example, and the compiler will not warn.

RAII can get you /very/ close, but since it's not enforced at the compiler level, Rust guarantee is definitely stronger.

My main point is that the strong guarantee comes also at a significant complexity cost, and I'm still not sold it's worth it in many conditions.


If you'd like a very complete answer to this question, http://cglab.ca/~abeinges/blah/too-many-lists/book/


Yeah, what I really wanted from Rust was basically OCaml without the GC. Or C++ with a proper module system, no preprocessor crap, some pattern matching and type inference, and a more reasonable syntax. Rust started that way and then the safety stuff took over the type system.

I can roll with it. It's probably worth it. I'd love a job programming in it. But it's not entirely the dream I had.


> but I've done more of a decade with C++

So have people at Mozilla (or people anywhere else) and there are still horrible bugs in Firefox or other similarly large projects.

> is too complicated for what you get

If you aren't convinced by arguments that safety is intrinsically valuable for yourself or your customers then another argument is that with safer languages you pay the price upfront (i.e. learning and programming with what seems a more complex language) instead of later spending weeks or months on catching a bug causing crashes.


I'm absolutely for safety and compile-time assurance. I wrote I've actually use ATS in embedded systems where I could to perform compile-time theorem proving. I used ATS, for example, to actually prove at compile-time the runtime heap memory requirements. This is not easy.

Rust is not on the front of research here, although the borrow checker is more pleasant to use compared to /many/ other implementations.

Memory safety is, of course, important, but security as a whole is so much more than just pointer dereferencing and lifetime checking. We got pretty far with C++, modern toolchains and OS support. Rust programs will be only marginally more secure by default than other classes of programs. Rust will not help with entire classes of other problems like side-channel attacks, any logical bug (like incorrect initialization of IVs, unchecked reflection), and so on.

Rust helps, and I do not criticize that. But as with any system, you have to evaluate it as a whole. Is it a big boost coming from C? Absolutely. C++? Not so much. Heck, you can use LuaJIT in embedded systems, and Rust has zero advantage there.


A bit of a strawman argument here.


I'll give you an alternate perspective. I've never touched C or C++ seriously, I code mostly in javascript, php and I have a passion for functional languages like haskell and elm. I've been following rust for a while and there's no way I couldve written multiple thread safe, concurrent programs, some with message passing between threads in C or C++. The code I would've produced in those languages would have been terribly full of bugs and unsafe, and yet in rust the compiler finds those errors for me and I was able to do it.

I think my knowledge of haskell helped immensely in learning rust though, since many concepts appear to be lifted from haskell. the records system is similar, the Option type is similar to the Maybe monad, Result is similar to Either. they both have algebraic data types, there are type aliases, there is a way to declare newtypes with unit structs, traits are similar to typeclasses, etc.


I don't think Rust will take off until there is editor integration of borrow checking similar to the way most people work in an IDE that checks syntax live.

It seems obvious in retrospect but if you can check syntax there is no reason to leave it for compile time. The danger is that you put time into things that you have to go back and change because you didn't know when something went wrong. I think the same thing can be said for borrow checking and any other static analysis integrated into a language. If it is left to compile time it is a guessing game with a lot less interactivity.

Of course some would say that that mentality is just for beginners, but we've heard the same thing about IDEs, syntax highlighting, auto completion, etc., and historically anything that makes programming easier and takes mental load away from a person has been a huge benefit.


The Rust people are hard at work on making the compiler incremental. I don't know if IDE integration is part of the plan, but it would certainly be great if Rust could expose the kind of incremental, stateful API that TypeScript does.

There's a good talk by Ander Hejlsberg about how it was designed from the ground up to support things like syntax highlighting and autocompletion, on the fly; as I understand, even syntax highlighting can be done by asking the compiler for the AST, as opposed to regexing, and it's all computed and cached as an incrementally built pipeline.

I remember using Eclipse for Java development years and years ago, which has an entire incremental Java compiler built in. As a result, it offered lightning-fast error display and autocompletion, because it never needed to compile more than exactly what you were editing. It was amazing, and ever since I have been disappointed that this approach did not catch on elsewhere -- either people didn't know about it, or they chose to ignore it.

Even Go -- whose toolchain is very much designed to be run programmatically by apps such as IDEs -- seems to have dropped the ball there. There is an autocompletion daemon (https://github.com/nsf/gocode), but it's not part of Go.


IDE integration is a major part of the plan.


I have a background in scripting languages, with some haskell experience, and I didn't find Rust all that difficult to learn. In fact, I've never written anything serious in C/C++ outside of when I was in university.

The rust compiler basically taught me how to write safe systems code, that's an amazing thing. Even for trivial programs, if I attempted to write it in C, it would be buggy and unsafe.


> The rust compiler basically taught me how to write safe systems code, that's an amazing thing.

It did no such thing: as one example, if there's an error in your model, would the compiler catch it? Maybe, maybe not.

Safe systems code requires a proper software development lifecycle (SDLC). Rust could be a part of that SDLC, but alone is insufficient to produce a safe system. Advertising it as such is common, but I think very misleading.


What is it you're referring to when you say 'model' ?


I was thinking of in the sense of model-based systems engineering (https://en.wikipedia.org/wiki/Model-based_systems_engineerin...)


the code is statically verified to be memory safe, so i dont really know what you're trying to get at


Being memory safe (and here we implicitly assume the compiler ecosystem is behaving as intended) does not guarantee the code is generally safe.


I was referring to memory safety. I would not have been able to write memory-safe C or C++ code, the Rust compiler helped me do that.

I don't know why you would think I meant 'safety' in some other way, considering Rust advertises that it is a 'safe' language, and they are obviously talking about memory safety.

It seems like you just wanted to construct an easy strawman to knock down.


I don't know Rust, but I know C and C programming can't be "easier" than anything else. The people that say that are either hard core C programmers that know how to juggle with void* and have had a career limited to targeting specific platforms or people that don't know what they're talking about. And the only thing worse than C is C++.

As for beginners, I simply can't believe that C can be easier than Rust, when C's idea of memory management is basically malloc() and free(), with everybody having a custom implementation, not to mention the gazillion string implementations, all broken and incompatible, along with having nothing resembling a portable memory model.

Sure, it makes sense in the context of the Linux kernel, but in the user space the fact that C and C++ are still used is a tragedy.


It's much easier to write simply code that compiles in C, and much easier to write complex code that is correct in Rust.


"For everyone" doesn't necessarily mean "easier".

C/C++ aren't "for everyone" because everyone has been hearing for decades that they are terrible languages and should only be used for certain kinds of things by certain kinds of people. True or not, that's basically what they've been hearing.

Nobody is saying that about rust. The message is more like "go for it, and tell us what you did; the compiler will help protect you (and your users) along they way".

If rust isnt easy, I think that's OK. Not ideal, but OK. Lots of people have goals that they're willing to work for.


Rust is significantly more complex than C and more complex than C++.

It seems to be a multi-paradigm language like C++. I guess it's advertised for system programming because it's looking for its niche, but I don't see it replacing C for low level stuff due to the learning curve.


I guess Rust may seem more complex, if you know C++ well, but from the side it may not seem like it.

For example, C++ Core Guidelines [0] is longer than Rust Book. Core Guidelines is itself a great example of things one needs to keep in the head while writing C++ code.

Look at implementation of something as basic as optional<T> [1]. The amount of things one needs to know is so high most C++ developers would never attempt it.

[0]: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...

[1]: https://github.com/akrzemi1/Optional/blob/master/optional.hp...


Using the size difference between the C++ code Guidelines and the Rust Book as measure of relative complexity seems misguided to me : Those two very different document, with very different purpose and target audience.


Which is interesting, because I implemented Optional<T> (well, Maybe<T>) in Nim v0.14 in 20 lines of code in a single evening, to see how difficult it would be!


The reason for the complexity of that implementation are compiler bugs/nonconfromance workarounds, a fully constexpr implementation and attempt to match a standard spec.


> a fully constexpr implementation

That's actually simpler to do in Nim than you'd think, but I understand what you mean. I might expand my implementation to be const and see how complex it makes it!


I feel like this is taking a very rosy-eyed view of C++.

C++ has a massive amount of accidental complexity. To mention just two: the difference between () and {} initialisation (for some types) especially in the presence of perfect forwarding, and a set of feature/their interactions that means there's a bug in the following very reasonable looking code (can you find it?):

  std::unordered_map<T, U> map;
  // ... initialize map ...

  const U *selected = nullptr;
  for (const std::pair<T, U>& p : map) { 
    if (condition) { selected = &p.second; } 
  }
  // use selected
(Credit to Effective Modern C++.)

Rust may seem more complex because the compiler is noisier, but it is significantly simpler to write correctly: a computer is finding the weirdest bugs.


That's not the point I am making. I am arguing that it's more difficult to go from idea to code in Rust because there are more programming concepts that one has to keep in mind. Additionally, the syntax is not user friendly.

You are right regarding the C++ accidental complexity though. Issues like the one you highlighted tend to happen when working with pointers. They are very hard to detect during code review (I didn't see it), but I would have rejected the code at review anyway because it should be replaced by std::find. Taking raw pointers into containers is also not reasonable except for vector.


I disagree that there's more programming concepts to keep in mind: the core ideas between Rust and C++ are the same (especially modern C++ with move semantics), but in C++ you literally have to keep thinking about them or your program is wrong, while Rust has the compiler checking them for you.

> I would have rejected the code at review anyway because it should be replaced by std::find

That specific example, sure, but there's other ways to have escaping pointers in for loops.

> Taking raw pointers into containers is also not reasonable except for vector.

The map types are explicitly designed to allow taking pointers, e.g.

> References and pointers to either key or data stored in the container are only invalidated by erasing that element, even when the corresponding iterator is invalidated. -- http://en.cppreference.com/w/cpp/container/unordered_map


I am not so sure, I've seen examples of Rust code where I had no idea what was going on, the syntax is heavy (e.g. sigils), it's encouraging functional idioms and introduces several concepts in addition to C++ besides ownership. I don't have time to list them, anyone can find what they are.

How many C or C++ programmers will understand this: (1..n + 1).fold(1, |accum, x| accum * x)?

Anyway, I am just offering my opinion as someone doing mostly C++ programming. I have no problem with a language like Swift or Go, but feel that Rust is too complicated for what it offers.

Taking a pointer into containers refers to &p.second. We now have a pointer to an iterator's member variable. That is unreasonable.

The reasonable thing would be to use an interator to keep track of the selected item, this is what they are designed for.


What's the bug ? (hopefully it's not just a missing break statement.


The problem is that the pair in the for loop should have a const T as the first parameter. Because it does not have that, you are actually getting a temporary back and storing a pointer to a temporary in selected.

The fix to avoid this is to use auto. If instead of pair, you used auto&& p: map, this would not be an issue


How do you measure complexity?


I measured it by looking at posted Rust code/projects on HN an relating that to what I know about C and C++.


That doesn't say _how_ you measure it when you look at the code.


watched it this morning with a fresh coffee and mind. Absolutely brilliant talk for anyone who wants to dig deeper into sys-programming. As a systems developer Rust is the first new language in 20 years that excites me as a serious contender and alternative to C. The talk discusses also higher level aspects such as closures and FP-paradigms.

Probably too early to say if Rust will become the lingua franca of system-engineering. I'd love to see them succeed and hope that it isn't trying to be everything for everybody. We already have enough of those.


PS: I love that Mozilla uses Rust for the media-parser to boost security. But in the context of wider system engineering, there's a danger that crates.io will become another malware distribution system (like dockerhub[0]).

Personally I think TUF[1] is still too academic and at least for now doesn't offer practical solution to this yet.

[0] https://blog.valbonne-consulting.com/2015/04/14/as-a-goat-im...

[1] https://github.com/rust-lang/crates.io/issues/75

EDIT: typos


They're slightly different problems. Docker images are, as everyone loves to point out, immutable. Once built, they're built, and if you want to patch a vulnerability you have to rebuild and replace.

A lot of folk on Dockerhub are just folk. They don't have security teams keeping an eye for relevant CVEs and triggering rebuilds.

I work for one of those companies (Pivotal) that does have a security team keeping an eye out for relevant CVEs. I work in buildpacks. If you send an app to a Cloud Foundry installation, you used the rootfs image and buildpack binaries that we built. When a high-value CVE lands we commit to providing replacement bits within 48 hours. We usually do it in much less.

But even with extensive automation -- and ours is pretty extensive -- this requires humans to watch the firehose of flaws. Most people who write a Dockerfile and forget it don't have that luxury.

Incidentally, Docker now offer security scanning of dockerhub images, for a fee.


> Incidentally, Docker now offer security scanning of dockerhub images, for a fee.

This is weird. I thought it would be in their best interest that dockerhub's images would have better reputations.


Well I think I may have mis-stated it. They offer scanning for Docker Cloud, which I understand to be their private registry / dev platform / once-and-future-PaaS. So it's an additional paywalled feature for private images, not for the public ones.

And, in fairness, it's a surprisingly tricky problem which many well-heeled customers see as worth paying for.

At Pivotal we've been working on "AppDog" to do something similar: inspect running applications on Cloud Foundry, tell you what dependencies are installed, what their licenses are, whether there are updates available and so on.

And yeah. It's harder than it looks. There are edge cases upon edge cases.

(Of course, nothing I say should be considered official comment etc etc).


I think they also offer scanning for repositories on https://hub.docker.com/account/billing-plans/ but only if you pay.

> Docker Cloud and Docker Hub can scan images in private repositories to verify that they are free from known security vulnerabilities or exposures, and report the results of the scan for each image tag https://docs.docker.com/docker-cloud/builds/image-scan/


I need to get better at actually reading people's websites :)


While i love rust, i do agree with @lkiux it can at time be over hyped.Rust memory feature (borrow checker ?), which seems to be rust claim to fame takes too much mindshare. Between RAII,deterministic destructors,move semantics, shared_ptr and allocator, it seems to me rust memory management feature is addressing problems that peolpe had 10/15 years ago forgoing the advances made in C++ and Dlang.

The feature IMO which make rust really great, usually only get mentioned in passing : well define compile time environement and compilation process,fast compile, sane macro system, ML like type system with pattern matching, great tooling and ecosystem out of the box.


  > forgoing the advances made in C++ and Dlang
Which ones are those?


"RAII,deterministic destructors,move semantics, shared_ptr and allocator" ?


Maybe I misunderstood what you're talking about then. Rust very much did not ignore those things. In fact, Rust has (most of) those things, but with stronger guarantees, and/or putting them more core to the language.

(Allocators are the asterisk here, those are still a work in progress.)


I not familiar enough with rust, but my point was that adding those feature to C lessen the need of a borrow checker (not that rust don't have those features.)


Move semantics is a great advance - we can have use-after-move bugs now, just what the world needed!





Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: