I wrote a lot of rust, but after some years it still feels unproductive. I do a lot of zig now and I am like 10 times more productive with it. I can just concentrate on what I want to code and I never have to wonder what tool or what library to use.
I know rust gives memory safety and how important that is, but the ergonomic is really bad. Every time I write some rust I feel limited. I always have to search libraries and how to do things. I cannot just "type the code".
Also the type system can get out of control, it can be very hard to actually know what method you can call on a struct.
I still think rust is a great tool, and that it solves tons of problem. But I do not think it is a good general purpose language.
> but the ergonomic is really bad. Every time I write some rust I feel limited.
> But I do not think it is a good general purpose language.
Remember that this is not a sentiment that's shared by everyone. I use Rust for tasks that need anything more complicated than a shell script. Even my window manager is controlled from a Rust program. I say this as someone who has been programming in Python for nearly two decades now. At this point, I'm about as fast in Rust as I am in Python.
I tried to get into rust for many years, I'm now in a C/CPP job (after Java/Python/Ruby and other gigs). What I've come to understand is that Rust's lifetime model is very difficult to work with whenever you have a cyclic reference. In C/CPP the same holds, but you deal with it through clever coding - or ignoring the problem and cleaning up memory later. Java, and other GC'd languages just work for these structures.
While the Rust devs believe such cyclic references are rare - I think this speaks mostly to the problem domain they are focused on. Relational models are everywhere in Apps, they are often common in complex systems software like databases, and they are fairly rare in firmware/drivers/system code code.
There are a few patterns for dealing with cyclic references, but they all end up requiring either unsafe or a main "owner" object which you clean up occassionally (effectively arena allocation). Having now worked in C/CPP - the idea of having unsafe blocks sprinkled around the code doesn't bother me, and many C/CPP components have some form of arena allocation built-in. I just wish Rust learning resources would be more upfront about this.
> Relational models are everywhere in Apps, they are often common in complex systems software like databases, and they are fairly rare in firmware/drivers/system code code.
It's not like that you can't write relational models in the safe Rust. The only forbidden thing is a reference pointing arbitrary memory, which is typically worked around via indices and often more performant in that way. It is much rarer to find applications that need an absolutely random pointer that can't be hidden in abstractions in my opinion.
> I just wish Rust learning resources would be more upfront about this.
While beginner resources don't dwell too much upon cyclic references, they don't consider unsafe blocks as unusual. All the material I've seen say that there are certain domains where Rust's compile-time safety model simply won't work. What Rust allows you to do instead, is to limit the scope of unsafe blocks. However, the beginner material often won't give you too much details on how to analyze and decide on these compromises.
Anyway, compile-time safety checks (using borrow checker) and manual safety checks (using unsafe) aren't the only way to deal with safety. Cyclic references can be dealt with runtime safety checks too - like Rc and Weak.
> Cyclic references can be dealt with runtime safety checks too - like Rc and Weak.
Indeed. Starting out with code sprinkled with Rc, Weak, RefCell, etc is perfectly fine and performance will probably not be worse than in any other safe languages. And if you do this, Rust is pretty close to those languages in ease of use for what are otherwise complex topics in Rust.
It is easier to tell it, "don't do it this time" than all the time.
It is no accident that while Val/Hylo, Chapel and Swift have taken inspiration from Rust, they have decided to not inflict the affine types directly into the language users, rather let the compiler do part of the work itself.
Agreed that I wish more beginner Rust books had a section about this. The pattern is quite simple, but it's hard for beginners who get stuck to realize that they need it.
Maybe for you it's unusual - in my previous work all the apps contained graphs, and I just joined a company where almost all the apps also contain graphs
I don't write Rust, but I never understood why graphs meant you need circular references.
Doesn't it just come down to the question of who owns the node?
If it's a tree, and parents are never removed before children, just make the child owned by the parent and keep a weak reference to the parent.
If it's a general graph, and vertices can exist or not exist regardless of edges, keep a list of them independent of the edges, and keep weak references in the edges.
If it's a graph where a one or a few roots exist, and nodes exist as long as there's a path from a root node to them, that sounds like a classic use case for Rc<>.
What's a frequently encountered case for such cyclic loops? Without details I'm drawn to trying to break the cycle, either by promoting the shared state to a container object for the set, or by breaking it out into it's own object that multiple things can point at.
I think a game is a good example, or anything that's kind of like a game in that it's modeling a world that's changing over time. Objects come and go, and they "target" each other or "require" each other or whatever sort of relationships the program wants to express. Those relationships end up forming a graph that might contain cycles.
Your parent said "frequently encountered" and while it's probably true that doubly linked lists may be "frequently encountered" in some people's code they're usually a bad idea and "don't use a list here" is often the right fix, not "conjure a way to make that safe in Rust".
It's very noticeable how often people who "need" a linked list actually only wanted a queue (thus Rust's VecDeque) or even a growable array (ie Vec).
Aria has a long list of excuses people offer for their linked lists, as well as her discussion of the time before Rust 1.0 when she sent lots of Rust's standard library collections to that farm up-state but wasn't able to send LinkedList.
Doubly-linked list is something you have almost no reason to ever write.
Parent field is something where you have a clear hierarchy (it's not really “cyclic”, so it's the perfect use-case for weak references).
When coming from a managed-memory language, this obviously requires some conceptual effort to understand why this is a problem at all and how to deal with it, but when compared to C or C++, the situation is much better in Rust.
Also, a parent field is something you should be able to infer, e.g, keep a stack of parents as you traverse down a search tree following the child pointers, instead of storing parent pointers in the tree nodes.
That's assuming you traverse the tree down from the root each time. Often you do, but there are cases where you don't -- e.g., if your goal is to determine the lowest common ancestor of two given nodes.
Technically true, but sometimes you want parent pointers. You then have a more general graph in the underlying representation, but it still represents a tree structure.
Same shows up in Postgres Query* struct for SQL. Copying memory between parser, planner, execution would be too expensive in large queries - so instead you have an arena allocated representation.
An Abstract Syntax Tree / or Double-Linked List both qualify, but they're also a lower level implementation detail than I'd expect to frequently interact with in a reference safety focused language.
I've still been meaning to write something in / learn Rust's ways of thinking; is there not an intended replacement for these data structures? Or do they expect it all to go under Unsafe?
> is there not an intended replacement for these data structures? Or do they expect it all to go under Unsafe?
For linked-lists, there's one in std and the majority of people should never have to write their own as it's error prone and requires unsafe.
For graph use-case then you can either use ECS, arena, ref counting or unsafe, but you're probably better off using/developing a dedicated crate that optimizes it and abstract it away behind an easy to use (and safe) interface.
The one in std uses unsafe. My main concern with learning rust is that you can spend ages trying to learn “the right way” of doing things in rust, when the right way really is to use unsafe.
No, the right way is to use unsafe primitives that have been tested, audited or even formally proven (like the ones in std).
Sometimes such a primitive doesn't exist and you should use unsafe yourself, but then you're the one supposed to make sure that your code is in fact sound. If you keep unsafe for small portions of the code you can reason about and extensively test so Miri gives you a good level of confidence, then it's fine to use unsafe. But it's the more expensive option, not the default.
Surprisingly, I am faster in Rust than any other language. Something about my prior experiences just made it click just the right way.
I don't want to program in anything else anymore. I don't want to deal with obscure C++ error messages, C footguns and lack of ergonomics, I don't want to deal with abstraction hell of Java, or the poor person's typing that python has.
I have been programming in Python for the past 6 years, I know all sorts of obscure details, and with rust, I just don't need to think about all of those issues.
> Surprisingly, I am faster in Rust than any other language.
Not really surprising, given that you have C and C++ background. That's what I was trying to highlight. Rust isn't a confusing or unproductive language as many project it to be - if you have the conceptual understanding of what happens on the hardware. Especially about stack frames and RAII. If you know those, the borrow checker complaints will immediately make sense and you will know how to resolve them.
Add rust-analyzer (Rust's language server) to it, you get real-time type annotations and a way to match types correctly in the first attempt. In my experience Rust also helps structure your program correctly and saves a ton of time in debugging. All in all, Rust is a fast way to write correct programs.
> Rust isn't a confusing or unproductive language as many project it to be - if you have the conceptual understanding of what happens on the hardware. Especially about stack frames and RAII. If you know those, the borrow checker complaints will immediately make sense and you will know how to resolve them.
I have reasonable understanding of "what happens on the hardware" (been writing kernel code for years), know modern C++ (with RAII and stuff) and Rust is confusing and unproductive language for me.
I had university courses on computer architecture and assembly, even before I took up Python as a hobby. I did have a little C experience before that. My entire perspective on Rust type system from day 1 (back in 2013, before Rust 1.0) was based on the hardware (primarily stack frames) and problems I had with assembly and C. There was never a point where the borrow checker didn't make sense. This is why I insist that Rust isn't hard to understand if you learn the hardware on which it runs.
Back then, people were debating the design decisions that led to the borrow checker, in public for everyone to see (on Reddit and IRC). They were trying to avoid memory safety issues in Firefox and Servo. They were even a bit surprised to discover that the borrow checker solved many concurrency bugs as well.
I took a different route to Goku (other commenter), I used to write a lot of C and C++ in university, did everything there, up until 2018-ish, I got a bit into rust and things just clicked, my understanding of memory was just not that good enough, and then my C skills skyrocketed as a consequence of learning proper memory management.
Then I got into Haskell, and functional programming, that made thinking about traits, immutability, and all functional aspects a breeze.
Then finally, I got into rust again, use it at work and personal projects. Somehow I managed to rewrite a project that took me 4 months in Python in about 4 days. It was faster, more robust, cleaner, and I could sleep at night.
I'd add that if you have some understanding of how memory ownership should be such that you don't end up with memory leaks, you are fine. The borrow checker just verifies that your mental model is correct, and removes some of the cognitive load from you.
> At this point, I'm about as fast in Rust as I am in Python.
This is factually impossible.
For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
This is not even considering more local complexities, like data structures with cyclical references.
How do you outright deny something as subjective as my personal experience? Besides, I'm not the only one in this discussion that made the same opinion.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
While GC'ed languages allow you to skip a proper initial design stage, it's a stretch to claim that it's not required at all. In my experience using Python, while the initial stages are smooth, such design oversights come back and bite at a later stage - leading to a lot of debugging and refactoring. This is one aspect where Rust saves you time.
> This is not even considering more local complexities, like data structures with cyclical references.
I'm not going to dwell on cyclical references, since there's another thread that addresses it. They point out a way to make it as easy in Rust as it is in GC'ed languages.
Meanwhile, the upfront architecture and data structure design isn't as complicated as you project it. Rust is mostly transparent about those - even compared Python. How well do you understand how Python manages Lists, dictionaries or even objects in general? I often find myself thinking about it a lot when programming in Python. While you need to think upfront about these in Rust, there's actually less cognitive overhead as to what is happening behind the scenes.
That is also not shared by everyone. If you have written enough Rust to have internalized designing for the borrow checker, you don't have to spend much time in a design phase.
The only time I find I have to "fight the compiler" is when I write concurrent code, and you can sidestep a lot of issues by starting with immutable data and message passing through channels as a primitive. It's a style you have to get used to, but once you build up a mental library of patterns you can reasonably be as fast in Rust as you are in Python.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
It's nearly the opposite. For larger programs in Python, you need an upfront design stage because the lack of static typing will allow you to organically accrete classes whose job overlap but interfaces differ.
Meanwhile, Rust will smack you over the head until your interfaces (traits) are well-organized, before the program grows enough for this to become a problem (or until you give up and start over).
How do I know? I'm stuck with some larger Python programs that became a mess of similar-but-not-interchangeable classes. RiiR, if I ever have the time.
> For larger programs in Python, you need an upfront design stage because the lack of static typing will allow you to organically accrete classes whose job overlap but interfaces differ.
You can also install pre-commit and mypy, and have static typing.
That's the entire point we're making. Rust's type system forces you to deal with the problem early on and saves time towards the end. It's not like that's impossible with Python with addons like mypy. But Rust's type system goes beyond just data types - lifetimes are also a part of the type system. I don't know how you can tack that on to Python.
> Rust's type system forces you to deal with the problem early on and saves time towards the end. It's not like that's impossible with Python with addons like mypy.
Definitely not - mypy's pretty good these days, and lots of people use it.
> But Rust's type system goes beyond just data types - lifetimes are also a part of the type system. I don't know how you can tack that on to Python.
Well, Python's objects are generally garbage collected rather than explicitly destroyed, so I don't think it'd make sense to have lifetimes? They don't seem a correctness thing in the same way that types do.
Lifetimes and borrowing are very much a correctness thing and aren't just for tracking when memory is freed. While you won't have use-after-free issues in a GCed language, you will still have all the other problems of concurrent modification (data races) that they prevent. This is true even in single-threaded code, with problems like iterator invalidation.
Mutability is a big one for correctness. In Python, any function you pass any (non-primitive) object to might be mutated right out from under you and you have no idea it's happening. In Rust, you have to explicitly provide mutable references, or you need to hand over ownership, in which case you don't care if the callee mutates its argument, because you no longer have access to it.
Factually it's not. It may be true that in a very very idealized thought-experiment, when someone has a perfect knowledge, never makes mistakes, doesn't have preferences, can type arbitrary fast etc, python needs fewer keystrokes, fewer keywords or such, thus is faster. Who knows. But in reality none of the assumptions above hold, also literally everything plays a much bigger role in development speed anyway.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
Every language requires this (if you want robust code), most just let you skip it upfront ... but you pay dearly for doing so later.
I disagree. I have similar observation. With modern editors and Language Server that are giving immediate feedback writing strongly typed languages doesn't differ than writing python.
If you are only writing small programs where perf doesn't matter, crashing doesn't matter, design doesn't matter - Python will be faster than Rust, because the only thing that matters is how you can write the code from 0 to "done". You can jump right in, the design stage is superfluous & unnecessary. There is nothing to optimize, the default is good enough.
If you are doing slightly more than that, Python and Rust become about even, and the more you need those things, the better Rust becomes.
I have to second the OP: ownership isn’t that hard. I just get used to structuring a program in certain ways. Having written a lot of C++ helps because the things Rust won’t let you do are often unsafe or a source of leaks and bugs in C++.
Having an editor with rust-analyzer running is massively helpful too since ownership issues get highlighted very quickly. I can’t imagine dealing with any language larger than C without a smart editor. It can be done but why?
I still find async annoying though.
My biggest source of friction with Rust (other than async) is figuring out how to write code that is both very high performance and modular. I end up using a lot of generics and it gets verbose.
I think this is a very valuable comment, and the replies don't do it justice.
I strongly agree from my own and my peers experience with the sentiment that latency from zero to running code is just higher in Rust than Python or Go. Obviously there are smart people around and they can compensate a lot with experience.
Honestly I found myself coding very much the same way in Rust as I did in Python and Go, which were my go-to hobby languages before. But instead of "this lock guards these fields" comments, the type system handles it. Ownership as a concept is something you need to follow in any language, otherwise you get problems like iterator invalidation, so it really shouldn't require an up-front architectural planning phase. Even for cyclic graphs, the biggest choice is whether you allow yourself to use a bit of unsafe for ergonomics or not.
Having a robust type system actually makes refactors a lot easier, so I have less up-front planning with Rust. My personal projects tend to creep up in scope over time, especially since I'm almost always doing something new in a domain I've not worked in. Whenever I've decided to change or redo a core design decision in Python or Go, it has always been a massive pain and usually "ends" with me finding edge cases in runtime crashes days or weeks later. When I've changed my mind in Rust it has, generally, ended once I get it building, and a few times I've had simple crashes from not dropping RefCell Refs.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
It seems that is not factually impossible.
Now, answering your question: It could be useful to use boxed types and later optimize it, so you get the benefits of rust (memory safety, zero cost abstractions) later, without getting the problems upfront when prototyping.
I've finally set my mind to properly learning a new language after Python, Haskell, and typescript. I'm looking into Rust especially because of how I've heard it interoperates with Python (and also because it's maybe being used in the Linux kernel? Is that correct?).
Rust is an excellent follow up to those languages. It's got many influences from Haskell, but is designed to solve for a very different task that's not yet in your repertoire so you'll learn a ton.
Not sure what you mean by "userland" drivers here, but support for kernel modules written in rust is actively being developed. It's already being used for kernel drivers like the Asahi Linux GPU driver for M1 Macs.
But you can write userspace drivers in any language, as long as that language has basic file I/O and mmap() support. There's nothing special about using Rust for userspace drivers.
I agree. I feel far more productive in C and C++ than in Rust at that point.
Rust feels like totally missing the sweet spot for me. It's way too pedantic about low level stuff for writing higher level applications, but way too complicated for embedded or writing an OS. In the former case I would rather take a C++, Java, Haskell, OCaml or even Go, and maybe sprinkle some C, and in the latter case C in macroassembly mode is far more suitable.
I still have a feeling that original vision of Graydon Hoare (i.e. OCaml/SML with linear types, GC, stack allocations, green threads and CPS) would be a much better language.
The problem with C and to C++ is that it’s 2023 and the CVE list is still loaded with basic memory errors. These come from everywhere too: small companies and open source all the way up to Apple, Microsoft, and Google.
We as a profession have proven that we can’t write unsafe code at scale and avoid these problems. You might be able to in hand whittled code you write but what happens when other people work on it, it gets refactored, someone pulls in a merge without looking too closely, etc., or even maybe you come back two years later to fix something and have forgotten the details.
Having the compiler detect almost all memory errors is necessary. Either that or the language has to avoid this entirely. Rust is the former class unless you use unsafe, and the fact that it’s called “unsafe” makes it trivial to search for. You can automatically flag commits with unsafe in them for extra review or even prohibit it.
I think nobody is arguing the need for static memory safety, just that the poor Rust ergonomics aren't a good tradeoff, especially for scenarios where C is useful. We need many more Rust alternatives that explore into different directions, Rust is already too big and "established" for any radical changes in direction.
On that regard, by packaging Modula-2 into a C friendly syntax, I do agree Zig is relatively interesting option, however not having a story for binary library distribution is an hindrance in many C dominated industries.
IMHO Rust's ergonomics problems aren't caused by the borrow checker itself, but have the same cause as similar problems in C++ (mainly a "design by committee" approach to language design and implementing features that should be language syntax sugar in the stdlib instead, which then directly results in the stdlib being too entangled with the language and "too noisy" hard to read code).
Apart from the static memory safety USP, Rust is repeating too many problems of C++ for my taste, and at a much faster pace.
I agree with this. The borrow checker itself isn't the problem. That's necessary to make you write correct and safe code anyway.
The problem is that there is too much syntax, too many symbols, and too many keywords. I just keep forgetting how to use impl and lifetimes and single quotes and whatnot. It makes it really tough to use as an occasional language. And if I can't do that, then how can I get confident enough to use it in my job?
> The problem is that there is too much syntax, too many symbols, and too many keywords. I just keep forgetting how to use impl and lifetimes and single quotes and whatnot.
This is exactly how I feel about Rust.
There are some good ideas in there, hiding behind a horrible language design. I am waiting for someone to provide a more developer friendly alternative with the simplicity of C or Go.
It uses compile-time reference counting / lifetime analysis / borrow checking. Which is mostly inlined to the point that there is none of the sort in the compiled output and objects can even live on the stack. It basically looks like Python but is nothing like it underneath, and of course with no GIL. You can run it on JIT or compile it to C++.
There's also Koka with the Perceus algorithm on compile time and looks like a much cleaner language than Rust. It also tracks side effects of every function in a type where pure and effectful computations are distinguished.
In that regard, strangely enough, I find with constexpr code, templates, and concepts, is easier to achieve most compile time code stuff, while staying in C++, than dealing with Rust macros.
It is, however those same companies aren't dialing full safe ahead knob either, hence why Microsoft just recently published a set of secure coding guidelines for C and C++.
These are two issues, which a theoretically orthogonal but in practice not so much. These are known as soundness and completeness. A good talk on topic [1]
Rust will reject a lot of sound programs, and that's a huge performance hit. You are hitting the incompleteness wall with multiple mutable borrowing, closures in structures are a huge source of pain as well. And usually the answer from the community is "use handlers instead of pointers", but this gives you, surprise surprise, manual resource management alike that in C, and the compiler won't help much.
Of course this is all subjective but for me the ergonomics of Rust is far too bad. It's a good step in right direction along with ATS, but I really hope we could do better than this.
I maintain very large C and C++ application and very rarely have any memory issues. Tools like Valgrind and Helgrind are excellent for finding and fixing problems. So switching to Rust is a very bad ROI.
I gave Rust a few chances, and always came out hating its complexity. I needed a systems programming language to develop a hobby OS[1], and Nim hit the sweet spot of being very ergonomic, optional GC, and great interop with C. I can drop down to assembly any time I want, or write a piece of C code to do something exotic, but the rest of the system is pure Nim. It's also quite fast.
Opposite experience for me. Writing Rust on embedded systems greatly improved my confidence and speed. When using C a small mistake often leads to undefined behaviour and headaches. Rust theres none of that - its been a game changer for me.
When you are developing hardware on an FPGA, a lot of hardware bugs look like they have locked up the CPU and strangely enough, a lot of undefined behavior looks exactly like a hardware lockup...
I am curious what kind of code you are writing? Is it very low level or very high?
>I know rust gives memory safety and how important that is, but the ergonomic is really bad. Every time I write some rust I feel limited. I always have to search libraries and how to do things. I cannot just "type the code".
You don't have to search libraries and figure out how to do things in Zig?
It's hard to describe, but in some languages, you spend a lot less time looking at reference docs and more time just naturally writing the solution. Lisp is a great example of that, if you get through the learning curve.
I suspect Zig libraries feel easier because they're doing easier things. I bet if you try to draw a triangle with the Vulkan API in Zig, you'll find yourself looking at the reference docs a lot.
Most of the time I can use my "general computer science baggage" to write the solution. At present, I do embedded and web business logic (wasm) where the UI is rendered by preact. For those two projects zig is working very well.
I agree with this general feeling, and it is hard to articulate
Rust forces you to figure out ahead of time where each bit or byte is going to go and on which thread and using which mutation scheme. I’m happy to play the game, but it feels tedious for anything short of a parser or a microcontroller.
It messes with my process because I like to get something working before I determine the best API structure for it
I can get 90% of the performance with Swift and it flows much more easily, even though Rust’s type system is more powerful.
Those two sentences feel in contradiction of one another. You don’t need worry to about where the bits go, you just need to know to call a method to move the bits?
Swift makes every type implicitly copyable on the stack, including object pointers (through ARC), so you don’t have to clone. You can even pass functions as variables without any hoops.
I love lots of things about Rust, though, and will continue to use it for lots of things. Cross-platform things, for one!
I rather use compiled managed languages like Swift, D and C# instead, they provide enough low level coding knobs for C and C++ style coding, while being high level productive.
Would add Go to the list, but only when I really have to.
Nim and Crystal could be alternatives, but don't seem to have big enough communities, at least for what I do.
However I do agree with the conclusion, Rust is a great language for scenarios where no form of automatic memory management is allowed, kernels, specific kinds of drivers, GPGPU programming, as general purpose, there are more productive alternatives, equally safe.
I used to be .NET dev and don't agree. Couple of reaons:
1) Modern Java is almost as good as C# with some things I can't give up in Java (static imports => succint code, Groovy Spock => succint tests)
2) Kotlin is better than C#
3) JVM has much much bigger ecosystem (almost all the apache projects are JVM oriented) and default web framework is much less code to type (SpringBoot) is much more productiv
4) JVM has wider variety of langs
For those reasons IMHO if you are small-mid company (or startup) it's wiser to choose JVM.
Kind of, maybe you need to do some low level coding and don't want to wait for Valhala, or make use of JNI and native libraries, GraalVM/OpenJ9 still aren't as integrated as .NET Native or Native AOT, e.g. writing native shared libraries.
Also Java lost the attention span of the gaming industry, besides Android casual games and Minecraft, there are hardly anyone else paying attention to it.
I didn't say no decent algorithm in C#, but for each performance sensitive algorithm/data structure there is a C and Java implementation at the least ( in my case Roaring Bitmaps).
In C# the solution is half baked or archived or abuses allocation.
I think Unity has way more with C# adoption in game dev than innate C# qualities.
This is a classic case of goalpost moving. The reason why so many algorithms are written in Java especially closer to academic side is because most curriculums in comp-sci often straight up not allow using anything else except Java, Python or sometimes C++. Having C# as an alternative in these is a luxury. There are also more people using Java in general. However, this does not make it a better language at solving these tasks, nor it is any suitable for writing high performance implementations for advanced vectorized algorithms which would push hardware, which is actually what you want when you start caring about such scenarios, which C# excels at.
I'm not moving the goalpost. I explained my examples in another reply. Want to write an engine mostly from scratch in C# and you need libraries that are low on allocation and for niche data/algorithms that games need? You're going to have a bad time(TM).
Sure you could use YAML parser, but it allocates everyone and their mother.
Can you find a Fluent localization in C#? Sure, but its outdated and archived.
Ok, but basic RoaringBitmap implementation? The repo is archived and not fully complete.
Why C# is used in game dev is incidental. It has more to do with Unity and XNA/FNA than any concrete quality of language modulo value types (but then again, most C# libraries don't focus on avoiding allocation and are just as happy as Java to construct a complicated hierarchy of classes).
I think Java is only good for long-running servers.
Java doesn’t support C interop. For many desktop and embedded projects this is a showstopper, here’s an example https://github.com/Const-me/Vrmac/tree/master/VrmacVideo That C# code directly consumes V4L2 and ASIO Linux kernel APIs, and calls unmanaged user-mode DLLs like libfdk-aac.so and liba52-0.7.4.so.
Native stack and value types in C# reduce load on GC, and the number of complicated tricks required from JIT compiler. This in turn helps with startup performance. This is critical for command-line apps, and very desirable for desktop apps.
Another thing missing in Java is intrinsics support, both scalar like popcnt, bitscan, BMI, etc., and SIMD like SSE and AVX.
Projects Panama & Valhalla seems to solve all your complaints:
> Java doesn’t support C interop. For many desktop and embedded projects this is a showstopper, here’s an example https://github.com/Const-me/Vrmac/tree/master/VrmacVideo That C# code directly consumes V4L2 and ASIO Linux kernel APIs, and calls unmanaged user-mode DLLs like libfdk-aac.so and liba52-0.7.4.so.
Part of Panama: check out the "Foreign Function & Memory API" [0]. The official docs [1] say it is a preview in 21 but it got stabilized in Java 22 (isn't out yet).
> Another thing missing in Java is intrinsics support, both scalar like popcnt, bitscan, BMI, etc., and SIMD like SSE and AVX.
Also part of Panama: see the "Vector API" JEP [2].
> Native stack and value types in C# reduce load on GC, and the number of complicated tricks required from JIT compiler. This in turn helps with startup performance. This is critical for command-line apps, and very desirable for desktop apps.
This is part of Project Valhalla [3], they're adding value types and actual generics, among other things.
That said, most of these are not done / not in a stable LTS Java release yet. We'll see how much better it'll be compared to C# (if at all) once they land.
Most real-live C APIs are using function pointers and/or complicated data structures. Here’s couple real-life examples defined by Linux kernel developers who made V4L2 API: [0], [1] The first of them contains a union in C version, i.e. different structures are at the same memory addresses. Note C# delivers the level of usability similar to C or C++: we simply define structures, and access these fields. Not sure this is gonna be easy in Java even after all these proposals arrive.
For a managed runtime, unmanaged interop is a huge feature which affects all levels of the stack: type system in the language for value types, GC to be able to temporarily pin objects passed to native code (making copies is prohibitively slow for use cases like video processing), code generator to convert managed delegates to C function pointers and vice versa, error handling to automatically convert between exceptions and integer status codes at the API boundary, and more. Gonna be very hard to add into the existing language like Java.
> "Vector API" JEP
That API is not good. They don’t expose hardware instructions, instead they have invented some platform-agnostic API and implemented graceful degradation.
This means the applicability is likely to be limited to pure vertical operations processing FP32 or FP64 numbers. The rest of the SIMD instructions are too different between architectures. A simple example in C++ is [2], see [3] for the context. That example is trivial to port to modern C#, but impossible to port to Java even after the proposed changes. The key part of the implementation is psadbw instruction, which is very specific to SSE2/AVX2 and these vector APIs don’t have an equivalent. Apart from reduction, other problematic operations are shuffles, saturating integer math, and some memory access patterns (gathers in AVX2, transposed loads/stores on NEON).
> most of these are not done / not in a stable LTS Java release yet
BTW, SIMD intrinsics arrived to C# in 2019 (.NET Core 3.0 released in 2019), and unmanaged interop support is available since the very first 1.0 version.
Well maybe you should use C++ or Rust instead of Java or C# in that case?
My point is if you are doing business (especially web) apps. Use one of JVM langs insted of C# because ecosystem is much bigger (and it has fresher langs as well like Kotlin - if that's what you care about)
> use C++ or Rust instead of Java or C# in that case?
Despite having to spend extra time translating C API headers into C#, the productivity gains of the higher-level memory safe language were enormous.
Another example, I have shipped commercial embedded software running on ARM Linux, and based on .NET Core runtime. The major parts of the implementation were written in idiomatic memory-safe C#.
> doing business (especially web) apps
Well, these business web apps are precisely the long-running servers I have mentioned. Still, the software ecosystem is not limited to that class of problems, and due to different tradeoffs Java is not great for anything else.
Java is surely keeping up but I can't name single Java feature that I miss in C# or is implemented better in Java. I haven't used Java in a long time though, just occasionally I read about new Java features and I've never said to myself "cool, I wish I had it in C#".
Static import are also available in C# for quite some time now (c# 6, released in 2015, and in C# 10 you can even make this import global for for project).
I haven't used Kotlin, is there any killer feature compared to C#? (except more succinct code in certain cases?)
Depending on how you look at it, better extension everything support on Kotlin's case, and a way to do DU, while it keeps being discussed for C#, people should just add F# to their codebase, but alas.
I've always found those sort of firms with C#, in my experience, have the best architected code. Proper domain-driven design, onion architectures, clean testable code... Some have legacy issues where they might not have the most cutting edge CI/CD pipeline or high automated test coverage, but the code itself can be very nice. I've never really experienced that level of consistency with a different language/company size.
Usually that means you aren't using trimming, .NET speak for dead code removal during linking.
Also remember that standard .NET runtime does a little bit more than Go's runtime, so it might happen that even with trimming, for basic applications Go ends up having an upper hand on file size.
On the other hand, I have had Go static binaries grow up to 200 MB and being require to use UPX to make it manageable, e.g. trivy.
Since I edited my comment, see my additional remarks regarding runtime capabilities, and the counterpoint of big Go binaries.
Also note that triming only works properly if the libraries have taken the effort to be trimmable, as the linker errs on the safe side and won't trim unless certain that it is really dead code, and not called via reflection.
I was making Windows 98 apps with Delphi 4, and they were 350 KB large
And I was upset that they were so big. Sometimes I used UPX. Or I kicked out all Delphi GUI libraries, and created the GUI with the Win32 API calls directly. I got 50 KB Hello Worlds.
I do not have Windows 98 anymore. But I still have Delphi 4 installed under Wine, so I just tried it out.
Just showing a messagebox from windows gives 16k
Using the sysutils unit though, puts it at over 40k. And with classes, it becomes 57k.
Not sure what they pull in. sysutils contains number/datetime parsing and formatting, and exception handling. classes has basic containers and object oriented file handling.
Let me give a real world example from my own experience.
I have built a Win32 desktop app with its core logic in Go; and then re-built from scratch using .NET (v7). The core logic involved a fairly complicated keyboard input processing based on bunch of config files.
This is an unfair comparison of apples to oranges by building the binary with the wrong flags. .NET produces smaller binaries than Go with NativeAOT (despite including more features).
- There is no point in chasing smallest possible binary size if it trades off performance and features one wants to use in production scenario, or comes with other tradeoffs that sacrifice developer productivity. I don't see anyone complaining about the size of GraalVM native images. As long as binaries are reasonably sized, it's not an issue.
- dotnet publish -c release -p:PublishAot=true definitely produces smaller binaries than Go as of .NET 8 (and no, you cannot use the argument that it's not released yet - it's in RC.2 which is intended for evaluation for adopting .NET 8 scheduled for release next month)
Agreed. I feel C# is appropriately rated on HN and other programming forums. It has perfomant memory options that other GC languages lack, and great builtin packages to use. Overall, it is a good language.
My biggest issue with C# though is how badly exceptions are handled given that it is a statically typed langauge. I wish functions explicitly defined the exceptions it can throw since a minor package bump could add an exception without your compiler warning you that it isn't handled. I much prefer Rust, Go and Zig's error handling to C#'s since those kind of issues don't happen.
> It has perfomant memory options that other GC languages lack, and great builtin packages to use.
As clarification for the audience, it isn't the only GC enabled language with C and C++ like capabilities, in fact there are several examples since the early 1980's.
The adoption push for Java and scripting languages distorced the understanding of what was already available out there.
Well, I see it as lack of fanboyism which is interesting and almost unique to the Java/C# ecosystem. A lot C# experts(and I mean REAL, low level experts) seem to also have very high Java expertise..
And those that have Java expertise but not C# seem to demur to those that do; image!
But it's still niche(around here and in the startup world) and gets lumped in with Java and together they are not "hip" or "agile" or whatever.
I've been out of the windows development game for a long time, so I haven't used C# since it strictly required a VM... what's pre-compiled C# development like nowadays? Are there major caveats? If you can emit plain old binaries in C# with no runtime dependencies, that would make it a truly compelling language IMO.
And as another question, what's the cross-platform (mainly Linux) support like in AOT-compiled C#? If it's just as good as in Windows and emits plain executables, I would probably consider it the best place to start for any new project. (Something tells me it's not...)
C# supports AOT since forever, NGEN was present in .NET 1.0. Not many people used it, because it requires signing binaries and only supports dynamic linking, with a performance profile towards fast startup.
On Microsoft side the Singularity and Midori experiments used AOT.
They influenced the AOT toolchains for Windows 8 store apps with MDIL (Singularity/Bartok), and Windows 10 store apps with .NET Native (Midori/Project N).
Now there is Native AOT, which supports CLI and native libraries,.NET 8 extends that to EF and ASP.NET frameworks. For GUI applications, maybe only fully on .NET 9.
Mono AOT has had support for ages, being used on iOS, Android, and Blazor.
Finally there is IL2CPP and Burst compiler from Unity.
In 8, NativeAOT also supports iOS (and even Android reportedly?) for, I assume, MAUI target to do away with Mono. Documentation on this definitely needs work, and there are projects that made it working with WPF, Windows Forms and Avalonia back in .NET 7. Arguably, none of those were particularly user-friendly but generated COM interop project for 8 was done specifically to improve this on Windows as well.
This. I have exactly the same experience, I can't believe how much I was able to ship with Zig and the code mostly feels like "done".
You can always improve it, but there's no need to. With Rust, I was never happy, even after 5 years, I was still thinking about better abstractions and implementing more traits, making it more generic, etc.
No, usually you don't. Rust has closures, iterators, generics, different traits for operator overloading, smart pointers, etc.
Zig doesn't have any of that. It's very interesting combination of low-level, predictable code, with meta-programming, where you get some of that abstraction back.
i.e. Zig does not have generics, but your function can return type, so generic list is just a function which returns a newly created struct.
Functions in Zig can be called both in runtime and in compile-time. You can force some expression to be called during comptime using a keyword, and sometimes the comptime is implied (like when you define top-level const)
If a function is called in comptime, it can also return types. So for example:
// this is a function which accepts a type
// if you accept type, you also have to restrict the arg to be comptime
// if the arg is comptime it still does not mean that the function cannot be called in runtime,
// but in this case, it returns type, so it is comptime-only function
// there are also cases where this is not true, like std.mem.eql(u8, a, b) which accepts type,
// but can be called in runtime because it does not return type
fn Wrapper(comptime T: type) type {
return struct { value: T };
}
const F32Wrapper = Wrapper(f32);
// @TypeOf(x.value) == f32
var x: F32Wrapper = .{ value: 1.0 }
Zig has `comptime` where you effectively generate Zig code at compile time by writing Zig code, no special generics syntax/semantics needed. It is very nice and powerful concept that covers lots of ground that in Rust would belong to the land of procedural macros.
Can't speak for why Zig doesn't have a problem but Rust is cursed by its success: it lowers the barrier for improvement enough to entice you to always improve it.
No. Rust forces you to spend endless hours doing mental gymnastics which shouldn't be needed in the first place (linked data-structures, owned arenas, or even just boxed slices are impossible in safe rust).
And you just keep refactoring/improving/desperately trying different ideas, because it never feels right.
It's ok if you don't agree but pls don't try to make my responses look like I like Rust, I don't and I'd happily get those years back if it was possible.
Yep. I can't remember method names for the life of me, which is why my best experiences have been with Go and Java: The IDE (always Jetbrains) knows, via the type system, what methods I can call.
Furthermore you can use `cargo doc` to generate a documentation website that had everything you can do or you can use docs.rs for this. Whoever wrote this didn't embrace the tooling and just gave up.
Wait, I am a bit confused. Does Zig have more/better libraries than Rust? I thought it's a pretty new language. The most limiting thing for me with Rust was the lack of libraries (vs. say Python or Node/JavaScript).
It doesn't. The ecosystem is very immature and even the official tooling is very unstable. It has a bunch of interesting design ideas but at this point it's more of an experimental language than a production ready one by most metrics. (And unless it finds some kind of corporate backing, this is unlikely to ever change).
Truly seamless because the zig compiler is also a C compiler, so the type information and calling convention works across languages at a level above any other I've encountered.
It's also an unfinished language. I agree Zig is promising, but it's not confidence inspiring when the creator is still making videos debugging the compiler.
True, but I think that the person you're responding argued that rust/C integration is also seamless. (In the general discussion I'd say they're right as C to Rust integration isn't much of an problem and you can use C libraries relatively easily in Rust as well, but at the same time when talking about Zig I don't think it's fair to put it on the same ground).
Given that Zig is memory unsafe it isn't either a good general purpose language.
IMO a good GPR is memory safe (no C, C++, Zig), is easy to use(no Rust), has strong static typing (no Perl, Python, Ruby) and is "stable" (no Scala). Lots of choices remain: Java, Kotlin, Ada, D, OCaml..
It somehow seems that Zig has most of the qualities that people like about C: clear, crisp, no huge Stdlib, good old straightforward imperative semantics, and readonably fast. But without lots of the cruft.
Unfortunely lacks support for binary libraries, and not yet a story for UAF other than the tooling we are already using in C derived languages to track them down.
You should try diving into num for like a month and see how you like it. It's different enough that you need to go past a certain kind of ledge to start liking it. Or at least that was my experience.
For me, it shares the most important benefits of Rust but with quite a lot more ergonomic coding model.
Nim on paper is great; it has many advantages over Rust in the general purpose "niche". Tragically, it's kind of stillborn. It's older than Rust, has orders of magnitude less mindshare, and has no companies with serious technical reputations backing it.
Yeah, you're not wrong about the mindshare problem. But it somehow at least in my mind differs from other "stillborn" older languages in that it keeps improving. The end result is that it still feels modern in the year 2023.
Since you have previously said that you are using Zig to do embedded programming for medical devices, I assume that it is your main pain point. I largely agree that the current Rust embedded story is not exactly for existing embedded programmers (and I guess you are one of them). Rather it looks more like a solution for existing application programmers. I don't think it's an intrinsic limitation of Rust the programming language, rather a specific community at this point happens to prefer this way. Still it is a weakness in some sense and Zig will be a good alternative.
Yes, I think rust is very good for higher level programmers wanting to code embedded like a regular OS. There are many great projects around using rust on embedded.
But me, I prefer to manipulate registers directly. Especially with "exotic" MCU where you have to execute write/read in specific number of CPU cycles. Rust makes that very hard.
By "wrote" you are meaning just coding or coding+debugging? Because other languages are easier to code, but hard to make error free, while Rust is hard to write but much easier to make bug free.
I use Rust a lot, and have been really keen on getting into Zig.
Not sure if much has changed (it was a while back), but my biggest problem was with finding and using 3rd party libraries (mostly for the boring stuff like DB connectivity, JSON/YAML parsing, logging, etc.).
E.g. even now if I search for "zig mysql library", the tops hits are all about people discussing it on reddit, than any actual library.
Give Copilot a try, it completely shift coding experience in Rust.
Especially this:
> I always have to search libraries and how to do things.
Once you pass the initial curve with crutch like Copilot, then you can be almost as productive (if not more, considering refactoring and testing) with your native first coding language
Perhaps my biggest critique is that crates.io has no namespacing. Anyone can just claim a global and generic package name and we mostly have to deal with it (unless you avoid using the crates.io repository, but then you'll probably have more problems...). Some of these globally-claimed generic packages are not really the best package to use.
Maybe it was a reaction against the Java-style reverse DNS notation, which is verbose and annoying, but a more GitHub-style user/group namespace prefixing package names would have been the nice middle ground.
I did some analysis on crates.io to find the top name squatters. Then I did some calculations and found that the top name squatter created their crates at a rate of about one ever 30 seconds for a period of a week straight.
I send the analysis to the crates.io team and pointed that they have a no-automation policy.
They told me that it was not sufficient proof that someone was squatting those names. That's my problem with crates.io is that they have a clear policy and they don't enforce it so all the short/easy to remember names for crates are already taken and there is nothing you can do to get it.
> Using an automated tool to claim ownership of a large number of package names is not permitted.
And
- Hey, I found that someone created crates at a rate of about one every 30 seconds for a period of a week straight.
- That's not sufficient proof of squatting.
Whoever answered that, was either supporting the squatter or explicitly in favor of the practice. I cannot conceive that someone would get that evidence in their hands, and in their right mind think that the claim is bogus. Hell, I'd even be willing to suppress the squatter with evidence of one new crate created every 30 seconds for one hour!
The only reasonable conclusion to make is that they didn't really care. But then don't save face and claim that you do. That's hypocrisy.
There's a secret effort in the Rust community to supplant Crates.io and create an entirely new package ecosystem with proper namespacing, security, and much better community.
Not naming names, but I know several people working to put Crates.io out to pasture.
There's a level of playing nice with them for the time being (eg. build reproducibility), but it's only KTLO.
Crates.io needs to die for Rust to thrive. They're a bungled, mismanaged liability. New code, new leadership.
Crates.io doesn't need to die necessarily. It needs some competition as a wake-up call.
Once a better alternative is out there, crates.io will either wither and die or improve. If it matches its competition in terms of quality and reliability, everyone is better off. If not, the alternative solution will take over.
I'm eager for this crates.io alternative to land, assuming they don't break too many projects in their improvements.
Drama avoidance and avoiding bikeshedding seem obvious. Much easier to present a working system than a design that will get nitpicked into irrelevancy.
That quite funny. Just like when some people formed a violent militant group to take down a violent tyranical dictatorship. Of course they would promise that they would absolutely de-arm right after the dictatorship has been overthrown, and immediately establish a peaceful democratic government with fair election. They absolutely would, would they? They would never turn into what they were formed to replace, would they?
This was my first thought too. And there are a lot of questions that will get asked, like, will all crate library names start being prefixed as well? So you end up with
use::bar; // changing to
use::foo::bar;
I assume the library names that can be overridden in cargo would still be accepted, and then it all gets a little messy. The transition would be very messy.
Write some automated analysis that looks up popular packages on npm, pub.dev, rubygems, nuget. "Rustify" the package names. Add to it frequently used words, maybe popular names, etc. Then, write a script that creates an empty package, registers a name on crates.io every thirty seconds, and then you have about 20k package names after a week that nobody can use.
> Maybe it was a reaction against the Java-style reverse DNS notation
I suspect it was less a reaction against anything and more just following the norms established by most other package managers. NPM, PyPI, RubyGems, Elixir's Hex, Haskell's Cabal... I'm having a hard time thinking of a non-Java package manager that was around at the time Rust came out that didn't have a single, global namespace. Some have tried to fix that since then, but it was just the way package managers worked in 2014/2015.
> I'm having a hard time thinking of a non-Java package manager that was around at the time Rust came out that didn't have a single, global namespace
The implication here is that namespaces in package managers weren't a known concept. Outside Java, NPM - probably the biggest at the time - not only supported them but was actively encouraging them due to collective regret around going single-global in the beginning. Composer is another popular example that actually enforced them.
Not only was namespacing a known widespread option, with well documented benefits, it was one that was enthusiastically argued for within the Rust community, and rejected.
NPM added namespaces in version 2, which was released in Sep 2014, just 2 months before cargo was announced. I don't remember anyone making a big deal about using scopes in NPM for several years after that, it was just there as an option. The announcement blog post of v2 only gives two paragraphs to scoped packages and explicitly frames the feature as being for enterprises and private modules [0]:
> The most prominent feature driving the release of npm 2 didn’t actually need to be in a new major version at all: scoped packages. npm Enterprise is built around them, and they’ll also play a major role when private modules come to the public npm registry.
My memory is that the industry as a whole didn't really start paying attention to the risks posed by dependencies in package managers until the left pad incident.
To be clear, I'm not saying that it was a good idea to not have a better namespace system or that they were completely ignorant of better options, just that they were very much following the norms at the time.
The left pad issue was kind of wild coming from the enterprise Java space. Supply chain attacks against open source software were already being taken pretty seriously, my last company had it's own Maven repository manager running that was used to stage and vet packages before they could be used in production.
I don't think the left-pad problem wasn't about package namespacing it was about the ability to unpublish packages as well the prevalence of micropackages caused by lack of a decent standard library.
Also npm's bad policy/decision to transfer control of package in the name of predictability(this should probably be avoided for packages that aren't malicious. You could argue for seizing broken/trivial and unmaintained packages that have a good name but even then it might be best to leave well enough alone).
I suppose you're talking about the original dispute which led the developer to unpublish his libraries (which npm stupidly allowed, and cargo didn't). There's a smaller chance of a company wanting a random package namespace then a package name but its not impossible (think Mike Rowe Soft vs Microsoft)
> I don't think the left-pad problem wasn't about package namespacing it was about the ability to unpublish packages as well the prevalence of micropackages caused by lack of a decent standard library.
It was "about" cavalier approach to the dependency supply chain. A dependency disappearing outright is just one of many failure modes it has.
> The left pad issue was kind of wild coming from the enterprise Java space.
This may be a little off topic for this comment thread but this is a little misrepresentative. Hosted private repos for enterprise weren't exclusive to Java at the time of left pad - anyone doing enterprise node likely had one for npm too & were probably well prepared for the attack. Such enterprise setups are expensive though (or at least take a level of internal resources many companies don't invest in) leaving the vast majority exposed to both js & java supply chain attacks even today.
At the time Nexus was free to self host, and that is what many smaller teams did just that to archive known good packages for the CI pipeline, I'm not in the Java space anymore so I don't know if that's still the case.
Yeah it's all a bit of revisionist history here, or I guess a bit ignorant. I had a friend who worked at Sonatype from pretty early days and they were, as I understand it, specifically working in this area of infrastructure for vetting, signing, license checking, etc. for corporate environments that needed to be extra careful about this stuff.
That crates.io launched without explicitly acknowledging this whole problem is either naivety or worse: already by then Java wasn't "cool" and the "cool kids" were not paying attention to what happened over there.
It's not that the industry wasn't paying attention until the 'left pad incident' -- that only holds if one's definition of "the industry" is "full stack developers" under the age of 30; I remember when that happened and I was working in a shop full of Java devs and we all laughed at it...
Maven's biggest problem was being caked in XML. In other respects it was very well thought out. That and it arrived at the tail-end of the period in which Java was "cool" to work in.
It's not revisionist history, the wording I chose was meant to acknowledge that there were segments of the industry that did take dependencies seriously. I'm very much aware that the Java world had a much more robust approach to dependencies before this, but "the industry as a whole" includes all the Node shops that were hit by leftpad as well as all the Python and Ruby shops that were using equally lousy dependency management techniques.
Rust chose to follow the majority of languages at the time. Again, as I noted in my previous comment, I'm not defending that decision, just pointing out that most of the widely-used languages in 2014 had a similar setup with similar weaknesses.
mainly, you can trust that anything under the foo/ namespace is controlled by only a smaller group of people, as opposed to the current situation on cargo where people pseudo-namespace by making a bunch of packages called foo-bar, foo-baz, and you can't trust that foo-bin wasn't just inserted by someone else attempting to appear to be part of the foo project. It also helps substantially with naming collisions, especially squatting.
If you want to check your dependency tree for the number of maintainers you're dealing with instead of the number of dependencies, this can be done with cargo tree + checking the Cargo.toml/crates.io ownership info for each of the found packages. I don't know if there's a command written to do that already, but I've done that with a small script in the past.
Fair enough. As I noted to another commenter, I'm not trying to say there was no prior art (if nothing else there was Maven), just that they were following the overwhelming majority of mainstream languages at the time.
> just that they were following the overwhelming majority of mainstream languages at the time.
They were trying to do better than mainstream languages in other areas and succeeded. IIRC on this front they just decided Ruby's bundler was the bee's knees.
The same developer who worked on bundler also worked on the initial version of cargo. That’s why they’re similar.
And at that time, it was a good idea. Ruby was popular and bundler + gem ecosystem was a big reason for its popularity. No one was worried that Rust might become so popular that it might outgrow the bundler model. That was only a remote possibility to begin with.
A mistake that many programmers make, as if baking one more feature on top would have made any difference that wouldn't be amortized in just a few weeks... Sigh.
The people that are working on the project haven't implemented namespaces, or any other security feature really, so what they say is immaterial. What they do is the only thing that matters.
The point is that it's much easier to make a mistake typing "requests" than "
org.kennethreitz:requests" (as a pure hypothetical.)
It also means that more than one project can have a module called "utils" or "common", which once again reduces the risk of people accidentally downloading the wrong thing.
Just some random Cargo security-related issues I noticed:
- No strong link between the repo and the published code.
- Many crates were spammed that were just a wrapper around a popular C/C++ library. There's no indication of this, so... "surprise!"... your compiled app is now more unsafe C/C++ code than Rust.
- Extensive name squatting, to the point that virtual no library uses the obvious name, because someone else got to it first. The aformentioned C/C++ libraries were easy to spit out, so they often grabbed the name before a Rust rewrite could be completed and published. So you now go to Cargo to find a Rust library for 'X' and you instead have to use 'X-rs' because... ha-ha, it's actually a C/C++ package manager with some Rust crates in there also.
- Transitive dependencies aren't shown in the web page.
- No enforcement or indication of safe/unsafe libs, nostd, etc...
- No requirement for MFA, which was a successful attack vector on multiple package managers in the past.
DISCLAIMER: Some of the above may have been resolved since I last looked. Other package managers also do these things (but that's not a good thing).
In my opinion, any package manager that just lets any random person upload "whatever" is outright dangerous and useless to the larger ecosystem of developers in a hurry who don't have the time to vet every single transitive dependency every month.
Package managers need to grow up and start requiring a direct reference to a specific Git commit -- that they store themselves -- and compile from scratch with an instrumented compiler that spits out metadata such as "connects to the Internet, did you know?" or "is actually 99% C++ code, by the way".
> > "We're pretending security is not an issue." has been the feedback every time this is raised with the Cargo team.
> Do you have a specific link where I can read this response, because this is not at all the responses I have read.
Those aren't people saying security isn't an issue but examples of concerns you have which is different.
For some of those, there are reasonable improvements that can be made but will take someone having the time to do so. While the crates.io team might not be working on those specific features, I do know they are prioritizing some security related work. For instance, they recently added scoped tokens.
For some, there are trade offs to be discussed and figured out.
Sure, but all the drawbacks you enumerate are also advantages for gaining critical mass. A free-for-all package repository is attractive to early adopters because they can become the ones to plug the obvious holes in the standard library. Having N developers each trying to make THE datetime/logging/webframework/parsing library for Rust is good for gaining traction. You end up with a lot of bad packages with good names though.
> Extensive name squatting, to the point that virtual no library uses the obvious name, because someone else got to it first.
Maybe the obvious names should have been pre banned. But I don't see the issue with non-obvious names either way you're going to have to get community recommendation/popularity to determine if
In ASP.NET land, I regularly work on projects where there is an informal rule that only Microsoft-published packages can be used, unless there's good reason.
You don't want to be using Ivan Vladimir's OAUTH package to sign in to Microsoft Entra ID. That probably has an FSB backdoor ready to activate. Why use that, when there's an equivalent Microsoft package?
When any random Chinese, Russian, or Israeli national can publish "microsoftauth", you just know that some idiot will use it. That idiot may be me. Or a coworker. Or a transitive dependency. Or a transitive dependency introduced after a critical update to the immediate dependency. Or an external EXE tool deployed as a part of a third-party ISV binary product. Or...
Make the only path lead to the pit of success, because the alternative is to let people wander around and fall into the pit of failure.
Crates.io has publisher information-- namespacing is not required for that. For example, here are all the crates owned by the `azure` GitHub organization and published by the `azure-sdk-publish-rust` team: https://crates.io/teams/github:azure:azure-sdk-publish-rust
Sorta—it looks like they were mostly just using that system by convention until May 2015, when they finally become enforced [0]. Still, that's a good one that I hadn't thought of, and they at least had the convention in place.
I'm honestly astounded at how badly many languages have implemented dependency management, particularly when Java basically got this right almost 20 years ago (Maven) and others have made the mistakes that Java fixed. With Maven you get:
1. Flexible version (of requirements) specification;
2. Yes, source code had domain names in packages but that came from Java and you can technically separate that in the dependency declaration;
3. You can run local repos, which is useful for corporate environments so you can deploy your own internal packages; and
4. Source could be included or not, as desired.
Yes, it was XML and verbose. Gradle basically fixed that if you really cared (personally, I didn't).
Later comes along Go. No dependency management at the start. There ended up being two ways of specifying dependencies. At least one included putting github.io/username/package into your code. That username changes and all your code has to change. Awful design.
At least domains forced basically agreed upon namespacing.
> Later comes along Go. No dependency management at the start. There ended up being two ways of specifying dependencies. At least one included putting github.io/username/package into your code. That username changes and all your code has to change. Awful design.
"github.io/username/package" is using a domain name, just like Java. Changing the username part is like changing the domain name--I don't see how this is any worse in Go than in Java.
If you don't like that there's a username in there, then don't put one in there to begin with. Requiring a username has nothing to do with Go vs. Java, but rather is because the package's canonical location is hosted on Github (which requires a username).
I don't know why so many programmer's use a domain they don't control as the official home of their projects--it seems silly to me (especially for bigger, more serious projects).
Slight difference is that it wouldn't break existing builds if you changed namespaces in Java. The maven central repo does not allow packages to be rescinded once they are published.
So that old version of package xyz will still resolve in your tagged build years from now even if the project rebrands/changes namespaces.
Note that in Java it is merely a convention to use domain names as packages. There is no technical requirement to do so. So moving to a different domain has no impact whatsoever on dependency resolution. Many people use non-existent domain names.
To be honest I really like how Java advocated for verbose namespaces. Library authors have this awful idea that their library is a special little snowflake that deserves the shortest name possible, like "http" or "math" (or "_"...).
Java did a lot of things right beyond the language and VM runtime, both of which were "sturdy" by the standards of the early 1990s. Using domain names for namespaces was one nice touch. Having built-in collections with complete built-in documentation was another excellent feature that contributed to productivity.
It may be a convention but in practice if you want to publish your package to Maven Central you need to prove ownership of your group ID domain. (Or ownership of your SCM account, which is in essence another domain).
> I don't know why so many programmer's use a domain they don't control as the official home of their projects
Not only that, but a commercial, for-profit domain that actively reads all the code stored on it to train an AI. Owned and run by one of the worst opponents of the OS community in the history of computing.
At least move to Gitlab if you must store your project on someone else's domain.
Yes, #3 in particular is important for many large corps where one team develops a library that may be pulled in by literally thousands of other developers.
The dependency management side of Maven is great. OTOH, I was astounded to learn today that Maven recompiles everything if you touch any source file: https://stackoverflow.com/a/49700942
This was solved for C programs since whenever makedepend came out! (I'm guessing the 80s.)
(Bonus grief: Maven's useIncrementalCompilation boolean setting does the opposite of what it says in the tin.)
100% agree. It's unbelievable what a PITA it is dealing with pip or npm compared to Maven even 10 years ago. The descriptors could get convoluted but you could also edit them in an IDE that knew the expected tokens to make things happen.
No you see Java devs have stockholm-syndromed themselves into believe that a giant stack of XML, or some unhinged mini-language are actually good, and much better than something the humans involved can actually read and parse easily and now to compensate with other ecosystems providing 85% of the functionality, with 5% of the pain, they’ve got to find some reason to complain about them.
Is this a joke? XML is horrible to work with, more boilerplate than information. Compare your average maven file to a cargo.toml and tell me which is easier to work with...
"XML is more verbose" is a lazy criticism in the same veign as "Python is better than Java because you can do 'Hello World' in one line".
Maven files have a simple conventional structure and a defined schema. Once you learn it, it's a breeze to work with. It's not like you need to write hundreds of lines of SOAP or XLST — which is actually why people started to dislike XML, and not because XML inherently bad.
Edit: I'd also take XML over TOML any day, especially when it comes to nested objects and arrays.
For a descriptor verbose is superior. It's way clearer what you're looking at. Matching a named end tag is much easier than matching a }. Also, XSD means you can strictly type and enumerate valid values and you will instantly see if you've written something invalid.
Maven stores every version of every library you've ever needed in a central location. It doesn't pollute your working directory and it caches between projects. And this is more of a Java thing than a Maven, thing, but backwards compatibility between versions is way easier to manage. There's no incompatible binaries because you changed the node runtime between npm install and running your project.
The inverse-style domain name thing does a really good job of removing the whole issue of squatting from the ecosystem. You have to demonstrate some level of commitment and identity through control of a domain name in order to publish.
I would also say that this puts just enough friction so that people don't publish dogshit.
crates.io demonstrates quite clearly that you either have to go all the way and take responsibility for curation or you have to get completely out of the way. There is no inbetween.
and i dont particularly think that using xml is that bad. The schema is well defined, and gives you good autocompletion in any competent IDE (such as intellij).
It took some iterations before maven 3 became "good", so people forget that it wasn't as nice before now! Unfortunately, it seems that the lessons learned there is never really disseminated to other ecosystems - perhaps due to prejudice against "enterprisey" java. Yet, these package managers are now facing the sorts of problems solved in java.
I have no problem with XML in general and even think it's still the better format for many things. But it's not really appropriate for a build config. Thankfully Maven now offers polyglot but I've seen no use of it in the wild.
URLs for packages makes a lot of sense. It works well in the land of Go. It also conveniently eliminates the need for the language to have a global packages database. Upload your package to example.com/your-thing and it's released! (You can, of course, still offer a cache and search engine if you want to.)
No, URL's don't make sense because your application shouldn't care where on the internet your dependency happened to be hosted when you integrated it. It's location has nothing to do with what it is.
By the time you're going to production, your vetted and locked dependency should be living in your own cache/mirror/vendored-repo/whatever so that you know exactly what code you built your project around and know exactly what the availability will be when you build/instantiate your project.
Your project shouldn't need to care whether GitHub fell out of fashion and the project moved to GitLab, and definitely shouldn't be relying on GitHub being available when you need to build, test, deploy, or scale. That's a completely unnecessary failure point for you to introduce.
Systems that use URL-identified packages can work around some of this, but just reinforce terrible habits.
URLs are well structured and unique, with a sensible default - sourcing the file from the internet - and ubiquitous processes for easily mapping the URL to an alternative location.
I.e., when you're going to run the production build, the URLs are mapped to fetch from the vetted cache and not the internet.
I don't see any downsides to allowing them as a source, or making them the default approach
> and ubiquitous processes for easily mapping the URL to an alternative location.
This seems strange to me because the whole point of a Uniform Resource Locator is to specify where a resource can be located.
It's a bit like saying "My project depends on the binder on shelf 7 in Room 42, sixth binder from the left. Except when I go into production, then use...." Don't tell me what binder it's in, tell me what it is.
I can see a case made for URIs, which is basically what Java did.
This was a big annoyance for me back in the day when I was dealing with XML namespaces. URLs never made sense for that use case and too many tools tried to pull XSDs from the literal URL which was always generally out of date, some projects switch to URIs like tag uris or URNs and it was much better, imo.
Fully qualified domain names (java/maven) aren't URIs. The latter are far more transient. Maybe a form of permalink could work, but that likely places too great a burden on package maintainers. I don't see that working out honestly.
Isn’t that why GOPROXY exists though? Not sure why you would need an internet connection. URLs don’t necessarily equate to the internet. Our internal and external packages are all locally hosted and work regardless of the internet being available.
> By the time you're going to production, your vetted and locked dependency should be living in your own cache/mirror/vendored-repo/whatever so that you know exactly what code you built your project around and know exactly what the availability will be when you build/instantiate your project.
In the Go world this would be "vendored" dependencies, that is, the dependencies are within your source tree, and your CICD can build to its hearts content with no care in the world about the internet because it has the deps.
The URL is useful for determining which version of a specific project is being used - "Oh we switched to the one hosted on gitlab because the github one went stale"
The advantage of using gitlab, or github, or whatever public code repository is that you get to piggy back off their naming policies which ensure uniqueness.
But, at the same time, there's no reason that the repo being referred to cannot be in house (bob.local) or private.
Having said all of that, the Go module system is a massive improvement on what they did have originally (nothing) and the 3rd party attempts to solve the problem (dep, glide, and the prototype for modules, vgo), but it's not without its edge cases.
Isn't that just delegating the problem? URL dependencies do not replace what crates.io does, and a modern language will still want something like it. You'd just end up with most every dependency being declared as crates.io/foo.
[registries]
maven = { index = "https://rust.maven.org/git/index" }
[dependencies]
some-package = { index = "maven", version = "1.1" }
Obviously Maven doesn't host any Rust crates (yet?), this is just a theoretical example. Very few projects bother to host their own registry, partially because crates.io doesn't allow packages that load dependencies from other indices (for obvious security reasons). The registry definition can also be done globally through environment variables: CARGO_REGISTRIES_MAVEN="https://rust.maven.org/git/index". Furthermore, the default registry can be set in a global config file.
In theory, all you need to do is publish a crate is to `git push upstream master`, and your package will become available on https://github.com/username/crate-name (or example.com/your-package if you choose to host your git repo on there).
Personally, I don't like using other people's URL packages, because your website can disappear any moment for any reason. Maybe you decide to call it quits, maybe you get hit by a car, whatever the reason, my build is broken all of the sudden. The probability of crates.io going down is a lot lower than the probability of packages-of-some-random-guy-in-nebraska.ddns.net disappearing
It doesn't help with the failure mode of dependencies disappearing, which forces people that care about it to vendor, which in turn brings its own set of issues.
Cargo does support URLs to git repos for dependencies. But crates.io is the official platform and almost every search I do on it returns at least one generically named entry with an empty repository that someone snatched away and never used.
I don’t work with rust on the regular, but this is so annoying with package repositories in general. No don’t use http-server, it’s bad, instead you have to use MuffinTop, it’s better. And then you just have to know that. The concept of sanctioned package names would be interesting, but probably chaotic in practice as the underlying code behind this sort of alias changes over time. This will remain a part of being a domain expert in any given ecosystem forever I think, hooray!
Crates.io names are human-meaningful and everyone sees the same names, but it's vulnerable to squatting, spamming, and Sybil attacks.
You could tie a name to a public key, like onion addresses do, but it's unwieldy for humans. (NB, nothing stops you from doing this inside of crates.io if you really wanted)
You could use pet names where "http-server" and "http-client" locally map to "hyper" and "reqwest", but nobody likes those, because they don't solve the bootstrap problem.
It's a problem with all repos because when you say "http-server should simply be the best server that everyone likes right now", you have to decide who is the authority of "best", and "everyone", and "now". Don't forget how much useless crap is left in the Python stdlib marked as "don't use this as of 2018, use the 3rd-party lib that does it way better."
So yeah... probably will be a problem forever. As a bit of fun here are some un-intuitive names, and my proposed improvements:
- Rename Apache to "http-server"
- Rename Nginx to "http-server-2"
- Rename Caddy to "http-server-2-golang"
- Rename libcurl to "http-client"
- Rename GTK+ to "linux-gui"
- Rename Qt to "linux-gui-2"
- Rename xfce4 to "linux-desktop-3"
Then you only need to remember which numbers are good and which numbers are bad! Like how IPv4 is the best, IPv6 is okay, but IPV5 isn't real, and HTTP 1.1 and 3 are great but 2 kinda sucked.
Very simple. If a company as big as Apple can have simple names like "WebKit", "CoreGraphics", and "CoreAudio" then surely a million hackers competing in a free marketplace can do the same thing.
> Perhaps my biggest critique is that crates.io has no namespacing. Anyone can just claim a global and generic package name and we mostly have to deal with it (unless you avoid using the crates.io repository, but then you'll probably have more problems...). Some of these globally-claimed generic packages are not really the best package to use.
This is true that with no namespace anyone can end up squatting a cool name, but with namespace you end up in an even worse place: no-one end up with the cool names, and it makes discoverability miserable, because instead of having people come up with unique names like serde, everybody just name their json serialization/parsing library json and user now needs to remember if they should use "dtolnay/json" or "google/json" (and remember not to use "json/json" because indeed namespace squatting is now a thing), and of course this makes it completely ungoogleable.
We've had the namespace discussion for hundreds of time in the various Rust town squares, and the main reason why we still don't have namespace is because it doesn't actually answer the problem it's supposed to address and if you dig a little bit you realize that it even makes them worse.
Having a centralized public and permissionless repository opens tons of tricky questions, but namespaces are a solution to none of them.
> everybody just name their json serialization/parsing library json and user now needs to remember if they should use "dtolnay/json" or "google/json"
What's the problem with that? You will have to explicitly state intention which is always a good thing.
> and of course this makes it completely ungoogleable
You are aware that just pasting username/projectname gives you exactly what you are looking for in the several top search engines, correct?
> We've had the namespace discussion for hundreds of time in the various Rust town squares, and the main reason why we still don't have namespace is because it doesn't actually answer the problem it's supposed to address and if you dig a little bit you realize that it even makes them worse.
OK, if you say so. Is this worse state of things documented somewhere?
If you have a namespace, can't people just globally-claim namespaces instead? like serde/serde or something similar. I feel if you really don't want people claim whatever they want, you have to do the Java package style where namespaces are tied to domain names.
While Java made that notation famous, it was already used in NeXTSTEP and Objective-C, hence why you will see this all over the place in macOS and derived platforms, on configuration files and services.
CPAN has the best model IMHO. Hierarchy that starts with categories. You build on top of the base stuff and extend it, rather than reinvent/fork something with a random name. Result is a lot more improvement and reuse, and more functionality with less duplication. Plus it's easy to find stuff.
Perl's whole ecosystem is amazing compared to other languages. It's a shame nobody knows it.
And the horrible decision to not make library-level ("module") and code-unit-level ("package") namespacing orthogonal. The former was an afterthought tacked on since the package system was designed to be used only within Google's monorepo and little care was paid to how it would work when it was released to the public and used more generally.
I want to understand what you just said, but I fear watering your language down a bit might be a tall ask with some people. Would you be willing to eli5 what you believe Go did that was a horrible decision with regards to module/package namespacing?
I think what they mean is: if you see a line like `import git.example.com/foo/bar/baz`, that could be package `baz` inside module `git.example.com/foo/bar`, or it could be package `bar/baz` inside module `git.example.com/foo`.
Also, even if you know it's the latter, package namespacing isn't strictly related to directory structure, so `bar/baz` has no specific meaning outside of the context of a go import. They could have used any other separator for package components - `git.example.com/foo:bar:baz` - but instead they chose the slash, making the scheme both technically ambiguous and easy to confuse for an HTTP URL.
Ah that makes sense. I think Go did somewhat stumble a bit in the early days due to this, especially with repositories in GitLab, where GitLab allows essentially a directory tree where your repository can be nested indefinitely in directories like `https://gitlab.com/mygroup/subgroup1/subgroup2/repository`.
I still don't think this is a huge issue, to be honest. Not one big enough for me to complain about, for sure. But it's definitely not ideal.
This needs to be resolved by every damned language.
Just make signed dependencies a universal default, point to an https page for the package vendor and use the signing key from there.
Neither node nor maven ever bothered to solve this, so we end up wandering the Wild West wondering when it will be that HR, or legal, or architecture comes knocking on the door to ask what we were thinking having a dependency on a dynamic version of Left Pad.
I'd kinda like to see what Cloudflare and Let's Encrypt could come up with if they worked together on at very least a white paper and an MVP POC.
Pay somebody either internally or externally to maintain a repo of all your dependencies and point your code at that. You won't get a left-pad incident. You won't get a malicious .so incident (unless you mirror binaries instead of source code).
Like if you ran out of screws to make your product with do you walk around the street and scrounge up some? No, you go to a trusted vendor and buy the screws.
I'm suggesting that the guys who've repeatedly proven themselves technically competent around security at scale might have a couple of useful ideas regarding how the industry might go about crawling its way out of this little security clusterfuck.
And perhaps even stop treating something as simple as a BOM as an enterprise feature, given that the overhead on such things is damned near zilch and the security implications are staggering.
There's reasons why Google projects don't go out on the internet to get their 3rd party deps.
They're all checked into Google3 (or chromium, etc.). One version only. With internal maintainers responsible for bringing it in and multiple people vetting it, and clear ownership over its management. E.g. you don't just get to willy nilly depend on a new version -- if you want to upgrade what's there, you gotta put a ring on it. If you upgrade it, you're likely going to be upgrading it for everyone, and the build system will run through the dependent tests for them all, etc.
And the consequence is more responsible use of third party deps and less sprawling dependency trees and less complexity.
And additional less security concerns as the code is checked in, its license vetted, and build systems are hunting around on the Internet for artifacts.
Agreed. The other thing I don’t really like is that you can’t split up things in the rust namespace hierarchy between crates (something that’s natural with jars in JVM). I would have liked to have defined things so that I could have the unicode handlers for finl live in finl::unicode, the parser in finl::parser, etc. but because they’re in separate crates rust gets upset about finl being defined twice and there’s no workaround for it. There are likely pitfalls I don’t see in what I want, so I live with it.
As I recall, I could do something where I could have a common root crate that would import and re-export the other crates with the namespaces modified on the exports and then control which crates are exported through feature gates, but it just seemed more hassle than it was worth.
I don't think a lack of namespace is that much problem. Sure, it is often annoying, but people are creative enough to create a short enough and still available crate name for most cases. Namespacing only makes sense for a large group of related crates---and it wouldn't give much benefit over a flat namespace.
As other mentioned though, a typosquatting is a much bigger problem and namespacing offers only a partial solution. (You can still create a lookalike organization name, both in npm and in Github.)
I love this lack of namespacing personally, because it means that whatever crate you see in a project is going to be the same as the crate your see in another one. Never need to alias crate names. It happens in Golang all the time and I really think namespacing packages was a mistake there.
Golang's problems aren't due to using namespaces, they're due to delaying too many decisions until too late.
Go has namespacing mostly because for a long time it didn't have a package manager at all, so people just used a bunch of ad hoc URL-based solutions mostly revolving around GitHub, which happens to have namespaces and also happened to lend itself to aliasing (because a whole GitHub URL is too long).
If you want to look at an actual example of namespacing done well, Maven/Java is the place to look. There is no aliasing—the same imports always work across projects.
I don't see the problem. Even with namespaces you'll have
brandonq/xml vs parsers/xml with no clue if one is better then another.
Also possibly with some confusion over whether things with the same name are forks or not. May make it a little more difficult to Google. Why not just have brandon_xml vs xml-parser and have a community list of best and most popular libraries?
I guess the only issue is that some generic/obvious package names are bad packages. That can have been avoided if they banned/self-squated most of the names. I suppose if you use dns namespaces and actually tie it to ownership of the domain name it might make sense but that would also cause issues(what if you forgot to renew the domain?).
The advantage is one of trust. If the `abc` developers build well known library `abc.pqr` are well trusted then I know I can use `abc.xyz` and everything else under the same namespace without (much) vetting.
We could even have `rust.xyz` for crates that are decoupled from `std` but still maintained by rust core devs such as `regex`.
> brandonq/xml vs parsers/xml with no clue if one is better then another.
You will have this problem only once. After an initial research you settle on one and move on with life.
I don't get how having to vet a dependency is going to be more difficult than before. The process is 99% the same, you still have to do the research work initially in both cases.
Yet to see any proof that namespacing has made things better in other ecosystems, are go style links or other types of namespaced imports any less prone to supply chain risks?
It's definitely a good thing that people choose new unique names for crates rather than dijan/base64 vs dljan/base64
Do understand the desire of having a crate for audio manipulation called "audio" but at the same time how often do we end up with "audio2" anyway? It's an imperfect solution for an imperfect world and I personally think the crates team got it right on this one.
> Yet to see any proof that namespacing has made things better in other ecosystems
It's really as simple as this: many libraries are generic enough implementing something that already exists. Let's say you want a library to manage the SMTP protocol. On crates.io, of course someone has already taken the "smtp" crate (ironically, this one is abandoned, but has the highest download counts, because it's the most obvious name). Let's say you disagree with the direction this smtp crate has gone, and you make your own. What do you call it?
Namespaces solve this problem. You'd instead of have user1/smtp and user2/smtp competing in feature sets. You can even be user3/smtp if you don't like the first two.
This is precisely what Java enables too. The standard library is in com.java.*; if you don't like how the standard library does something, you can make com.grimburger.smtp and do it yourself. If you choose to publish to the world, all the more power to you. It doesn't conflict with the standard library's smtp implementation.
This is a common critique, and although I don't have insight into why the original decision to not have namespaces was made, the current outlook is that until issues related to continuity are resolved, it's a no go:
That article starts with the premise that “it’s a feature, not a bug” then goes on to describe a whole bunch of things I consider to be anti-features of a packaging system that has a flat namespace.
The first section says it discourages forking. I consider this to be bad. Nobody’s code should be more important purely because it squatted a better name.
The Identity section actually makes the case that flat registries make naming harder.
The section on Continuity is “we’ve tried nothing and we’re all out of ideas”. Make up an org name and grandfather all packages in the flat namespace into that special org. Also this is already a problem because packages in the flat namespace do get abandoned, then forked, and then we have the associated issues.
The section on Stability seems to take it as a given that crates.io should be the only registry. I don’t. It also seems to conflate cargo with rustc for the benefit of the argument.
The squatting section describes only anti-features and I don’t consider the author’s legitimate use cases to be legitimate reasons to squat.
I think the only legitimate problems that need addressing are the ergonomics of accessing namespaced packages throughout transient dependencies and backwards compatibility with non-namespaced code. But the fact that these are real problems does not, to me, make a flat namespace a “feature”. It’s just easier to implement.
It’s okay for it to be a mistake that takes effort and time to fix.
Another option would be to grandfather all packages into their own org. So serde becomes serde/serde. This way you don't need to manage permission rules in the legacy "all" namespace.
You get some oddities such as serde-derive/serde-derive but the package owners can choose if they want to move to serde/derive or leave it in a separate namespace.
> It also looks like (soon) you’ll finally be able to configure global lints for a project. Until now, you had to hack your solution to keep lints consistent for projects.
> I questioned my sanity every time I circled back around to the Clippy issue above. Surely, I was wrong. There must be a configuration I missed. I couldn’t believe it. I still can’t. Surely there must be a way to configure lints globally. I quadruple-checked when I wrote this to make sure I wasn’t delusional.
You create a .cargo/config.toml in the workspace root so it covers all your crates.
The only limitation is that rustflags are not additive, so if you have other sources of rustflags like the RUSTFLAGS environment variable it will overwrite this setting.
I'm learning Rust because it seems clear that it's going to be important professionally. I wish I loved it, I really do. I see the benefits. But, at least so far, it's one of the most unpleasant languages I've used. I keep hoping that as I gain proficiency, I'll stop disliking it, but as I climb higher on the learning curve, I'm not really warming to it.
It's fine. It won't be the only language I'll be proficient in while being averse to it at the same time. But I heard so many people proclaiming their love for it that I expected to enjoy it, too.
As a counter example, I love programming in Rust. Fighting the borrow checker ended a long time ago. Even the errors are rare these days, except in cases I trigger them in order to examine the types. Rust compiler also seems to have improved in accepting broader cases that are valid.
For me, the key to understanding the borrow checker was understanding the underlying memory model. Rust memory model is the same as that of C, with some extensions for abstractions like generics. The borrow checker rules seem arbitrary at first. But it's deeply correlated to this memory model. The real value of the borrow checker is when I trigger it unintentionally. Those are bugs that I made due to lapse in attention. What scares me is that another language like C or C++ might simply accept it and proceed.
Yet another pleasant side effect of Rust's strict type system and borrow checker is that they gently nudge you to properly structure your code. I can say for certain that Rust has improved my code design in all the languages that I use.
My experience is that most programs dealing with ordinary problems don't need such complicated data structures. In cases you do, Rust has a few options:
1. Use Rust's runtime safety checks, using Rc, Weak, RefCell, etc. It will be as ergonomic as GC'ed languages (no productivity loss). While this has a runtime performance penalty, it will still be mostly comparable to other languages. This works for most use cases.
2. If you need the last ounce of performance, just drop all automated safety checks and do it manually using unsafe. Even if you make a mistake, your debugging will be limited to the unsafe blocks. This approach isn't unusual in Rust.
3. Use either the standard library or something on crates.io that does what's given in 2. Rust's generics make it easy.
1. Option 1 quickly degrades into unrefactorable mess of TypedArenas, nested Rc<RefCell>>, lifetime annotations everywhere, and the like. It is unmanageable, which is why even libraries like PetGraph do not use it.
2. No, I don't need every last bit of performance. C++ performance is OK. Or any other sane language like Nim, Crystal, etc. I need async, though, which is yet another hell in Rust.
3. This absolutely can't be done with standard library. There are libraries like PetGraph, which solve this particular problem (by some very unobvious approaches and bits of unsafe code), but there are many problems like it which don't have any libraries for it yet.
I do have hard problems. I want a language which makes hard problems easy, and impossible problems hard. Rust is definitely _not_ such a language, and it makes me like 5 times less productive, and sucks all joy out of programming.
Use one of the many libraries that have safe abstractions over unsafe code? I don't know why people think you need to roll your own? I guess that's just what people do in c/c++, doesn't seem very productive...
There are definitely no less libraries for C++. I am a scientific software engineer, and occasionally I do develop new things. And Rust makes writing new system-level software way harder, which I don't understand, as it is a system-level language.
The lack of default/named function arguments is what still gets me. It's such an absolute basic programmer ergonomics feature shared by most popular languages; even C++ has had defaults since forever.
It would have been nice for Rust to have Default/named/optional function arguments because the proliferation of slightly differently named functions that do the same thing would go away.
Good question! There's no single thing, I think, and the things I dislike about it aren't even really technical criticism or the like. They're more... aesthetic? I find the syntax unpleasant, for instance. It's extremely opinionated and a few of those opinions are ones I disagree with.
Also, just generally, it tends to make even simple things pretty complex. I understand why and am not really objecting to that, but it does make using it a bit like running with lead shoes.
I expect that the latter part may get better as I use it more (but perhaps not -- there are other languages I'm fully competent in but dislike for similar reasons).
I share your sentiment about Rust. Fine language for sure, I just don't enjoy programming in it. Simply put, I don't like using the abstractions that the language encourages.
For what it is worth, I found pursuing Zig to be a breath of fresh air. It's a promising alternative in the same space as C (and also Cpp and Rust). Checkout Andrew Kelley's introduction to the language.
There are a few, and they aren't something that I can't live with, of course. A great example of the sorts of things I'm talking about are Rust's insistence on camel case and snake case.
> After two decades of JavaScript and decent experience with Go, this is the most significant source of frustration and friction with Rust. It’s not an insurmountable problem, but you must always be ready to deal with the async monster when it rears its head. In other languages, async is almost invisible.
I am a former C and C++ programmer who lived calling into pthread almost every week for a decade. I use async rust everywhere now.
I don’t get the hate that async gets. In my opinion, everyone should be using async for everything. Including stuff that’s seemingly single threaded “simple” stuff.
I think it is the infectiousness of it. Especially in embedded or wasm contexts, the predominant async may not be the async you want. Wasm being the author's use case would definitely have provided a different perspective.
Similarly, I find tasks that use or reuse large buffers to avoid the performance hits from allocation, often benefit from old fashioned thread pools. Bump or shard allocators can make this work ok, but in the cases where you are cpu bound on tight loops of vectorizable operations, thread pools perform better. Async is a good tool, but there are contexts it isn't optimal for.
Recently tried writing some async Rust to compare the error handling when nested async calls are made to how errors are handled in Go, and it seemed like the trivial example I was trying to write up simply couldn't be done without involving Tokio. That barrier simply doesn't exist in Go, or C#, or Typescript.
For instance, you apparently cannot `await` in the main function without a decorator you import from, you guessed it: Tokio.
You need an async runtime to run async code yes, and Rust's isn't built in. Why does that matter though? Rust has a decent package manager; add the dependency and move on.
Why are you trying to avoid Tokio lol, tokio is the defacto async runtime in rust, saying you're trying to avoid it is like saying you're trying to avoid async while writing async, somehow people act like if they merged tokio into std and instead of #[tokio::main] or whatever you had to do #[async::main] it would somehow be better.
Once you stop fighting the fact that tokio = async rust for 99% of cases, things are quite smooth.
Tokio simply doesn't meet the ise case of some people doing wasm, and many people doing embedded. That isn't a huge deal, just don't use it right? Except many otherwise usable crates seem to adopt tokio unnecessarily.
The problem that I am running into at the moment is that a few things like the rhai Engine aren't Send and I am trying to use them in an async closure. What GPT-4 suggested was creating a tokio Runtime inside the thread and then block_on(). I will try it tomorrow. (This is the first significant Rust project for me.)
1. Not applicable unless it is absolutely required, and not something I will consider until I have exhausted other options, since there is a reason they have not made it Send already. It will likely be quite complex and enlarge the scope.
2. I am using a normal thread but when I make it async the compiler wants Send.
3. The await point is in code that uses the engine. I am not sure there is another good option, since I need to use an API that has several libraries all of which are async.
The block_on is to allow the tokio Runtime created in that thread to execute/poll it
So, thank you for your input, I will test out the suggestion that I mentioned above, and then maybe look into spawn_blocking if that doesn't work.
That ChatGPT suggestion is dodgy. Tokio is going to complain when you create a runtime while in async runtime, and refuse block_on in an async context.
You can have multiple runtimes, but create them ahead of time, in synchronous main, and keep a Handle to them.
> 2. I am using a normal thread but when I make it async the compiler wants Send.
This is likely because you are using the multi-threaded scheduler, which requires futures to be Send, even if you’re running only a single thread. This is because Tokio is based on a “work stealing” runtime, so in “normal” operations, expects the futures themselves to be able to be shuffled around threads where necessary.
For you use case, try running the single threaded executor, additionally try the “local set” executors. These do not require Send as they are statically guaranteed to be confined to the thread they are spawned on. Block on will also work.
Out of curiosity though, how is the interaction with Rhai performed, are you passing around the engine, and executing the code at certain points where applicable? Do you just need certain results from it sometimes? Etc?
It being !Send means its not safe to be sent lol, that's down to the way they implemented rhai engine not rust. It's just that rust catches that it's not safe to send because of the trait bounds.
The way closures made it easy to encapsulate code and state in an object you can run at any time, async encapsulates code and state for ability to run and pause or cancel at any time.
This is handy for I/O that can be interleaved and cancelled. You can (ab)use it for other things like generators or various DIY multitasking operations. It can also be a state machine generator (e.g. AI of actors in a game).
But I think OP just meant async for typical networking and DB interfaces. And yes, this usually implies the Tokio dependency.
Programming in Rust is really not like being in an abusive relationship. The compiler is trying to help out as much as possible, especially since rustc has the best error messages in the world.
The OSes want programmers to handle resources correctly, and the Rust compiler makes that task a breeze. We are in a more abusive relationship with our OSes, than the Rust compiler. How about the hardware? Doesn't that need to run assembly in a correct way? That counts as an abusive relationship as well.
Rust's error messages are one of a kind. There no other compiler which comes even close.
As a side note, i used latex lately, it's error messages are horrendous. What a nightmare to figure out what's wrong by inspecting the error.
> The compiler is trying to help out as much as possible, especially since rustc has the best error messages in the world.
Generally good, but man do I hate how any error in my async function causes every recursive call site to generate an error about how the Future is no longer Send and Sync. Literally an entire console scrollback of errors with the actual syntax error buried somewhere in the middle.
I believe this is in our radar and waiting on the new trait solver, but just in case if you have a repro I would appreciate a ticket to improve the diagnostic.
In this case, the first two errors are clear. The next 3 provide multiple, redundant context blocks. The upshot is that this simple example results in rustc printing 11 lines of useful error messages and 86 lines of useless messages. Add to that the fact that the useful error messages need not be at the top or bottom of the error list, they can be anywhere inside of it, depending on the declaration order.
That is a different problem than the one I thought you were seeing.
We do spend a lot of time trying to silence errors that are irrelevant. We also get a lot of complaints when fixing a single error produces a wave of new errors that were hidden due to failures in earlier stages. It's a balancing act.
Also, specific errors are verbose in order to give people a fighting chance to fix their issue. An error that is too verbose is annoying. An error that is too terse will leave users in the cold as to what the problem was. It's yet another balancing act.
> Could I ask you why you didn't consider doing so when you first encountered this problem?
Thanks for looking into it. I have a filed a bug against Rust before, but that was a clear bug, not a poor error message. Remember, the only time a user sees something like this is when they are trying to do something else. The only reason I looked into it just now is because you explicitly asked.
We consider poor error messages to be bugs. I know most people don't, so I've made it one of my missions to yell it from the mountain tops to encourage people to report more often. We can't fix what we don't know about. The majority of the current good errors were a reaction to things people filed in the past.
Rust's compiler is the first one I've seen to use the word perhaps.
My main gripe is that I still don't fully comprehend lifetimes and the compiler can't really help me every time, because it (understandably) errs on the side of caution.
Whenever you see weasel words like that, it means the compiler knows that the issue you encountered could be what it is saying but the necessary metadata to figure out for sure is inaccessible to it. It's the problem with classical stage-oriented compilers. A compiler designed for diagnostics from the beginning would end up looking like a plate of spaghetti where you can call type checking from the parser, to give an example.
>It's the problem with classical stage-oriented compilers. A compiler designed for diagnostics from the beginning
I'm not sure any compiler, even one "designed for diagnostics," can gather the "necessary metadata" from inside my brain based on my original intentions. If the compiler could unambiguously interpret what I meant to write, it wouldn't have needed to fail with a compilation error in the first place. Whenever I see something like "perhaps" or "maybe" in a compilation error, the information just didn't exist anywhere the compiler could possibly get to, and it's just a suggestion based on common mistakes.
As one of the main people working on Rust compiler diagnostics, I find this comparison beyond distasteful. The tooling is not capricious in its restrictions and we go out of our way to make it communicate to people with as much empathy and support as possible.
I'm sorry you're having a bad experience. I've found changing a couple habits developed in other languages helped me to have a good experience with Rust's diagnostics.
1. Reading the error messages. I was used to error messages verbosely printing a lot of details which were mostly irrelevant and letting the programmer sort it out, and I developed a habit of skimming them. I had a better time with Rust when I realized it was giving me information that was mostly relevant, and that I should actually read them.
2. Interpreting error messages as feedback rather than failure. I was used to error messages meaning I had done something wrong, and getting them all the time was frustrating. Watching Jon Gjengset's coding videos, I was struck that he wrote code to trigger errors deliberately in order to get feedback from the compiler. I now try to work through Rust error messages the same way I would failing unit tests; I make some small changes, I knock out the errors, and then move on to the next subtask. This keeps the number of errors small and manageable, and gets me into a tight feedback loop with the compiler.
Relatedly, I encountered the idea (in a blog post I no longer remember) that the Rust compiler is more like an automated pair programming partner than other compilers. This change in mindset really helped for me; thinking of it as a friendly helper rather than a loud complainer made the work lighter. (That's why I personally don't like the 'abusive partner' metaphor, for me that mindset makes it toilsome and miserable. Also because I've had an abusive partner, and it's nothing like that, and I could do without the reminder.)
3. Setting up my IDE to show me inline type annotations and error messages. This made the feedback loop tighter, I only have to drop into a terminal for complex or unfamiliar error messages. With practice I can fix most errors using a one-line summary.
4. With practice, I've internalized Rust's semantics, and I've stopped painting myself into corners with unnecessarily cyclic data structures and such. I also read a blog article or maybe a tweet (which I've also forgotten) about giving yourself permission to use Box or clone and not sweating every copy or allocation. It pointed out that if you were writing Python, everything would be an Arc<T>, and you'd think nothing of it; in Rust the performance cost is explicit, so you agonize over it, but a lot of the time it's not a big deal to copy some data or to perform an allocation.
Wasn't my intention. I'm sorry if I came off that way, I can see how my comment might be presumptive or pretentious. My bad. You don't have to do any of that. You don't even need to write Rust in my book, it's not the one true language or anything. That's just what's working for me personally.
My first six months with Rust were very painful and I struggled to write the simplest programs. (This was in fact, the first six months of my second attempt - the first time I tried to learn it, I gave up.) Eventually I learned how to work with it, and since then, it's my absolute favorite language to work in (though presently I mostly write Typescript and SQL, because I'm writing webapps). I tried to summarize what happened along the way and what changed.
I would encourage you to file tickets whenever you encounter deficiencies in the tooling, including bad diagnostics. We take them seriously. As for whether our efforts are accomplishing anything, trying out older releases can be eye opening at how much has changed.
This really is an inappropriate comparison. Can we be serious for a bit? A professional-grade tool providing professional-grade feedback is not remotely like an abusive or even turbulent relationship.
> I started writing tests in Rust as I would in any other language but found that I was writing tests couldn’t fail.
This is a common refrain in C++ testing: if it compiles then it's probably correct.
> Rust has accounted for so many errors that many common test cases become irrelevant
In practice, if you think this way I think it's a sign that you aren't testing the right things in those other languages. You should be testing business logic, not language stuff.
If you look at your test code and think "I would test this in JavaScript, but I don't need to do that in Rust" then just delete the test.
I’ve definitely experienced it in Haskell and Rust. I can believe some C++ could be that way, but I’ve never experienced it, but then again those projects didn’t have useful tests either. I think with C++ a lot of this depends on domain and the quality of the code and libraries.
A null pointer exception is a bug that breaks business logic. There's no "business logic instead of language stuff" because the language stuff is the foundation that business logic rests on. If you don't test against failure modes, what's even the point in testing?
To close the loop, Rust doesn't include a `null` type and you wouldn't encounter something comparable in idiomatic Rust (because you'd be using eg Option::map to handle None cases gracefully), so this is a class of test that would be common in Java and C that is close to irrelevant in Rust.
To be more specific, Rust does have among other things:
std::ptr::null() - an actual null pointer, probably the zero address on your hardware, and this isn't even an unsafe function. On the other hand, you won't find many uses for a null pointer so, I mean, congrats on obtaining one and good luck with that.
std::ptr::null_mut() - a mutable null pointer, similarly unlikely to be of any use to you in safe Rust, but also not an unsafe thing to ask for.
But, these are pointers, so they're not values that say, a String could take, or a Vec<String> or whatever, only actual raw pointers can be null.
Sure, you can panic unwrapping a None, but there are two important distinctions.
First, this is a controlled panic, not a segmentation fault. The language is ensuring that we don't access the null pointer, or an offset from the null pointer. Null pointer access can be exploitable in certain circumstances (eg, a firmware or kernel). Your use of "exception" suggests you're thinking about it in Java terms however, and Java is equivalent here.
Second, you can only encounter this in explicit circumstances, when you have an Option<T>. Wheras in languages with a null type, any variable can be null implicitly, regardless of it's type.
>First, this is a controlled panic, not a segmentation fault.
But a segmentation fault is also controlled
>Your use of "exception" suggests you're thinking about it in Java terms however, and Java is equivalent here.
No, I am thinking in Delphi terms. It is overspecialized to Windows userspace. Windows gives an Access Violation, and that can be caught, and Delphi throws it as exception
>Second, you can only encounter this in explicit circumstances, when you have an Option<T>. Wheras in languages with a null type, any variable can be null implicitly, regardless of it's type.
Delphi has both nullable types and null-safe types
A segmentation fault may not happen (which is to say, we may corrupt memory or worse) if the null pointer is mapped into memory (as in an embedded or kernel context) or if it's accessed at a large offset, which results in a pointer that's mapped into memory. This may be rotten luck or it may be exploited by an attacker.
> No, I am thinking in Delphi terms.
Fair enough, I don't know Delphi. I'll take what you're saying about it as read.
If an exception breaks business logic, then you can test the code by testing business logic.
How do null pointer exceptions arise?
Inconsistent data: you have code paths that implicitly assume invariants. Trivial cases like “this field is not provided” and more complex ones like “these fields have a specific relationship”.
You can often move those assumptions into the data structure in any language.
Then your tests become a matter of generating and transforming data from a holistic perspective instead of micromanaging individual code paths.
A null pointer exception is a free runtime check. I really don't understand the fuss about null. The "most expansive design mistake in computer science" and whatever.
Free runtime check of what? The point is that if nulls don't exist, there is no need for a "check". Runtime or otherwise. If I write a type that models Foo, I want it to model Foo and not "Foo or null". If I want to model "Foo or no data" then I use a separate type that makes my intention clear (in Rust spelled `Option<Foo>`). Languages where nothing is precisely Foo and everything is "oh by the way this is only possibly Foo" are deeply, deeply flawed.
There is surely something wonderful about the kind of C++ programmer who figures that, since their unusable broken garbage compiled it's probably correct.
Remember unlike most languages you'd be familiar with C++ has IFNDR, which has been jokingly referred to as "False positives for the question: Is this a C++ program?". A conforming C++ compiler is forbidden from telling you in some† unknown number of cases that it suspects what you've written is nonsense, it just has to press on and output... something. Is it a working executable? Could be. Or maybe it's exactly like a working executable except it explodes catastrophically on Fridays. No way to know.
† The ISO standard does identify these cases, but they're so vague that it's hard to pin down everything which is covered. My guess is that all or most non-trivial C++ software is actually IFNDR these days. Just say No to the entire language.
> A conforming C++ compiler is forbidden from telling you in some† unknown number of cases that it suspects what you've written is nonsense, it just has to press on and output... something.
It's not quite that bad - a conforming C++ compiler is permitted to error out and not compile the program. It just doesn't have to.
Many of the cases of IFNDR are semantic constraints, especially in C++ 20 and beyond. As a result of being semantic constraints it's generally impossible to diagnose this with no false positives. The ISO standard forbids such false positives so...
Can you give a more concrete example of the kind of thing you're talking about? Like, if you try to sort something and your comparator implementation for that type is not transitive, the compiler can silently produce a broken binary?
Surely in the undecidable cases the compiler is allowed to produce a binary that errors cleanly at runtime if you did in fact violate the semantic constraint, and any sane implementor would do that. (Not that any sane person would ever write a C++ compiler...)
> Like, if you try to sort something and your comparator implementation for that type is not transitive, the compiler can silently produce a broken binary?
It's not merely about whether your comparisons are transitive, the type must exhibit a total ordering or your sort may do anything, including buffer overflow.
> Surely in the undecidable cases the compiler is allowed to produce a binary that errors cleanly at runtime if you did in fact violate the semantic constraint,
I don't think I know how to prove it, but I'm pretty sure it's going to be Undecidable at runtime too in many of these cases. Rice reduced these problems to Halting, which I'd guess means you end up potentially at runtime trying to decide if some arbitrary piece of code will halt eventually, and yeah, that's not helpful.
I've written about it before, but I should spell it out: The only working alternative is to reject programs when we aren't sure they meet our constraints. This means sometimes we reject a program that actually does meet the constraints but the compiler couldn't see it.
I believe this route is superior because the incentive becomes to make that "Should work but doesn't" set smaller so as to avoid annoying programmers, whereas the C++ incentive is to make the "Compiles but doesn't work" set larger since, hey, it compiles, and I see Rust's Non-Lexical Lifetimes and Polonius as evidence for this on one side, with C++ 20 Concepts and the growing number of IFNDR mentions in the ISO standard on the other side.
> the type must exhibit a total ordering or your sort may do anything, including buffer overflow.
Sure. But there's no requirement for the compiler to be a dick about it, and hopefully most won't.
> I don't think I know how to prove it, but I'm pretty sure it's going to be Undecidable at runtime too in many of these cases. Rice reduced these problems to Halting, which I'd guess means you end up potentially at runtime trying to decide if some arbitrary piece of code will halt eventually, and yeah, that's not helpful.
At runtime can't you just run the code and let it halt or not? Having your program go into an infinite loop because the thing you implemented didn't meet the requirements is not unreasonable.
I'm sympathetic to the idea that there could be a problem in this space, but without a real example of a case where it's hard for a compiler to do something reasonable I'm not convinced.
> I've written about it before, but I should spell it out: The only working alternative is to reject programs when we aren't sure they meet our constraints. This means sometimes we reject a program that actually does meet the constraints but the compiler couldn't see it.
> I believe this route is superior because the incentive becomes to make that "Should work but doesn't" set smaller so as to avoid annoying programmers, whereas the C++ incentive is to make the "Compiles but doesn't work" set larger since, hey, it compiles, and I see Rust's Non-Lexical Lifetimes and Polonius as evidence for this on one side, with C++ 20 Concepts and the growing number of IFNDR mentions in the ISO standard on the other side.
Meh. I'm no fan of the C++ approach, but I'd still rather see C++ follow through on its strategy than half-assing it and becoming a watered-down copy of Rust. People who want Rust know where to find it.
Historically correctness wasn't seen as an important goal in C++ and so no, I don't think any of the three popular C++ stdlib implementations will do something vaguely reasonable for nonsense sort input. It's potentially faster (though bad for correctness) to just trust that this case can't happen since the programmer was required to use types with total ordering. So yes, I'd expect it to result in bounds misses in real software.
I wouldn't think you could get bounds misses without doing extra comparisons, so I'd expect real-world sort implementations to just fail to sort, which doesn't seem particularly unreasonable. But in any case, it's a huge leap from "existing C++ stdlib implementations behave badly in this case" to "the C++ standard requires every implementation to behave badly in this case".
Can you give an example of non-C++ code that a modern compiler (MSVC, clang, g++ or something) successfully compiles with no diagnostics? I’m genuinely curious. If not, this just sounds like more C++ FUD because the spec doesn’t define everything under the sun and allows a certain amount of leeway to compilers for things like emitting different error diagnostics.
Per the C++ standard ([lex.name]/3), this program is ill-formed:
> In addition, some identifiers appearing as a token or preprocessing-token are reserved for use by C++ implementations and shall not be used otherwise; no diagnostic is required. [...] Each identifier that contains a double underscore __ or begins with an underscore followed by an uppercase letter is reserved to the implementation for any use.
Thus, the compiler theoretically has the liberty to emit whatever it wants for this program.
Neither GCC nor Clang produces a warning under -std=c++20 -Wall -Wextra (Clang only produces a -Wreserved-macro-identifier under -Weverything), and MSVC doesn't produce a warning under /std:c++20 /Wall.
In practice, most examples of ill-formed programs where compilers issue no warnings occur with discrepancies between different source files that are linked together; e.g., declaring a function as inline in one file but non-inline in another, or declaring a function with two different sets of default arguments in different files, or defining the same non-inline variable or function in different files, or defining the same inline function differently in different files.
Don't forget since C++ 20 all programs which rely on a concept but don't fulfill the (unchecked) semantic constraints of that concept are ill-formed. This allows C++ to truthfully say that C++ programs have the same properties as Rust programs in this regard, because although most real world C++ may fail those constraints and they aren't checked, as a result those aren't technically C++ programs at all so the fact they're broken and don't do what their programmers expected is magically not the fault of C++ even though the compiler seemed fine with it.
> Thus, the compiler theoretically has the liberty to emit whatever it wants for this program.
In theory, sure. In practice, what does it do? We can look and see.[0][1][2] (I don't know of any compiler that emits garbage in the example you listed). For some reason, there's this sentiment that's arisen that treats undefined behavior as some sort of bugaboo that's capable of anything and everything (including summoning nasal demons).
Here's a question that I can't find an answer to. Is machine code well-defined (as in, can it contain undefined behavior)? If the answer is yes, then all Rust programs can also contain undefined behavior, because they eventually turn into machine code after all. If the answer is no, then that means once a C++ program is compiled into machine code, the executable is a well-defined program.
This whole nasal demon garbage was a good hyperbole at the time to explain that, yes, undefined behavior is bad. But it's been taken to this extreme where people will use arguments like this to try and convince others that if your program has undefined behavior, then it could summon a nasal demon, because who knows what could happen.
In reality, that won't happen. In reality, a signed integer overflow can result in a meaningless program, or a security vulnerability, or many other bad things, but it doesn't mean that the compiled machine code is all of a sudden inventing new instructions at random that the CPU will happily accept. It doesn't mean that your program will turn your OS into a pac-man game. It doesn't mean that it's impossible to find the root cause of the issue and remove the undefined behavior.
In practice, undefined behavior means that your program is broken, but you can look at the compiled program and trace exactly what it does. Computers are machines, they behave predictably (for the most part). They fetch an instruction, execute the instruction, and read/store results to memory. If you look at the instruction and memory (the cache stored on the specific core that the processor is using), you can reliably predict what the CPU will do after executing that instruction.
And yes, multithreading exacerbates the effects undefined behavior and makes it more difficult to debug. But you can debug the issue, even if it means you have to look at the machine code directly.
So while, yes, a compiler can emit garbage when it encounters the example program you gave, using that as an argument for why undefined behavior is bad is dumb. Because in reality, compilers will do something that makes sense, and if it doesn't, you can just look at what it produced. In all the examples you listed, I'm sure that if you created a small example illustrating those situations, the compiled program would do what you expect it to. (Like in the case of having different default args, it will probably just throw an error like this[4]).
Sorry for the rant, it just annoys me that we can't discuss undefined behavior without somebody making an argument that doesn't matter in virtually every case. Undefined behavior is bad! But there are good, real, common examples that show why it's bad! Use those instead instead of talking about how compilers are technically permitted to spew trash when they encounter a program that looks normal, because I'd be very surprised if any modern compiler exists that does spew trash for a normal looking program.
This talk by Chandler Carruth seems pretty good on explaining the nuance of undefined behavior[5].
First of all, and most importantly, we're not talking about Undefined Behaviour, which happens at runtime, but about IFNDR (Ill-formed, No diagnostic required), which means at compile time your program has no meaning whatsoever because it's not a well-formed C++ program after all but your compiler doesn't tell you (and because of Rice's Theorem in many cases cannot possibly do so as it can't determine for sure itself)
This is a conscious choice that C++ made, and my belief is that once you make this choice you have an incentive to make it much worse over time e.g. in C++ 11, C++ 14, C++ 17, C++ 20, and now C++ 23. Specifically, you have an incentive to allow more and more dubious code under the same rule, since best case it works and worst case it's not your fault because if it doesn't work it was never actually a C++ program anyway...
Still, since you decided to talk about Undefined Behaviour, which is merely unconstrained misbehaviour at runtime, let's address that too.
For the concurrency case no, humans can't usefully reason about the behaviour of non-trivial programs which lack Sequential Consistency - which is extremely likely under Undefined Behaviour. It hurts your head to even think about it. Imagine watching an avant garde time travel movie, in which a dozen characters are portrayed by the same actor, the scenes aren't shown in any particular order and it's suggested that some of them might be dreams, or implanted memories. What actually happened? To who? And why? Maybe the screenwriters knew but you've no idea after watching their movie.
Today a huge proportion of software is processing arbitrary outside data, often as a service directly connected to the Internet. As a result, under Undefined Behaviour the unconstrained consequences may be dictated by hostile third parties. UB results in things like Remote Code Execution, and so yes, if they wanted to the people attacking your system certainly could turn it into a Pac man game.
We should strive to engineer correct software. That we're still here in 2023 with people arguing that it's basically fine if their software is not only incorrect, but their tools deliberately can't tell because that was easier is lunacy.
> First of all, and most importantly, we're not talking about Undefined Behaviour, which happens at runtime, but about IFNDR
From the link I posted, from cpp reference, which gives a definition for what constitutes undefined behavior:
>> ill-formed, no diagnostic required - the program has semantic errors which may not be diagnosable in general case… The behavior is undefined if such program is executed
And as for the rest of your argument, you still haven’t answered the one question that matters. Is an executable file with machine code well-defined? If it is, then once you compile a C++ program, the generated binary is well-defined. And once again, all the superfluous arguments about what could happen are irrelevant when there’s no context provided. There’s lots of different types of undefined behavior. An integer overflow by itself does not make your program susceptible to remote code execution. It’s the fact that the integer overflow gets stored in a register that later gets used to write to invalid memory, and then that invalid memory has harmful code injected into it that gets executed.
We should strive to engineer correct software. I agree. It’s all this hand waving about how all undefined behavior is equal that irks me. Because it’s not true, as evidenced by the example listed above about the program that’s “technically” ill-formed C++, but in practice compiles to a harmless program. Engineering requires an understanding of what could go wrong and why. Blindly fretting about all undefined behavior is useless and doesn’t lead to any sort of productive debates.
Engineering is all about understanding tradeoffs. Which is one of the reasons the C++ spec does not define everything in the first place. The fact that in 2023 people don’t understand that all decisions in software come with a tradeoff is lunacy. And once again, I agree that undefined behavior is bad, but to pretend that the existence of IFNDR means the whole language is unusable is silly. People use C++ everyday, and have been for almost 40 years. I’d like more productive conversations on how rust protects the user from the undefined behavior that makes an impact, not about how it doesn’t have IFNDR because that’s irrelevant to the conversation entirely.
> the one question that matters. Is an executable file with machine code well-defined?
While you've insisted that's somehow the one thing which matters I don't agree at all. Are programmers getting paid to produce "any executable file with machine code" ? No. Their employer wanted specific executables which do something in particular.
And there aren't "different types of undefined behavior" there's just one Undefined Behaviour. Maybe you've confused it with unspecified behaviour ?
In the integer overflow case, because that's UB (in C++) it's very common for the result to be machine code which entirely elides parts which could only happen under overflow conditions. Because overflow is UB that elision didn't change the program meaning and yet it makes the code smaller so that's a win - it didn't mean anything before in this case and it still doesn't - but of course the effect may be very surprising to someone like you.
The excuse that "We've done it for decades therefore it can't be a bad idea" shouldn't pass the laugh test. Did you notice you can't buy new lead paint any more? Asbestos pads for ironing boards? Radium dial wristwatches? "That's a bad idea, we shouldn't do that any more" is normal and the reluctance from C++ programmers in particular shows that they're badly out of touch.
I feel like Rust finally broke the idea that programmers should be in complete control and completely conscious of everything the compiler is doing. It hasn't been that way in decades, compilers are freaking magic. But Rust undid a lot of that with borrowing. People became comfortable with the compiler knowing better than them. I just wish we could relax further: We should never be explicitly iterating forward over a collection unless we need this behavior for the algorithm. Things should be implicitly parallel. Etc. Give me rusty bash.
That seems entirely opposite to my Rust experience.
Rust is quite transparent in what it does, and is very conservative with compiler magic. The language doesn’t do heap allocations, doesn’t do reference counting, it doesn’t even have implicit numeric type conversion. It won’t implicitly copy types that did not ask to be implicitly copyable, and even that is only legal for types that can be copied with a simple shallow memcpy.
Rust uses zero-cost abstractions all over the place, which means it’s predictable what code they will compile to, and that will be typically something simple. Std types have well-known basic layout, so you know that e.g. iteration over a Vec is going to be a loop incrementing a pointer, and there’s no implicit parallelism.
Describing borrowing as “compiler knowing better than the programmer” is a weird way of looking at it. Borrowing is like type checking. You declare a type to be temporary, and try to use it as long-lived, you get an error. It’s the same as if you declare function to return struct Foo, but returned struct Bar instead. Yes, compiler “knows better than you”, because you just wrote a bug.
Borrowing still compiles to direct pointer usage without a GC (it’s literally guaranteed to be identical to a C pointer in C ABI structs and functions), and you can override lifetimes with unsafe if you think know better than the compiler.
> We should never be explicitly iterating forward over a collection unless we need this behavior for the algorithm. Things should be implicitly parallel
There is already a crate providing parallel iterators. You just rename the iter() call and that's it. I don't agree it should be implicit though.
I think "not in control of what the compiler is doing" is overstating things a little bit. In some ways, Rust gives the programmer more control than C does. For example, Rust has standardized support for inline assembly, but inline assembly in C relies on vendor-specific extensions.
But to your point, the convenient defaults are very different. Unsafe typecasts require a lot of ceremony and careful thought in Rust, and they have to follow more rules than in C. In particular, references are all effectively "restrict" in Rust, and it's really easy to screw that up when you do unsafe cast from raw pointers to safe references, which is a big incentive not to write code like that if you have a choice.
I should've said "stabilized" instead of "standardized" to avoid stepping on this conversational landmine. But an important practical difference is that Rust supports inline assembly on Windows. (Correct me if I'm wrong, but I think MSVC mostly does not support inline asm.)
As someone who does a lot of unsafe rust, including tagged pointer foo, I strongly disagree.
If anything I want more explicit control, alleviated with an even more expressive type system. Ideally rusts type system would just be a prolog variant imho.
You're skipping the most needed improvement of all: fixing the issue where scripts with unquoted variables will seem to work until they contain a space. Almost every single bash script of more than a dozen lines I've seen outside of large open-source projects fails if a user puts a space in a file path, because the programmer didn't understand the insanity of variable quoting rules! I don't know of any programming language with a more common major footgun.
We use a Kotlin scripting variant for our own shell scripting needs at Hydraulic, it's pretty nice because we've joined it with a lot of internal APIs that make working with the filing system and network easy. It fits all those requirements and more (you can declare flags easily at the top level, it has built in progress tracking for slow operations etc).
The Kotlin type system is comparable to Rust without the borrow checker. It has non-null types, generics, etc.
It's not really a "product" per se but there's an old version for download and some lightweight docs here:
We've never worked out what to do with it. Ideas and feedback welcome. It's pretty nice to use, albeit you need to use IntelliJ to edit scripts if you want IDE features.
Now you have me curious - where do you see the need for floats in shell? You are more creative than myself, because I am struggling to come up with a situation where I would lean on this. A "path" type would be by far more useful to my work.
I have a script which uses wmctrl to tile windows when I press a shortcut key (for when I'm using an editor and want the editor window to take up most of the screen).
I guess I could avoid floats using multiplication, but I would rather keep the code as simple as possible.
It has unpredictable performance because of lazy evaluation. Other high-level functional programming languages like OCaml, Standard ML, and Scheme can be compiled and achieve fairly high performance.
Learning when to invest in type constraints and when not to is an important lesson. It’s not unique to rust, though it might express a little differently. I’ve dealt with excessively typed c++ and excessively abstracted and typed Java and they have the same class of refactoring problems. I’ve also dealt with plenty of undertyped and under documented go, where there are specific values all over the place which turn into runtime footguns - and these can be truly dire to refactor as well, you get an earlier sense of progress but you ship bugs to users most of the time. There’s no magic answer to this set of trade offs. Rust gives you tools to mostly pick your place, on this specific axis it provides an unusually broad choice.
"Rust screams at you all day, every day, often about things that you would have considered perfectly normal in another life."
A good C compiler does this when you turn on all the flags. I like languages/compilers that let you selectively disable the screaming and let you write bad code on purpose. Bad code that works but can be written fast is often better than perfect code that takes forever to write. Once you have a bad but working POC, you can make it less bad.
"It’s got no problem attracting new users, but it’s not resulting in dramatically improved libraries or tools. It’s resulting in one-off forks that handle specific use cases."
Age has nothing to do with that. Attracting core devs is hard, and putting lots of effort into making it attractive is necessary. On top of that, cultural conventions are set by the early adopters, and a lack of convention is often just as sinful as a bad existing convention.
Take Python for example. They took a lackadaisical approach to development and runtime environments, and as a result there's 50 competing ways to develop or run a Python program. Their most-used package repository, PyPI, has been a mess for years. Nobody builds on top of existing packages, names make as much sense as a random word generator, the ecosystem is rife with malware, you can't even search for a package on the command line, etc etc. None of that is the language's fault, it's the community and core team's fault for sitting on the sidelines rather than leading. Culture matters more than the tech it's centered around.
(I'm not trying to pick on them, I just know their problems better. C has been around for a half century and its community never really put together half the solutions more modern languages did)
> A good C compiler does this when you turn on all the flags. I like languages/compilers that let you selectively disable the screaming and let you write bad code on purpose. Bad code that works but can be written fast is often better than perfect code that takes forever to write. Once you have a bad but working POC, you can make it less bad.
That doesn't really work. All unsafe lets you do is dereference pointers or call unsafe functions. That's not gonna speed your development up during prototyping.
You can instead wrap everything in Arc<Mutex<T>> and .clone() liberally, though.
Always find it funny that people think unsafe {} means that rust ignores everything in the block, it just enables 4-5 additional abilities that are well documented, it's still doing most of its safety checks.
It will work if you only use raw pointers everywhere, like it's C. Don't use slices or strings; just pass a pointer to the first element of an array, and either pass its length separately or provide a sentinel value (like C's null terminated strings). Navigate balanced search trees using aliasing, raw, mutable pointers. Etc. This person compares the translation of C to Rust, literally versus idiomatically: https://cliffle.com/p/dangerust/
There will probably be weird behavior, though, because Rust optimizes based on assumptions about boxes and references. For example, if you have 3 raw pointers to some object live at once, and you give some library function a mutable reference made from one of those pointers, the compiler will optimize assuming it has exclusive access to that object, and it may make incorrect assumptions.
> All unsafe lets you do is dereference pointers or call unsafe functions.
And all those let you do is drop lifetimes and wave goodbye to the borrow checker. You just have to be explicit about it.
(What I mean is, the borrow checker checks borrows, it checks references. In unsafe you can make a reference forget what it's referring to. Just change &'a T to &'static T, and then the borrow checker is doing nothing.)
"Just mark everything unsafe" seems to be the motto of many (most?) crate developers. There's so much "unsafe" in dependencies used by so many Rust programs. It's a timebomb waiting to go off.
I checked the six most recently published crates on crates.io (blablabla, nutp, tord, g2d, testpublishtesttest, hellochi, at the time of writing). Three of those (blablabla, testpublishtesttest, and hellochi) did some variation on printing `hello world`. g2d seems like an interesting graphics library. tord provides a data structure for transitive relations, which is also neat. No crate contained over 250 lines of code. Unsurprisingly, none of them contained unsafe code.
Elsewhere in this thread, it's been pointed out that as a consequence of crates.io having a global namespace, plus lax enforcement of an anti-squatting policy, there are a lot of namesquatting packages. Those presumably contain no unsafe code.
tokio contains unsafe code. rand contains unsafe code. regex contains unsafe code. time contains unsafe code. (method: a smattering of packages chosen from blessed.rs; result: every one that I checked except serde containing unsafe code; epistemic status: eh -- I grepped the codebases, ignoring things that were pretty clearly tests, but might have accidentally included some example code or something that's not part of the core library? Please let me know if I've misattributed unsafe usage to one of these projects, or if I've managed to select a biased sample!)
I'd certainly believe a straightforward reading of the claim "80% of crates have no unsafe code"...but that seems almost meaningless, given that a not-insignificant portion of crates contain basically no code at all? I'd be much more interested in a weighted percentage by downloads: I'd be wildly impressed if 80% of crate _downloads_ contained no unsafe code, and would be somewhat unsurprised if the number was well below 50% -- crates with more functionality would be more useful and therefore more download, but also more likely to use unsafe code, I'd imagine.
Edit: I just noticed crates.io has a most-downloaded list[0] -- I might end up running some numbers on top packages there tomorrow morning, for some more solid data.
What is the fact that many foundational ecosystem crate contain unsafe code supposed to prove? That's the entire point of the language. That someone writes a really good regex crate once and then the rest of us don't have to write unsafe to use it. It seems like you have a fundamental misunderstanding about the goals and purpose of rust.
Libraries safely wrapping unsafe code in safe interfaces, and everyone reusing those safe interfaces is like, the whole point of…reusable libraries???
Also, you’re replying to someone who I’m fairly sure is on one of the core Rust teams, if not closely involved, I’m somewhat more inclined to trust _them_ when they say 80% of libs don’t contain unsafe (given that it cleanly meshes with my own experience of Rust libraries).
Instead of looking at the crates themselves, you might want to check your (or others') Rust application with https://github.com/rust-secure-code/cargo-geiger to get a sense of effective prevalence. I also dispute that the presence of unsafe somewhere in the dependency tree is an issue in itself, but that's a different discussion that many more had in other sub-threads.
No, 1/5 of Rust code contains unsafe directly. Also, "the" point (in the area of safe/unsafe) is to manage unsafety and provide safe abstractions from unsafe foundations. If you go deep enough there's always something which will be unsafe (the type system would have to able to proof the whole universe otherwise), but most programmers will not have to write such code themselves (no, most of use do not write double-linked lists each day) - they can just use it (e.g. by using the standard library). And if they have to write unsafe code the unsafe parts are restricted to certain areas of their code. Areas they can then invest extra time and care to make certain their assumptions hold true.
At the risk of getting downvoted into oblivion, that sounds a lot like TypeScript (compiler yelling at you to fix things but you just want to try an idea without perfecting it).
I'm glad to see there's a wide range of opinions here.
I'm not too concerned with performance or safety (since my code hasn't been seen very often).
I use rust just because it has a lot of "libraries that help me develop".
While other language libraries have a poor search experience and ranking system, Rust has a great library search system.
I use it because I don't have to read blog posts like C/C++ or java to find libraries that help me develop, or wade through unnecessary libraries like C# or go, I can use the rankings, and it has a good documentation system.
This has mostly been my experience too. I like using Rust because it reduces the cognitive load I have to deal with for many things (performance/safety are nice, but not a priority).
As you've said, finding great libraries is easy, and so is adding/building them. I find this a nice change from e.g. Python (where there are so many way of different competing ways dealing with packaging/dependencies).
Also thanks to the error handling, sum types and the like, I don't have to worry so much that I've deal with all errors, handled all enum variants, etc., the compiler takes care of that.
Some days I have misgivings, but not about the language. About the crate situation. Too many important low-level crates stuck at 0.x. I've previously written about problems with the high-performance 3D graphics libraries, but only game devs care about those.
At the language level, the big problem is back references. Sometimes you do need them, and the only safe way to do them at present involves reference counts in the forward direction and weak references in the back direction. Then you have to call .borrow() and .upgrade() too much. I'd love to see a static analysis solution to that.
(Rough outline of such a solution: Owning object belongs to Owner trait. Owned object belongs to Owned trait. Owned object has .owner() and .owner_mut() functions which retrieve references to the owner.
Owners probably have to be pinned, so they can't move while a back-reference exists. If an Owner changes a reference to an Owned object, the back reference is automatically updated.
That's the easy part. Now figure out how to prove by static analysis that a specific use of this does not violate Rust's no-aliasing rules (N read-only, or 1 mutable). This looks do-able for the non-mutable case, because having a non-mutable reference to both owned and owner is OK. Mutability, though, is tough. Anybody thinking about this?)
> Too many important low-level crates stuck at 0.x.
Is it fair to say that Rust failed to supply a robust set of standard libraries comparable to other modern languages? Or was the language aimed at level geared towards implementing rather than providing libraries? If it's truly a systems language, then what library features are essential, and what are 'nice to have'?
IMHO a lot of these 0.x libraries would be called 3.. In other languages
S.
They often have better quality than what I find in "mature" packages in npm or PyPi.
Rust kind of has a perfectionist touch to it which makes it really hard to say "this public API is stable". So people stay at 0.. way longer than normal.
I'm fine with that.
The lack of namespacing and heavy namesquatting is rather what's annoying.
There was a conscious decision to avoid providing a vast standard library because those inevitably become outdated with time. Building a standard library is relatively easy, maintaining for decades is hard.
But it’s a trade off because there are concrete upsides to a large standard library. I wrote about this more - Rust has a small standard library (and that’s ok) - https://blog.nindalf.com/posts/rust-stdlib/
That does seem to align with my perception of where Rust wants to be: a low-level systems language. For example, the stdlib provide a time package that deals with duration and instant, sufficient to interact with file systems and such, but leaves dealing with dates, times, and all that complexity to the chrono crate.
What I would encourage the Rust community to focus on is adding security-sensitive functionality to the standard library. Not the big wad of complexity that is cryptography, but a standard PRNG/RNG/CSPRNG would fit.
Also, I do think it's a bit of a mistake to not at least define a relational database API. As it is we have sqlite, mysql, and postgres with similar but slightly different APIs.
I don’t think low level languages mean you need a small standard library and high level means you need a large one. You’re pattern matching between Python, Java, Go (all large, older) and now Rust (small and newer).
Rust wants to be high level as well, allowing people to solve all kinds of problems of varying complexity in different domains. And to some extent, Rust has succeeded in empowering folks to do this. Rust has found its way into “systems” like browsers (Chromium, Firefox) and Operating Systems (Windows, Linux) but also web software, infrastructure (firecracker), developer tools (ruff, turbopack) and other domains.
Like I said, the request for a larger standard library is a reasonable one. But I don’t think it’s going to happen, because Rust seems to be gaining traction in multiple domains without it. By the numbers it looks like it’s growing 10x every 3 years (https://lib.rs/stats). In such an environment, the folks at the wheel would double down on their approach rather than fundamentally rethinking it.
Not that I know of. But few other languages have a borrow checker.
You can do this in Rust at run time, with Rc, Weak, and .upgrade().unwrap(). If none of the .unwrap() calls fails, you did it right. If you can prove the unwrap calls can't fail, you don't need the run time checking. So the goal is to check at compile time something you can now check at run time at higher cost. That's what Rust is all about.
Tokio isn't the fastest conceivable runtime. Tokio isn't the smallest conceivable runtime. Tokio isn't the simpilist conceivable runtime. Tokio does not port to all conceivable environments. Tokio isn't the async-std runtime.
And so, for some or all of these reasons, right or wrong, various Rust libraries wed themselves to runtimes other than Tokio.
You can thunk around these things, but it's miserable, yielding subtle, inscrutable code that you will not understand in six months when you have to maintain it. And the real fun begins when you need to handle Err, and all the abstraction leaks around Send and threads ruin your day; more thunking, Boxing errors (omfg) and even subtler and more inscrutable code.
I think what they were trying to say is that Tokio ended up being a general purpose async runtime so naturally it can't be the most optimal solution for every case. But because various libraries kind of force you into Tokio, it's very hard to use a runtime that's optimal for your specific problem. You'd have to give up on a lot of convenient libraries out there.
First, I have to wonder whether any of this can be legitimately elevated to the level of a "culture." Right now, according to GitHub "insights," tokio-rs has approximately 2.0 regular, every day contributors. And that's the most widely used Rust async runtime. Everything else is likely even more thin.
Second, Tokio has a lot of share because it was early and aggressive in actually delivering usable async documentation and code. I recall, years ago, reading Tokio documentation on how futures worked and grasping these concepts all the way from file descriptors one might epoll() in C, up to the Rust abstractions, and thinking "hell, I could write an executor from first principles based on this." Tokio earned the advocates it has, as oblivious to the real state of things as some of them might be.
I think the real problem is that async Rust is incomplete. As I said elsewhere, async Rust syntax is fine. The implications of async Rust exposed some papercuts in the language that have had to be dealt with since, but the core syntax is fine. I believe that can be attributed to the serious minded attention that the syntax received from people way, waaay up the cognition curve. The part that didn't get enough thought was async runtimes. In an ideal world, one would develop a Rust library that utilizes and/or implements asynchronous calls and transparently, flawlessly run on any correctly implemented runtime alongside any other number of libraries also utilizing and/or implementing async calls.
That is not the case, and the damage that's doing is severe. For every one person, such as myself, that will dare attempt to articulate this pain and, in the process, certainly revealing clear evidence of blatant ignorance, a thousand others just silently gave up.
Yes I think we agree, though I'd quibble about the async syntax; I use it but I have philosophical concerns about it. But I do think tokio is on the whole well implemented. But it is not a good thing for a single runtime to dominate the language like that.
I've at least thought-experimented with what it would take to write my own code agnostic enough that it could run on both e.g. tokio and e.g. monoio etc. and, well, it just can't happen. Even if you find neutral/unbundled implementations of locks, channels, utilities, to depend on instead, you end up stuck at: task spawning, and any kind of I/O. The former, to me, is a glaring absence from the language standard; async should not have gone out the door without support for it.
I can't argue too much on your view of the syntax. I employ the weasel word "fine" because my experience with it is that I have little to no trouble understanding and using async Rust syntax: I can read and write async Rust and conceptually grasp what is likely going on in the runtime. I have to allow that perhaps it isn't sufficient, and maybe even that this is a factor in the runtime problem.
But there are other, non-syntax issues, such as synchronization primitives, IO events, etc. that are clearly underspecified. No maybes about it.
> I am a massive proponent of test-driven development. I got used to testing in languages like Java and JavaScript. I started writing tests in Rust as I would in any other language but found that I was writing tests couldn’t fail. Once you get to the point where your tests can run – that is, where your Rust code compiles – Rust has accounted for so many errors that many common test cases become irrelevant.
I wonder what kind of tests the author was writing? The test cases I write for my code is for weird edge cases I detect during testing, like parameter over or underflows, correctness of parsing functions, etc. Things that do fail. I don't get why you'd write tests for trivial things that can't fail.
When writing in loosely typed languages, some folks aim for 100% line coverage, to make it more feasible to refactor down the line. Which then of course ends up being a lot of tests for fairly basic stuff that the type system could help you with...
I think such tests are pretty useless regardless of language. If you work on say a banking system testing whether getBankingStatement() returns something of type string is a waste of time. It just slows you down when you refactor its return value to a BankingStmt instance. While testing what happens in low memory conditions when a user simultaneously deposits and withdraws while another user issues a charge back against them is extremely useful.
line coverage but also branch coverage. Every safe navigation operator used in "defensive" coding, ex. `foo?.bar` is actually another branch that needs to be covered if that's the metric people are using. All of those tests are unnecessary in a language where the type is known at compile-time.
I never programmed in Rust, but I had enough experience in C programming to know that segmentation faults are annoying to debug. I heard Rust prevents memory management problems at compilation level, so it forces you to create safer program. Given this, I want to ask Rust programmers here: For people who have no experience in managing memory at coding stage (basically programmer who have no experience in C-like language), can such people appreciate what Rust aims to deliver?
For people who've never written C or C++, Rust's biggest selling point is usually performance. Porting code from Python to Rust for example often gives 10-100x speedups right off the bat. You could get the same speedups by porting to C too, but Rust lets you do that without giving up the memory safety and package management convenience that you're used to.
Even when you don't care about performance, another issue that comes up sometimes is keeping track of mutable state. If you've ever relied on bytes instead of bytearray or tuple instead of list* to guarantee that no mutation is happening in some Python code, you know what I'm talking about. Rust can give you a similar level of control over who gets to mutate what, without making you change the type of your data or pay the cost of copies. It's basically the const/non-const distinction from C, but much stricter. Another way of saying the same thing is that Rust gives you a lot of the legibility/correctness benefits that you'd expect from a functional programming language, but you get to code in the usual imperative style.
* And even then, that only prevents assignment to the elements of the tuple. You can still mutate an element internally if it's not also an immutable type.
>> Porting code from Python to Rust for example often gives 10-100x speedups right off the bat.
As long as you don't do a couple of things that, while easy to remember not to do once you know them, a programmer used to languages like Python and Javascript might be oblivious to. Things like:
* Forgetting to use buffered IO wrappers
* Using println!() for high-volume console printing rather than locking stdout and writing to it manually
I think so, I wrote an entire video game from scratch in Rust and my experience has been absolutely delightful. I have to manage nulls via Options, cast things explicitly, cannot do dumb stuff like modifying an array while iterating it, etcetera. It’s guiding my code in ways that I wasn’t able to myself, and eventually leading to essentially what is a bug-free game. Imagine!
The experience has been far more than just borrow checking, it’s opened up my mind to a lot of concepts that my previous trials with Java, JavaScript, and Python fully obscured in arcane ways. Rust’s compiler is wonderful, and once you get past the ergonomic struggles (which I did by just going through Advent of Code), you’re set.
I’m not sure. Looking through different Rust libraries, I tend to see three different styles, depending on previous programming experience. Each mimics the prior experience, with benefits.
* Everything is a struct with concrete types. This is closest to C. It benefits from the improved memory safety, without sacrificing speed.
* Everything is a Box<Rc<T>>. This is closest to dynamic garbage-collected languages, but with vastly improved performance.
* Everything is an “impl Trait”. This is closest to templated C++, but with much better ergonomics.
So, while I think I agree with your specific statement that the improved memory management may only be fully appreciated by those that cut their teeth on C’s segfaults and dangling pointers, I don’t think that’s the only benefit of Rust as a language.
Many garbage collectors (e.g. CPython’s) will use reference counting as part of the process. If a reference count hits zero, the object can be destructed, prolonging the time between mark-and-sweep passes.
The performance difference, though, is mainly in switching over to a compiled language in the first place.
Mostly to mimic the data structures allowed in languages where objects are held by reference. In Python, I could write a tree structure as `namedtuple(“node”, [“lhs”, “rhs”])`. If I tried to write a similar structure in Rust as `enum Node{Leaf, Branch(Node, Node)}`, the compiler rightfully complains that it would have an infinite size. But the indirection introduced by Box would let it be stored as `enum Node{Leaf, Branch(Box<Node>, Box<Node>)}`.
If you're used to e.g. OCaml, and the problem you're solving does not require avoiding GC, then Rust is a more cumbersome language for no real benefit.
If you've never used an ML family language then Rust might still be a breath of fresh air even if you don't care about memory management.
In my opinion, until you have spent a lot of time debugging C/C++/assembly issues (memory corruption, null pointer crashes, segfaults, build system problems, exception inception, etc), Rust would probably seem like a total waste of time.
I dunno if it's just me but coming from typescript and c# background it honestly wasn't that hard to grasp things, Maybe it's just me but it seems so much easier to grasp than C.
In that scenario Rust can be attractive because of its high performance compared to languages with a garbage collector. No matter what Java fans will tell you, any runtime with a GC will have worse overall performance for most large codebases than a compiled language with no GC. Sure, microbenchmarks might not look too bad, but complex software is a different matter.
Rust is great for services such as API endpoints, where low latency and high throughput at a low cost are more important that hot reload during development.
I heavily appreciate the simplification of the memory model. I was able to go from comfortable with typescript/js to comfortable with rust with essentially zero knowledge of manual memory allocation in just a few months.
disclaimer: I'm not a big rust programmer, but I appreciate it.
I think it's a matter of perspective/background. Depending what language you're coming from you might appreciate a lot of the more modern language features, or binary size, or performance. On the other hand, you might already be working with a pretty cutting edge language with a garbage collector or other and decide that Rust has some cool benefits but not worth the switch.
Side comment but I looked at the documentation, github repo and even code for wick and I still have zero idea what it does. And I am rust developper full time...
> It also looks like (soon) you’ll finally be able to configure global lints for a project. Until now, you had to hack your solution to keep lints consistent for projects. In Wick, we use a script to automatically update inline lint configurations for a few dozen crates.
It's trivial to configure global lints per crate. The author's problem comes down to having multiple crates per project. Which is something people do sometimes, and should be supported, but there's a big difference between the two interpretations. I would go so far as to say most Rust projects are only one crate (and so support "global" linter configuration)
I believe procedural macros always need to be defined in a separate crate, so at least all projects that use procedural macros will have multiple crates
Still, that's two global configurations to manage, where the original wording was easy to misunderstand as "you have to override linter warnings at every single line where they occur"
Do you need memory safety? Then why use Rust when you could use Java, JS, Python, etc? You can't "disable" memory safety in those languages.
Do you need bare metal performance? Then why use Rust when you could use C or C++, which have much larger ecosystems, platform support, more mature tooling, etc.
Do you need BOTH memory safety and baremetal performance at the same time? Then there really aren't many other options besides Rust.
But what I've started wondering lately is: are there really that many situations where you actually need both of those things at the same time? C++ tends to get misused a lot, but I feel like the same thing is happening with Rust.
If you're remotely sane, you'll always want memory safety. That's not really an optional property, because even in C/C++ compromising memory safety throws you straight into the land of nasal demons. The question is whether you want the compiler to ensure memory safety, or to do it completely on your own without language support.
Personally, I've never met anyone who can ensure memory safety in their C/C++ without onerous restrictions even more severe than those rust imposes, but maybe you're superhuman.
And saying c++ has more mature tooling seems insane too. Anyone who has had to mess with depedency trees using git submodules and cmake, conan, bazel, vcpkg, and meson would not call the tooling, oh and hunter too!, mature.
I'm willing to admit that is likely, could you enlighten me?
I've done professional c++ for a couple decades. My pipelines usually were conan with some dependencies that were not. The used clang tidy and format, san, cppcheck, coverity and tested release and debug builds with clang and gcc. Coverage was sometimes just clang. But sometimes gcc as well. Tests were usually google test. The team standard IDE was vscode.
Rust has rustfmt which is one standard. It has clippy which in addition to actual defects it forces commin idioms so code looks familiar. It has san and llvm coverage. It has miri. It supports aflop for fuzzing. And it has a single tool for package management and build. Edit: IDE is still vscode and it works great.
The only tooling missing for rust would be formal analysis, which is in work (and which isn't that great a story on c++ either) and gcc. A gcc front end is in work, and there is mrustc to generate c from rust, but yes, gcc front end support would be nice to get all those extra targets without the extra work.
A full 70% of security vulnerabilities are caused by memory safety issues. As professionals we need to get serious and have memory safety as a baseline requirement.
So we shouldn't use Rust at all, since Rust is not completely memory safe since it let's you disable the borrow checker. We should be serious as professionals and use safe languages like Java, JS, Python, etc.
But of course there are use cases where you need memory safety guarantees and bare metal performance. In those cases, sacrificing some memory safety by using Rust is an acceptable tradeoff I think.
> Rust is not completely memory safe since it let's you disable the borrow checker.
No, it doesn't. What's interesting isn't so much that random HN posters believe this sort of thing, because hey, who needs to know anything about a topic to post their opinion on a forum right? No, what's fascinating is that this applies to people like Herb Sutter in his "cpp2" language, here's Herb:
> I don’t like monolithic "unsafe" that turns off checking for all rules
Nobody does that. It's possible Herb knows that and is being deceitful but honestly I think it's more likely that without investigating at all Herb has just decided everybody else is an idiot...
I don't consider myself a Rust expert, but this feels like a semantic argument? You can use `unsafe` to erase lifetimes, right? I'm not saying it's a good practice, and it's not globally disabling the borrow checker or anything like that, but in theory it's possible to produce unbounded lifetimes that are incorrect. In practice it probably doesn't happen very often. Certainly less often than UB in C or C++
The exact same code, without the transmute (just returning x directly from an unsafe block), fails a type check related to borrowing. Specifically, the compiler can't see why (without a transmute) it should believe that the reference x now has a different lifetime 'b instead of 'a. Which is fair because it doesn't.
The transmute is you claiming it does, and since transmute is an unsafe function that's entirely on you to make sure you're right about that. So, lying has the expected effect.
The borrow checks aren't magic, they can't look into your soul - but they are running inside unsafe code too.
* Edited to make explicit that the alternative is still marked unsafe and yet doesn't work
All memory-safe languages are built on an unsafe foundation. The Java HotSpot VM is very unsafe C++. That's just how computers work.
You aren't meaningfully sacrificing memory safety by using Rust because the unsafe sections are clearly marked out. That's similar to unsafe C#, Python with ctypes, and many other memory-safe languages.
Marking code unsafe lets you write memory-safe code that the borrow checker can't verify. That increases the risk of bugs but it is not the same as disabling memory safety. It is a subtle distinction.
Don't forget safety from data races. Rust's borrow checking provides much stronger protection against those than java, go, c# etc. So we shouldn't use them either.
Beyond memory safety, Rust's other advantages over C++ are simply that it has a far better type system (by nature of ADTs), cleaner and more consistent syntax, cleaner tooling overall, and a far better module/modularity story.
That, and the borrowing/safety aspects of Rust also shake out into threading/concurrency, where the compiler does a pretty good job of forcing you to adopt better practices in regards to state sharing, locking, concurrency.
Cargo, though... I am coming around to thinking isn't great -- workspace support remains half-assed, especially... and crates.io is amateur-hour for reasons many people have pointed out elsewhere on this post. I have been tempted to switch my personal projects to bazel, with 3rd-party deps checked in/submoduled.
In addition to memory safety and performance, Rust offers something that, as far as I'm aware, no other mainstream languages offer: protection from data races.
And it offers those things along with a fantastic type system, and great tooling.
It's only mentioned off hand in one sentence at the end, but have people found that Rust is hard to hire for? In my experience it's relatively easy to filter for good candidates when you're hiring for Rust, whereas, just as a point of contrast, I've noticed it's more difficult to find frontend developers who are good at TypeScript (since a lot of frontend developers just use plain JavaScript).
I don't think it is unusually difficult to find Rust programmers. The challenge is finding Rust programmers who also have expertise in systems software development. Rust was designed to be a systems language but ironically it primarily seems to attract developers that do not have expertise in systems software.
This isn't necessarily a problem. C++ and Rust can coexist pretty well in practice, and Rust is a good entry point for learning how to write systems software.
Sadly I have found the flip side, too: as someone with a systems software interest/background (I wouldn't say expert), the # of jobs in the Rust ecosystem that are not glorified webdev/microservicing (or worse, crypto) is actually quite small.
So I guess I wouldn't be surprised that the applications are trending that way, too, as that is where growth is happening right now it seems.
That's the flip side of the same coin as your parent comment, I guess - because Rust is more accessible than C or C++, a lot of people have started using it for stuff that's not hardcore systems software, and as a result it's not as good a filter for that stuff as C and C++ are.
On the other hand, we use Rust on the backend of our web based saas product, and it serves as a great filter for quality developers for us.
As an outsider, I often hear about async Rust being less than ideal. Perhaps I don't understand, because I haven't dipped my toes in the water yet... but I do most of my work in Kotlin with Coroutines, and concurrency is everywhere in the UI. I can't imagine working in a language having a major deficit in this space.
Are there any efforts to overhaul or completely rethink this?
It’s not just that. There’s a few big problems with Rust async aside from the normal coloring problem that is inherent to async and not worth talking about.
* The async runtime and async functions are decomposed but tightly coupled. That means while you could swap out runtimes, a crate built against one runtime can’t generally be used with another unless explicitly designed to support multiple. I believe C++ has a similar problem but no other major language I’m aware of has this problem - that’s typically because there’s only a single runtime and it’s embedded in the language. Things like timers and I/O are not interoperable because the API you use has to be for the runtime you’re running under. I believe there’s work ongoing to try to remedy this although in practice I think that’s difficult (eg what if you’re using a crate relying on epoll-based APIs but the runtime is io_uring).
* async in traits. I believe that’s coming this year although the extra boxing it forces to make that work makes that not something 0-cost you can adopt in a super hot path.
* async requires pinned types which makes things very complex to manage and is a uniquely Rust concept (in fact I read a conceptually better alternative proposal for how to have solved the pin/unpin problem on HN not too long ago, but that ship has long sailed I fear).
* The borrow checker doesn’t know if your async function is running on a work stealing runtime or not which means there’s a lot more hoop jumping via unsafe if you want optimal performance.
* async functions are lazy and require polling before they do anything. That can be a bit surprising and require weird patterns.
Don’t get me wrong. The effing-mad crate is a fantastic demonstration of the power of algebraic effects to comprehensively solve coloring issues (async, failability, etc). But I think there’s stuff with runtime interop that’s also important. I don’t think anyone is yet seriously tackling improving the borrow checker for thread per core async.
I mean... that's not really the pain that people are referring to when they are struggling with async Rust, especially when you compare it other languages like JavaScript that also have a difference between async and regular functions.
Unison [1] has algebraic effects as a first-class feature. They call them "abilities" there. You can make async about as transparent/opaque as you want it. Docs on abilities in [2].
I don't know that anybody is running a company on it yet, no. They've got a couple big milestones coming down the pike, which might make it more attractive to people.
For me I am working on a side project in Rust and had very few issues, Rust felt quite straight forward up until the point where I had to mess around with async. For my domain I need to use a framework that expects you to set up a shared mutable state for the connection pooling in an opinionated way. Suddenly I encountered arc and mutex and very cryptic trait error messages for code doing way too much magic. All I wanted was to share a SQLite connection in an ugly way to get my mvp out but I was dealing with absolutely incomprehensible trait issues.
In general I’ve noticed that there are a lot of code generation modules that can generate very cryptic error messages if you stay clear of their happy paths.
You would run into the same issues if you tried to do it with threads and rust too.
Any kind of shared resource is a tedious experience, that's basically it's selling point.
It is frustrating becasue if you go look for help, people are always condescending about global shared resources like that but there really isn't a better way.
It feels like most of the hatred for rust async is for people that try to act like Tokio isn't async rust and that for some reason you should try to randomly avoid tokio for some reason.
Maybe because you want to keep the dependency tree under control and bringing in tokio suddenly adds fifty crates you never asked for. Every crate is a potential liability!
There's efforts to overhaul it, basically there's a lot of stuff that has to be more complicated (as you'd expect) if you want to add support for async somewhere, and understanding the error messages can be pretty mentally taxing when you get it wrong; and then there's a lot of cases where the async version of some pattern just doesn't work right now due to limitations in the compiler or the language itself, and hopefully it'll work someday, but in the meantime you need to use workarounds.
Here's a better explanation by somebody smarter than me about why Rust chooses what it does, what else you could choose and what the price is: https://without.boats/blog/why-async-rust/
tl;dr: You can have different (nicer to program) abstractions, but you can't have them in the same language you use to write firmware for a $10 electronic device, or Linux drivers, and we already have languages like Go and Javascript whereas we did not have a safer alternative to C++.
For typical applications I am learning nim, which has python syntax, c speed and size, and could be memory safe too.
I really feel nim deserves more love, for that you can balance coding-speed(write like python script), size , performance and security, and cal leverage c and c++ libraries without FFI, no other languages can have those at the same time.
I've tried it a few times and it has great features on paper but to use it gets in your way too much. I can spin up a C# dotnet project write and test the code 10 times faster than in Rust. It might not perform as fast but the hot code can be written in a small C library using code/runtime analysis tools to catch any memory safety issues.
Writing performance-sensitive code in C/C++ and calling it via interop used to be the way to go during .NET Framework days but since then has become a performance trap.
Especially for small methods, calling them through interop is a deoptimization because they cannot be inlined, and involve GC frame transition (which you can suppress) as well as an indirect jump and maybe interop stub unless you are statically linking the dependency into your AOT deployment. In case the arguments are not blittable to C - marshalling too. Of course, this is much, much faster than anything Java or Go can offer, but still a cost nonetheless.
It is also complicates the publishing process because you have to build both .NET and C parts and then package them together, considering the matrix of [win, linux, macos] x [x64, arm64], it turns into quite an unpleasant experience.
Instead, the recommended approach is just continuing to write C# code, except with pointer and/or ref based code. This is what CoreLib itself does for the most performance-sensitive bits[0]. Naturally, it intentionally looks ugly like in Rust, but you can easily fix it with a few extension methods[1].
Thanks, I haven't had to do dropping to C for a while as the improvement in performance of dotnet along with features like AOT, Span<t> etc close the gap enough for the domain I work in.
Good to know you can remain within the framework and get decent performance though with the unsafe pointers/refs. Would be interesting to see a good benchmark using only C# with latest features and Rust, although I cognisant of the fact there is more to it than pure performance (binary size, dependencies, GC etc).
Rust standard library is far more conservative when it comes to vectorization and auto-vectorization is far more fragile than people think, both links - they beat it in performance significantly ;)
Rust is great, but one thing I’d like to see is an interpreted, dynamic, less strict version of it that could be used for prototyping and gradually typed into compiling Rust code. In other words, a new programming language doing to Rust the reverse of what Mojo is trying to do to Python.
Biggest turn off for me was Rust's horrendous syntax. Dangling apostrophes, wrapping the hell out of things, etc. Just look at this Result<Arc<T, A>, Arc<dyn Any + Sync + Send, A>>. Absolutely horrendous.
That’s a shallow problem that goes away as you start using the language.
Lifetimes need something to stand out as an identifier. 'a is weird, but works. Whether it could use a different ASCII sigil is a bikeshed problem.
Types borrowed <> from C++ to look less weird to C++ programmers. But this again is just a surface level issue. Semantically, the wrapper types are incredibly useful. Having all nested types spelled out is convenient when reading code – you know what you're getting and what are the standard properties of it.
As much as I love rust. Its just shocking to learn these newer languages like go didn't take any learnings from jvm and dotnet world for something simple as dep/package management
I thought the likening of the strictness of the language and compiler errors and warnings to an emotionally abusive relationship was quite offensive.
I programmed in C and C++ for years, to compiler messages don't make me feel like I should be taking them personally. Sure beats having a runtime error any day.
I wonder if those with that attitude have come from a background where they used dynamically typed languages or non-compiled languages more, or it is something even people with a wider experience find particularly onerous with Rust?
This is written by someone who has obviously written a lot of Rust code. I like its balance. It feels fair and not fanboyish like these write ups often do.
Re: async, rust is a down to the metal language. IE library yes, runtime no. Async implementations are all either runtimes (Javascript), or libraries that implement a runtime (Python). Under the circumstances I think it's fair that rust has less than ideal async.
> I still like threads, but the I'm old and uncool.
Hey, I resemble that remark.
But I disagree with it. Green threads give you all the advantages of async, but with less of the hairs. In particular no special syntax or change of programming style is required. Yet underneath green threads and async just different styles of event driven I/O, so both run at similar speeds and excel at the same tasks. (Actually green threads should run faster, as storing state on a stack is generally faster than malloc.)
I have no idea why Rust abandoned green threads in favour of async. Actually, that's a partial lie - there have been far too many words wasted on explaining why. The problem is the reasons they give look to be an caused by design decisions they made in their implementation. The primary objection seems to be speed. The current async is indeed faster than their old green thread implementation. But that was caused by their choosing to avoid coloured code in their green threads (maybe they were copying Go?). Other objections were similarly to do with the implementation they threw away, not green threads themselves.
> Green threads give you all the advantages of async
They require more memory over stackless coroutines as it stores the callstack instead of changing a single state. They also allow for recursion, but its undelimited meaning you either 1) overrun the guard page and potentially write to another Green thread's stack by just declaring a large local variable 2) enable some form of stack-probing to address that (?) or 3) Support growable stacks which requires a GC to fixup pointes (isn't available in a systems lang).
> green threads should run faster, as storing state on a stack is generally faster than malloc.
Stackless coroutines explicit don't malloc on each call. You only allocate the intial state machine (stack in GreenThread terms).
> The primary objection seems to be speed
It's compatibility. No way to properly set the stack-size at compile time for various platforms. No way to setup guard pages in a construct that's language-level so should support being used without an OS (i.e. embedded, wasm, kernel). The current async using stackless coroutines 1) knows the size upfront due to being a compiler-generated StateMachine 2) disallows recursion (as that's a recursive StateMachine type, so users must dynamically allocate those however appropriate) which works for all targets.
> They require more memory over stackless coroutines as it stores the callstack instead of changing a single state.
True, but in exchange you don't have to fight the borrow checker because things are being moved from the stack. And the memory is bounded by the number of connections you are serving. The overheads imposed by each connection (TCP Windows, TLS state, disk I/O buffers) are likely larger than the memory allocated to the stack. In practice on the machines likely to be serving 1000's of connections, it's not going to be a concern. Just do the arithmetic. If you allowed a generous 64KB for the stack, and were serving 16K connections, it's 1GB of RAM. A Raspberry PI could handle that, if it wasn't crushed by the 16K TCP connections.
> They also allow for recursion, but its undelimited meaning you either 1) overrun the guard page and potentially write to another Green thread's stack by just declaring a large local variable 2) enable some form of stack-probing to address that (?) or 3) Support growable stacks which requires a GC to fixup pointes (isn't available in a systems lang).
All true, but also true for the main stack. Linux solved it by using 1MB guard area. On other OS's gcc generates probes if the frame size exceeds the size of the guard area. Lets say the guard area is 16KB. Yes, that means any function having than 16KB of locals needs probes - but no function below that does. Which in practice means they are rarely generated. Where they are generated, the function will likely be running for a long time anyway because it takes a while to fill 16KB with data, so the relative impact is minimal. gcc allows you to turn such probes off for embedded applications - but anybody allocating 16KB on the stack in an embedded deserves what they get.
And again the reality is a machine that's serving 1000's of connections is going to be 64bit, and on a 64bit machine address space is effectively free so 1MB guard gaps, or even 1GB gaps aren't a problem.
> No way to properly set the stack-size at compile time for various platforms.
Yet, somehow Rust manages that for it's main stack. How does it manage that? Actually I know how - it doesn't. It just uses whatever the OS gives it. On Windows that's 1MB. 1000 1MB stacks is 1GB. That's 1GB of address range, not memory. Again, not a big deal on a modern server. On embedded systems memory is more constrained, of course. But on embedded systems the programmer expects to be responsible for the stack size and position. So it's unlikely to be a significant problem in the real world. But if does become a problem because your program is serving 10 of 100's of concurrent connections, I don't think many programmers would consider fine tuning the stack size to be a significant burden.
> No way to setup guard pages in a construct that's language-level so should support being used without an OS (i.e. embedded, wasm, kernel).
There is no way to set up the main stack without the kernel's help, and yet that isn't a problem? That aside are you really saying replacing a malloc() with mmap() with the right flags is beyond the ken of the Rust run time library authors? Because that is all it takes. I don't believe it.
> The current async using stackless coroutines 1) knows the size upfront due to being a compiler-generated StateMachine 2) disallows recursion (as that's a recursive StateMachine type, so users must dynamically allocate those however appropriate) which works for all targets.
All true. You can achieve a lot by moving the burden to the programmer. I say the squawks you see about async show that burden is considerable. Which would be fine I guess, if there was a large win in speed, or run time safety. But there isn't. The win is mainly saving on some address space for guard pages, for applications that typically run on 64bit machines where that address space address space is effectively an unlimited resource.
The funny thing is, as an embedded programmer myself who has fought for memory I can see the attraction of async being more frugal than green threads. A compiler that can do the static analysis to calculate the stack size a number of nested calls would use, set the required memory aside and then general code that so all the functions use it instead of the stack sounds like it could be really useful. It certainly sounds like an impressive technical achievement. But it's also true I've never had it before, and I've survived. And I struggle to see it being worth the additional effort it imposes outside of that environment.
Javascript to Rust is a big step for sure. It's probably a less painful journey if you already have a background in Typescript/C++. This article has good feedback though and I hope the language evolves to address some of it in the future.
> It also looks like (soon) you’ll finally be able to configure global lints for a project. Until now, you had to hack your solution to keep lints consistent for projects. In Wick, we use a script to automatically update inline lint configurations for a few dozen crates. It took years for the Rust community to land on a solution for this, which brings us to…
Wow, as the author of that feature, I'm surprised to see someone was so passionate about it. I've found that many times I've been having to tell people why they should care about it.
> I don’t know why. Maybe the pressure to maintain stable APIs, along with Rust’s granular type system, makes it difficult for library owners to iterate. It’s hard to accept a minor change if it would result in a major version bump.
There is a tension between people wanting features and people not wanting to deal with version bumps. I've seen this a lot in maintaining clap, especially when it went from unmaintained for years to having active releases.
As for cargo, the compatibility guarantees are tough. Take the lints table. We can't throw in a first try, knowing we can fix in in a cargo 2.0. We are tied into the rust project itself which means we have the same compatibility guarantees. This is one reason we generally encourage trying ideas out in third-party plugins before we integrate them in directly since they can break compatibility.
> You can’t even publish a crate that has local dev dependencies
You can; cargo automatically strips them. However, if you tell cargo that there is a version of it in the registry (by setting the version), then it must be published. This is why when I redesigned `cargo add` for being merged into cargo, I made it so `cargo add --path ../foo --dev` will not add the `version` field. We do need to find ways to clarify that the purpose of the version field is for looking it up in the registry.
Allowing the dev dependencies to be stripped also helps with issues of circular dev-dependencies.
> However, many developers break large projects down into smaller modules naturally, and you can’t publish a parent crate that has sub-crates that only exist within itself.
The most complex part is the Index, figuring out how to represent it in the metadata tables we maintain so we avoid having to download every `.crate` file.
I also worry there might be tech debt around assumptions of there being a single version of a package when nested packages will easily break that.
> You can see the problem manifest in the sheer number of utility crates designed to simplify publishing workspaces. Each works with a subset of configurations, and the “one true way” of setting workspaces up still eludes me. When I publish Wick, it’s frequently an hour+ of effort combining manual, repetitive tasks with tools that only partially work.
I'm a bit confused on this point. While there are things to improve around publishing workspaces, I'm not sure how this relates to setting workspaces up or what problems they've had with that. I'd also be curious what problems they had with releasing packages. I don't think I've seen issues from them in cargo-release's Issues.
One thing I don't appreciate is nonorthogonality of certain uses and control structures that cannot be refactored into a function despite structural equivalence without introducing a multiple borrow conflict.
I think Rust is one of those excellent tools where it does well to interface into a high-level language like python.
Need performance, security, and reliability? Build that part in Rust, and have the rest be executed and orchestrated in python. It also forces great design patterns in the form of encapsulation and a strong API.
I think the idea of being monolingual in codebases is a silly limitation and lots of development teams would be a lot more productive if they embraced the idea of polylingual codebases.
Rust binaries are by default nowhere close to 500MB. If they are not small enough for you, you can try https://github.com/johnthagen/min-sized-rust. By avoiding the formatting machinery and using `panic_immediate_abort` you can get about the size of C binaries.
Comment is based on the rust-std package I'm currently seeing in VoidLinux. It is over 500MB.
3 packages will be downloaded:
3 packages will be installed:
libexecinfo-1.1_3
libexecinfo-devel-1.1_3
rust-std-1.73.0_1
Size to download: 136MB
Size required on disk: 509MB
Space available on disk: never enough!
Cheacking Debian and NetBSD it seems like the Rust standard library is smaller, much less than 500MB.
The simple question is how much space is required for an installation of the Rust compiler. On VoidLinux, it's significantly more space than for a GCC installation. 509MB is just too much for my tastes.
I definitely agree with the first negative point. Rust libraries often feel weirdly restrictive and unfinished, even after having a lot of time to mature. Somehow they just don't end up feeling that much better. I just don't understand why though, is there something about Rust that makes this a problem?
Is Rust a replacement for only C/C++ or also Java/Python/C#/TS? In the latter, I already get some memory safety which I believe is one of the biggest selling points of Rust. Are other benefits of language strong enough that we look into it to develop projects traditionally built in Java or Node?
>I also now get irrationally upset when I experience a runtime error.
Had this experience in my first Rust program. It turned out Rust had silently changed the types of some variables based on some code hundreds of lines later, which caused an overflow at runtime. Spooky action at a distance!
Can’t really accuse programmers of being creative with words and phrases. Most of them will be rehashes of blogs and memes from the last couple of decades.
It really depends on what your previous programming experience is, if you can state which languages you know now, someone can give a better response to this.
This made me not want to try Rust. I was coding vanilla JavaScript for a very long time before trying TypeScript but when I did I was amazed at how much easier refactoring became. That alone made me very excited about type systems and I instantly started looking at Rust, thinking it would be even better. Apparently not. Maybe the sweet spot (for me) is TypeScript.
In what way? Rust (cargo technically) might have the best support for offline compilation of any language I use. `cargo vendor` does pretty much everything I want it to.
Also you can download documentation for offline using `cargo doc` so you can easily lookup stuff during a flight. Although you do need to figure out all your dependencies ahead of time so that you download their documentation before you leave.
Just FYI, `cargo doc` doesn't download the documentation. It re-generates the documentation from source. You only need the source code to generate documentation.
To clarify, they aren't disabling the lints in the section you've highlighted. `deny` makes a lint a failure instead of a warning. This is similar to `-Werror` in C. `allow` is the directive to disable a lint. Eg, I often start new or disposable projects with `#![allow(unused, dead_code)]` just to keep my IDE clean while the code is in an unpolished state.
Rust offers you great control when you need it (to the same level of C++, basically. This is what it was designed for). And even the Carbon docs say that if you start from scratch prefer languages like Rust.
No it doesn’t. The vast majority of what is taught in a college C++ course would be either impossible or extremely difficult to do in rust. You couldn’t implement std::tuple or std::variant for instance.
The carbon docs never recommend rust, they just urge you to use something already available.
"Programming in Rust is like being in an emotionally abusive relationship. Rust screams at you all day, every day, often about things that you would have considered perfectly normal in another life. Eventually, you get used to the tantrums. They become routine. You learn to walk the tightrope to avoid triggering the compiler’s temper. And just like in real life, those behavior changes stick with you forever."
This is a pretty negative and unfortunate take on rust that I cannot recognize after having spent the past 2 years professionally writing backend rust.
I did write c++ before that so it's likely that it takes some experience with the pain of an unhelpful (and in some cases downright hostile) to fully understand and appreciate the so called "yelling" rust does.
From my perspective every lifetime complaint the compiler has is a deeply appreciated hint that not only saves you countless hours of debugging down the line but also makes me feel confident/safe in the code which you never could achieve in a c++ codebase without making defensive copies everywhere.
Being able to lend/borrow data without fear truely is rust's greatest strength and something people coming from GC languages have a hard time appreciating.
It's a bit of an oversimplification, but a little quip I've used when talking about developing in Rust vs. other languages is, it's a question of where/when do you want the pain. In Rust, it's at development time (and hiring and ramp-up time); in C++ it's at runtime; and with GC languages it's at billing time when you have to pay for that extra compute and RAM. There's no way to get rid of the pain entirely.
Sometimes billing time is your customer's patience, their phone battery, or other things like that which make your product worse and give your competitors an edge.
>...which make your product worse and give your competitors an edge
I would find that argument passable if some of the biggest products from the largest companies functioned better. Performant software seems like a distant memory.
And yet the big tech companies are perpetually re-writing their whole stack, or going out of their way to create new compilers for their language in order to lower their billing time.
Salaries are dominant in early stage startups, and infrastructure costs becomes dominant as you scale (also probably a SaaS-centric oversimplification). You can get into trouble in the middle, where you have enough scale to be paying jaw dropping cloud bills, but you don't have the staff to move to a cheaper architecture and you may still need to focus on getting new features out to drive growth (or to attract investment).
In startups that chasm is generally filled with VC money paid to cloud vendors, but if you're bootstrapping or not looking to be a unicorn, you probably will want to drive down infrastructure costs much earlier.
I tend to agree with you, but your aside about it being Saas-centric is really important.
One need only open Microsoft Teams to see the “billing time” costs. If the billing time cost is not paid by the developers or their company, then it stop being a cost to them and it becomes an unpriced externality. Who cares, who even knows, that this app we’re writing runs slowly on regular people’s old hardware.
Yes but there’s a difference in who feels the pain. Compile time pain is felt by devs, who have the constitution and ability to fix such issues. Runtime pain is felt by users and can result in data loss and security issues in the wild, and is harder to debug.
One kind of pain has a bigger blast radius than the other. I prefer compile time pain to runtime pain.
>From my perspective every lifetime complaint the compiler has is a deeply appreciated hint that not only saves you countless hours of debugging down the line but also makes me feel confident/safe in the code
Meh, the fact that Rust's compiler is overly strict is just a truth (All compiling programs are valid, but not all valid programs are compiling), and I find it lowkey annoying that every time someone has a problem with it, rustceans jump to the defense to the point of almost denying the OPs experience.
I mean on a case by case basis it might simply be a difference of experiences and both are valid, but on a larger scale it does look like a bit of a pattern, at least to me.
Ofcourse, there restrictions aren't just a whim of a Rust core team (usually ;p ), and do come from practical limitations but regardless, it's fair to be frustrated at them.
Indeed, I’ve been writing rust professionally since 2015 (yes before 1.0) and I would consider the first encounter with the compiler as maybe it yelling at me, because I was used to C++. But it actually taught me a lot about why the code I was writing may have seemed okay at first glance, but actually introduced a tricky corner case bug that I hadn’t considered. The borrow checker got it tho, and I had to rearchitect my code to avoid that edge case. Now the code is better, free of that bug, and I won’t make that mistake in the future.
Was that the result of a process in which the compiler “abused” me? I don’t think so.
Its wild how different the perspective is depending on your background
The author also wrote
>[Rust's] dev tooling leaves much to be desired
I can't even begin to comprehend such a statement. Rust's tooling ecosystem is the number one reason why I want to convince my colleagues to give it a shot. Its literal heaven coming from C++ with cmake...
In general, the error messages are also nice and offer suggestions on how to fix the problem. Far from being yelled at, it's more akin to pair programming. "Try doing this..." "Did you mean to do this?"
Did you mean non-GC languages? GC languages by default have at least memory safety because the GC frees for you when an object is detected to no longer be accessible.
I continue to be delighted by indications that real high level languages compiling to Webassembly are supplanting the godforsaken nightmare of JavaScript. I think anyone focusing solely on JS is overdue for looking at those languages and getting up to speed on the huge differences before they wake up to find that the market for making and fixing kerosene lamps has been replaced by fusion reactors.
I need to use ChatGPT as a programming sidekick/pair programmer/mentor. Yes, I am not afraid to admit - I need an AI assistant and won't program without it - AI is far too valuable.
If ChatGPT struggles with a programming language that is a big problem.
I have had a great experience with ChatGPT with TypeScript, SQL, Python, golang, C++.
I tried ChatGPT with Rust and it essentially failed - it does not understand the language.
AI has become an essential tool for me in programming if the language doesn't do AI or doesn't do it well then that's a huge mark against it.
I'm not trying to shame you, but how long have you been coding? I've not yet had the urge to ask an AI for coding help, ever. The questions that are decoupled enough from a field/project that it might be able to answer seem trivial and easy just to search on.
Plus, I'm way more trusting of a Stack Overflow post or even a blog post than what AI generates. I mean, AI hallucinates all the time or generates things that are subtly wrong, so unless you can write the code yourself how can you trust the generated stuff?
I'm pretty shocked by the grandparent, but on reflection, I think this is the future.
In The Grapes of Wrath, Steinbeck writes about the travails of a family's trip to California in search of work during the Great Depression in the 1930s. Tom Joad, the father, fixes the compression in his blown engine by wrapping a copper wire around the cylinder, then running the motor until it melts and recreates the seal. It's such an ingenious hack, and reflects such deep knowledge of how an engine works, that it's stuck with me for decades.
Today, I bet fewer than 5% of vehicle owners even know how to check their tire pressure. But it's not like fewer people are driving, or cars are more likely to be found by the side of the road with a blown engine—it's just far less necessary to know these things. I think programming will go this way, too.
steel melts around 1600, the engine would blow up waaay before.
the problem with that story is that if the "seal" is made from a simple material that melts by the engine heat, then it will not seal for long. (or at all.) but likely it's not what Steinbeck wrote.
At what temperature does copper soften enough to squish into a seal shape?
I seem to remember my dad shimming a cylinder that had lost compression with a copper ring; I was a kid so I don't remember any details except it wasn't intended to melt, and whatever it was intended to do worked and we saw the car still being driven around town fifteen or twenty years after he sold it.
piston ring seals are usually made of cast iron or steel. copper would probably work too for a while, it's good at conducting heat, there's ample cooling in engines, so it wouldn't melt, just wear out very quickly ... and then the engine performance degrades as the sealing gets worse and worse (and it starts to eat oil, soot gets everywhere, exhaust becomes visible), mpg goes down, but ... the car would probably run. (loss of 1 out of 4 pistons is not a catastrophic failure)
I've been coding since I was 8, so... 30 years, about 17 professionally.
I'm not averse to new tools at all, but I've yet to see where I would want copilot or chatgpt. The problems I work on daily, like most professional devs, are very specific to how I adapt a large proprietary codebase to do new things to fit specific business requirements, or working with designers and project managers to figure out how we should solve these intricate problems together. AI can't help with that. It'd take a 20 page prompt for it to roughly understand the business even.
It can help with toy problems like how to write a well known algorithm, but given that these are well known algorithms and it's basically just copying from open source repo's (with scary legal ambiguity), it's of little use to what I actually do day to day.
I'm amused that as a lead you'd "give me a warning ". I've literally never worked with a lead or manager that cares what tools people use, only that the work is good. Actually, if anything a lot of workplaces are asking people to seek approval before using AI, because of the many thorny issues (copyright being one of many)
(BTW, I do use chat gpt, just not for coding. It's useful for creative tasks or summaries. I still google what it says because it does make shit up)
> If I was your team lead I'd give you a warning, same as I would if you didn't use an IDE or source control.
I really think you mix up
a) common rules as they are needed to work together and (like style or source control)
b) local tooling / someone is organizing his own workspace
including an dictator-like "MY way of working is the best!" attitude after disovering whatever works out for YOU. If I was your team lead, I guess I would give you a warning. :-D
Additionally, I guess your programming problems are very generic.
I tried for a long time for chatgpt to help me write a simple function that generated a random array and then summed along the columns. Could not do it.
Pasting in an error message, made chatgpt spit out another version of the code with some other error.
This is just horrendous. So instead of writing this trivial piece of code yourself you get an AI to write it for you. Then instead of reading and understanding the code you just run it to see what happens. Is there a problem? Just try again, or immediately grab a debugger, because everyone knows that nobody could possibly read and understand a trivial two-line function!
Yes, there's something wrong here. But it isn't "not using chatgpt"...
Of course that works, but that's not exactly a task that's likely to trip me up or be a bottleneck in my coding efficiency. The things that are are typically either of sufficient complexity i don't even know how to begin describing it to ChatGPT in a useful way, or require enough domain knowledge it's not worth the effort.
Of course, it's still an issue if ChatGPT can't generate trivial rust code, but that seems unrelated to the quality of the language itself.
I know rust gives memory safety and how important that is, but the ergonomic is really bad. Every time I write some rust I feel limited. I always have to search libraries and how to do things. I cannot just "type the code".
Also the type system can get out of control, it can be very hard to actually know what method you can call on a struct.
I still think rust is a great tool, and that it solves tons of problem. But I do not think it is a good general purpose language.