Hacker News new | past | comments | ask | show | jobs | submit login
Carefully exploring Rust as a Python developer (karimjedda.com)
159 points by EntICOnc on Nov 13, 2022 | hide | past | favorite | 70 comments



Hey everyone, author here, happy you're liking my article. I have gotten a lot of feedback on this since I posted it. If there's anything I can add or update or even improve, let me know. I've been using Rust more and more every day and if I can motivate even just one person to give it a try then I'm satisfied. :) Have a great Sunday!


Maybe in a future article you might tackle, “what I learned from Rust dev that I successfully transferred back to Python.” =)


If you are a "python developer" which in my book means you default to Python for any project and try your best to avoid any other language even bash and js(/typescript), Rust is probably not the best thing for you.

I am a Python developer. This is pretty much what defines me. Being a general purpose developer with familiarity to one langauge is what helped make me my living until recently. Now, being a devrel I need to be familiar with a BUNCH of languages at least on the syntax level for our SDKs.

The third/fourth language for a Python developer the default suggestions are Go, Rust, C#, Java/Scala etc....it varies but there is a pattern. A low level language that frequently makes it to HN front page or something that your supervisor wants to explore because they say it in HN front page.

Even though we are a Go shop and Rust is hyped as hyped can be, I don't consider either to be as fun as Python. The syntax and logic feels absolutely weird. Python gets you up and running in no time. But these language just doesn't feel write. Too much verbosity, too much diy, too much etiquette. The things that make these language loved to actual "SWE" makes me hate them, as I am not a "SWE", I just love solving problems programmatically even though it inherently makes me a bad programmer but I am a passable Python Developer and I can live with that.

Then after much doubt and much imposter syndrome-ness, I found it, the perfect third/fourth language for a Python developer, Nim. The tag line itself says all, "zero dependency executables". It is closest thing to feel like Python while it keeps everything elegant (in SWE terms, I guess). This language is a godsend if you are making custom tooling.

Give it a shot, if you made it through this rant.


I suppose there's something to be said for "feel" and surface syntax, especially for novice coders dealing with smaller-scale problems. The C-like syntax of something like Rust is quite elegant and popular, but it might not be for everyone.


> I recommend VSCode if you're starting out as a few extensions that could help a lot and guides on how to use those extensions are very accessible. I recommend rust-analyzer as the only extension for VSCode.

Minor nit: these two sentences seem to contradict each other.


Loved the “shallow dive” approach, it makes Rust look a lot more approachable


Agreed, this is great


> In this case, we'd add thiserror = "1.0" right after [dependencies] and we're in business.

You can use `cargo add` to add dependencies, it's part of the standard set: https://doc.rust-lang.org/cargo/commands/cargo-add.html

> Writing tests

This section is very odd, I don't think I've ever seen tests set up that way (but I might be wrong).

Normally, you have:

- a tests submodule to unit-test internal behaviour (gated with cfg(test))

- a top-level `tests` subdirectory, which contains integration tests leveraging the public API of the crate

The dead_code strictures are also unnatural, but I guess the author just didn't call their functions from `main.rs`, and didn't re-expose them either?

Anyway usually in that case you'd just `allow` the warning at the toplevel.

> Add the following three lines to your Cargo.toml and you're good to go: reqwest = { version = "0.11", features = ["json"] } and tokio = { version = "1", features = ["full"] } and serde_json = "1".

FWIW reqwest has a `blocking` submodule, so you don't have to explicitly care about async to use reqwest (though it'll still be pulled as I think reqwest just implements the sync API on top of the async one).

Since the article doesn't actually cover async it seems unnecessary to bring it up.

Incidentally, Rust's async is very similar to Python's semantically, but thanks to the static typing it's also significantly more reliable and easier to use (because the compiler can tell you when you're not using / awaiting a coroutine, whereas python will blithely let you ignore it, and only unhelpfully tell you that you've got unawaited coroutines somewhere at program termination).


dead_code is required because Rust compiles each integration test as a separate crate. This causes spurious warnings to be emitted if you want to share code between tests. See https://github.com/rust-lang/rust/issues/46379

Modules and crates continues to be one of the most confusing aspects of Rust, for me at least.


Here dead code is on the SUT not on the test utilities. In that situation the SUT would normally be either used or pub, here it’s not because the SUT is artificial, library-type code in a binary crate.


This is a good and slightly random intro. Kind of nice that it tells you about stuff other than the borrow checker.

Minor nit though: Rust doesn't have exceptions.


Is has panics which are functionally equivalent to (unchecked) exceptions.


They are not even close. In Python they are a control flow mechanism (ie StopIteration), in Rust they signal a program-terminating error.


Python is exceptionally exception happy.

Java has the checked/unchecked exception divide, where checked exceptions for expected errors that need to be handled specifically, and unchecked exceptions for unexpected errors, matching the use cases of Result and panic respectively in Rust. Unfortunately Java's checked exceptions are cumbersome and don't integrate well with interfaces or generics.

In C# the role of exceptions is not so clearly defined, largely due to the lack of a good alternative error handling mechanism. You have influential developers like Eric Lippert (then part of the C# compiler team) argue against using exceptions for expected errors https://ericlippert.com/2008/09/10/vexing-exceptions . Though I'd go further and would call the use of exceptions to signal IO errors an unfortunate design decision of the language/framework designers. My own experience as web developer is that exceptions will almost always be caught only at the request handler level and turn into an http error.

In C++ the standard library doesn't use exceptions for most errors. And it's common to avoid exceptions entirely (e.g. the google style guide forbids them).

So I'd say the role of unchecked exceptions is not too different from panics in many other languages, though the situation tends to be more messy in Rust due to legacy code and the lack of good alternative error handling mechanisms.


That's a difference in usage. In the default configuration, panics can be caught and thus are functionally identical.

The actual difference is that you can set panic=abort as the user of the code (it's a compiler parameter), in which case there is no unwinding and the program aborts immediately.

To me, the lack of possible handling and recovery (based on a compilation flag) is what arguably disqualifies it as an exceptions system.


Panics _can_ sometimes conceptually share the same implementation as exceptions, but given that their usage is not the same, and the fact that the ecosystem doesn't use panics as exceptions, it's a bit of stretch saying they're "functionally" the same.

A bit like saying trains and chairs are functionally the same since they both can have wheels; when there's the crucial difference that chairs don't go choo-choo.


> given that their usage is not the same, and the fact that the ecosystem doesn't use panics as exceptions, it's a bit of stretch saying they're "functionally" the same.

No, these don’t matter. Every langage with exceptions uses them differently, that doesn’t make Python’s no-exceptions and Java’s exceptions.

Regardless of how it’s used, it either is or is not an exceptions system. That’s what “functionally” means.

For instance Go’s panics are an exceptions system, you can always recover() and handle the exception during its unwinding.

> A bit like saying trains and chairs are functionally the same since they both can have wheels

No, it’s like saying an HSR and an ore train are functionally similar, because both are trains even though they’re used differently, in different contexts, and with different performance profiles.


You are arguing on technicalities. By your logic signals are also exceptions, and so is poking a screwdriver into the RAM port. Makes no sense.


On hw level, memory access errors and hardware malfunctions are often called exceptions. For example, on x86 the MCE architecture does this.


Windows exceptions are the equivalent of UNIX signals.

On hardware level exceptions are a synonym for traps.


Keep in mind that (although irrelevant at beginner level), if you’re writing any degree of Unsafe Rust, you need to ensure all your unsafe functions are exception safe, so they aren’t just a trivial concept.

https://doc.rust-lang.org/nomicon/exception-safety.html

Also, there is something similar to try/catch in Rust to catch exceptions, although the docs emphasize that you shouldn’t use for typical error handling. It’s mainly for when Rust is called from the outside (typically a C program). This is because you don’t want a Rust exception to unwind to external code, or all kinds of weird undefined behavior will happen.

https://doc.rust-lang.org/std/panic/fn.catch_unwind.html


An exception can be caught


A panic can be caught with catch_unwind. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html


However you can set panic=abort in which case no unwinding takes place.

This makes panics completely unreliable for error reporting.


Eher depends on what you want from an exceptions mechanism.

There has always been a debate what exceptions are for. Python leans heavily towards "exceptions don't have to be exceptional, they can be normal control flow", while Rust is in the other side of the spectrum closer to "if it's something you might want to handle it's not exceptional enough to be an exception". Java is somewhere in between. All of them are valid view points.

A language without reliable exception mechanism is C, where even the most unusual errors are communicated through return codes or global flags that you are expected to check manually


> Eher depends on what you want from an exceptions mechanism.

Not really. If it instantly terminates the program without recourse, it's not an exception system: by definition, exceptions come with exception-handling of some sort.


When using panic=abort you can still report the error via the panic hook, you just can't recover from it.


A Rust panic can be caught, too.

It's just bad form to do so in Rust because it's bad for maintainability to design programs like that.

Rust is opinionated and very much NOT an anything goes language. There is a culture and the culture says do not catch panics. If you feel the need to catch a panic, make it a regular value instead.



This is by choice, while it is really convienient to interrupt execution flow by throwing an arbitrary value, it's extremely hard to know whether calling library code can throw, and if yes, what kind of errors, without exceptional documentation.

The Result<T,E> type is very explicit, and can be easily ignored, composed or "re-thrown" with the "?" suffix without nested try/catch blocks. Return value based error handling is something Go, Elixir and other more functional languages have also adopted.

Panic is there to aleviate the really exceptional circumstances, when the trade-off for possible program termination is worth the much simplified error handling, such as when casting to a smaller integer type in case of overflow, or locking a mutex when it may be in a posioned state (which, funnily enough, can arrive when a panic occurred whilst the mutex was previously locked, i.e. the mutex guard is dropped during a panic).


> Panic is there to aleviate the really exceptional circumstances, when the trade-off for possible program termination is worth the much simplified error handling

It would be nice if Rust grew "panic annotations" so that we could determine shallowly and with automated tooling whether functions could panic. It would make it easy to isolate panicky behavior, and in places where it is absolutely necessary to handle, ensure that we do.


This kind of already exists in the form of #[no_panic] [1]?

> If the function does panic (or the compiler fails to prove that the function cannot panic), the program fails to compile with a linker error that identifies the function name.

1: https://github.com/dtolnay/no-panic


Almost anything will panic when you're out of memory, as allocating is regarded as infallible (due to above mentioned tradeoff).


Yes I know. I wasn't complaining about the lack of exceptions in Rust (I kind of hate exceptions) - I was pointing out that calling Rust errors "exceptions" is not correct (and probably misleading).


Would anyone have any recommendations on material for learning Rust? Is The Rust Programming Language still the best way to start?


Intro/conceptual: "A Firehose of Rust, for busy people who know some C++"[1]. Beginner's: "Rhymu's Videos"[2] and/or "Doug Milford" videos[3], advanced: "Jon Gjengset" videos[4]. Finally, "The Little Book of Rust Books"[5].

Rust is easy to learn by studying (in order to understand its unique concepts such as the borrow checker and lifetime annotations), but Rust is hard to learn by experimentation alone (trying 100 different syntax variations to see what compiles).

[1] https://www.youtube.com/watch?v=FSyfZVuD32Y

[2] https://www.youtube.com/c/rhymu8354/videos

[3] https://www.youtube.com/channel/UCmBgC0JN41HjyjAXfkdkp-Q/vid...

[4] https://www.youtube.com/c/JonGjengset/videos

[5] https://lborb.github.io/book/


The best way to start is probably Rust By Example https://doc.rust-lang.org/stable/rust-by-example/ and/or Rustlings https://github.com/rust-lang/rustlings/ . The key is to do those exercises "the hard way" and get some familiarity with the mechanics of coding in Rust. Once you've done that, reading TRPL will make a lot more sense.


I'm not a Rust expert by any means, but know enough to be mildly dangerous.

The Rust Programming Language is, to my mind, one of the reasons for Rust's popularity. Every language would go far by having a book of that quality freely available and accessible to such a wide audience. It's, truly, a feat.


My favorite as an old-timer who is comfortable in most things from assembler to Mathematica has been "Programming Rust" from O'Reilly [0]

The book does a really good job at explaining the thinking behind the abstractions available in the standard library. To me, this was the book that made everything click, so that I did not feel I was fighting against the language.

[0]: https://www.oreilly.com/library/view/programming-rust-2nd/97...


There are plenty of resources out there to get started, the one you mention is a very good one and can get you pretty far. My personal recommendation would be to decide on a small hobby project, simple in scope and complexity, to get started and figure out how to solve that using Rust. Just like with any other tool, the only way to learn how to use it is to use it. This takes some time but for me it's the best way to learn. So far there is no challenge Rust presented I couldn't solve using the tools at my disposal: Stackoverflow, Google and YouTube. For example, "a tool that takes some data from an API, does some stuff and stores the result in a database" or "an application that gets the latest 10 closing prices for a stock and sends a telegram message with a prediction". The most important part though (at least to me) is to write about it and share your experience, it helps cement the knowledge and might help others who want to start out too.


I have a bunch of links here https://michal.io/docs/dev/rust


There are some similarities, like the module system, and the "self" convention for methods which in Rust is a keyword.

At some point the similarities with Python stop and the similarities with C++ start. Although without the sins of C++ (like decades of backwards compatibility going back to plain C), which greatly reduces the decision fatigue you get as a C++ developer.

Unlike C++, Rust can be learned. I maintain that nobody truly knows C++. There will be always C++ code that cannot be explained by an expert. Everyone learns a dialect of C++ that they feel comfortable with, but the entire space of what's possible within C++ and the implications of each feature at compile-time and run-time became too complex to be learned.


I hear this often, and I get it, it's sort of true. And it's also false.

While the entirety of C++ is impossible to learn, it took me SIGNIFICANTLY less effort to get a grasp of C++ to the point where I could do everything I needed to do with it. Sure, we can call that a dialect, but it's enough.

Rust is an ongoing effort, because you need a lot more of the language as a whole to click before you get to the point where the compiler stops slapping you about elementary things.

Now, if I was designing an application where memory safety and security was very important, I'd probably try and step up that rust knowledge. But in my field (data science) that's just not a big concern and I can code up a C++ solution MUCH faster than I ever could in Rust.


One surprise perhaps is that both Python and C++ have multiple inheritance whereas Rust doesn't have implementation inheritance at all (Rust's traits can inherit but data structures and implementations cannot).

Both C++ and Rust have similar Quiz sites:

https://dtolnay.github.io/rust-quiz/

https://cppquiz.org/

You'll notice immediately that the Rust questions aren't trivial - this language does have some opportunities for confusion and weird ways to write things that surprise you and the quiz explores some things you'd probably never do and shouldn't do and their consequences are not obvious.

But on the other hand you will quickly also realise that while "That's Undefined Behaviour" could be the correct answer in C++ (it's the correct answer for several of the default questions) it's never going to be the right answer in (safe) Rust by definition.


I love the article, and it gave me the essential pointers to start with. I loved it so much that it inspired me to write a piece focusing on python vs. rust in data engineering. I leave the article here, in case of interest: https://airbyte.com/blog/rust-for-data-engineering


Darn I wrote a similar article. https://dublog.net/blog/rust-1/index.html


Is there something similar to Cython for rust that can be integrated with python to do most of heavy lifting while still being able to code in python framework


There's an entire ecosystem built up to support this. PyO3 [1] connects Rust to Python, Maturin [2] is a build tool to package Python wheels easily, with a host of popular examples using Maturn + PyO3 [3]. There's even extensions to bind easily to numpy and asyncio [4][5].

[1]: https://github.com/PyO3/pyo3

[2]: https://github.com/PyO3/maturin

[3]: https://github.com/PyO3/maturin#examples

[4]: https://github.com/PyO3/rust-numpy

[5]: https://github.com/awestlake87/pyo3-asyncio


What are the main advantages rust has over python?


Really hard question to answer simply. They're two languages that are at very opposite ends of the spectrum, yet they can usually both accomplish the same goal. I think the main difference is that Rust is significantly faster and uses less memory in the majority of cases, but also harder to learn and to reason about. Good package manager, tooling, etc, is a nice to have.

Python focuses on being simple, interpreted and dynamicaly typed, Rust requires you to specify the exact types of all the things (like C/C++) which allows it to generate really optimal compiled machine code before execution, it has more information to work with. Accessing a struct/"object" field is not a hash table lookup, it's a direct pointer access, as the exact size of things are known at compilation time.

If you're writing a script, a tool, a small game, it's simpler to use Python. If you're writing a database engine, a mission-critical piece of code that has to behave predictably without the possibility of random GC pauses, anything that has realtime constraints such as audio, lower-level languages like C, C++ and Rust are a must.

A lot of higher-level folk seem to enjoy Rust as well, especially in networking/the web, for whom the speed is worth the additional difficulty and complexity.


Speed is obviously the main one. I know that's not normally seen as so valuable in Python land because there's more of a focus on optimising the hot path rather than the entire codebase, but starting out in a fast language can give you a lot of flexibility in terms of performance later on. For example, I tried to enabled gzip encoding for responses in a Python web app recently, and it just turned out to be too much for the system to handle. But an experimental rewrite of the same project in Rust was so much more performant, that, had we gone down that route, we'd have probably been able to compress our responses considerably. This is what I mean by flexibility: by having all-round great performance, you can fit more stuff in before you need to start worrying about optimisation.

There's also type safety. Not just in terms of having strict compile-time types, but also in what those types can actually do for you. With enums, for example, you can specify different states, but also which data is available in those states. For example, you might model messages passed between different nodes as an enum, where one message contains the current date, and another message contains the number of active users. This way, the compiler proves that you can't end up in impossible situations: having a "date" message that doesn't have an attached Datetime instance, say.

Finally, the developer tooling is often leagues ahead of Python. It's possible to get by in Python with venvs, pip, maybe a requirements file, and so on. But there are often a lot of edge cases that this setup doesn't cover, so there's a lot of this party tools to cover up sharp edges - pip-tools, Poetry, etc. And even then, you often end up in odd situations, particularly if you're trying to retrofit these features onto an existing system.

In the meantime, cargo Just Works™. As in, cargo itself covers 90% of the cases I run into (managing dependencies, managing multiple projects, running tests, building, etc), and the 10% is usually very easy to add: cross compiling with the `cross` tool, linting with `clippy`, IDE support with `rust-analyzer` etc. Managing a Rust project is just so much easier than managing a Python project.

Different people will probably say different things, but as someone who used to be a big fan of Python, but now tends to use Rust for most side projects, those were the big advantages that did it for me.


Without having actually written any Rust, but being familiar with it and having written a fair bit of Python, the most obvious ones are: speed & static binaries, with cargo+crates coming a close second.


Bragging rights


Evils is in details.

After many years of programming, now i don't care much about "best practice", what i care is in the details. In Rust, it's how i take care of memory usage of every variables.

Or, write the most ugliest code that works the most performant as possible.


> Obviously I'm skipping a few things like "what is an enum?", "what does that pub stand for?", "why are those hashtags for?" and that's on purpose. Their utility can be deduced from running the code, their understanding can be deferred after seeing them in action.

Great writing!


I'm thinking it's best not to make Rust a low-entry-barrier resume keyword.

Rust is hard, and people should realize it's hard.

One of the problems with C is that it was way harder than most people realized (we just accepted crashes, and then rampant security vulnerabilities, as normal), and so few practitioners got sufficiently competent at it.

One of the problems with Python is that you don't have to understand much, to install a bunch of libraries and copy&paste snippets found in Web searches, to call something done.

Rust will help you with analogs of some of the conventions that a very skilled C programmer might use. But if someone only wants to get a well-paying job and be able to call their sprint tasks done, they probably want Python or Go, because Rust will make them cry.

If I Web search for a Rust thing, I want to be finding genuine hardcore people talking. (This was actually one of the biggest benefits of working in Scheme and Racket -- very high signal:noise ratio, because the practitioners were generally only there because they were very serious and/or very curious. I'm moving to Rust, and hope to find similar.)


I disagree pretty strongly with this: it should be incumbent upon you to filter your interests, not upon Rust to filter out potential users.

Rust’s entire raison d’etre is to make secure, performant computing available to everyone. It makes no sense to keep it elite; the idea is to improve the entire state of affairs.

Besides, it’s a big language, and has a wide range of practical idioms: I almost never use reference-counting types in my code, but it’s a perfectly legitimate (and sometimes necessary) feature that also makes the language more accessible to newcomers.


There is no real reason or place for gatekeeping in tech like this. When I was 15 I didn’t know anything and C was my first language. It led to a long and fruitful career in tech for me. Just stick to the hardcore groups, they are always out there. A motivated teenager can figure out this “hard” language. I was writing assembly, using function pointers, running a MUD, and competent at GDB by the time I was 17. Plus Rust is really easy to start with and the books are free. I had C for Dummies.


Not gatekeeping. Just not pretending it's easy.

Another tech community I'm in is known for being extremely helpful to newcomers who are interested (e.g., anyone who's interested will typically get an expert answering their questions). That's different from saying the subject matter is easy, and mass-advertising it to people looking for easy.


But a bunch of it is easy.

Are Unsafe Rust and Procedural (as opposed by "By example") Macros easy? No, unsafe Rust is potentially really hard. If you are implementing a new concurrency primitive or something you're going to need at least the expertise you'd want for the same work in C++. However in Rust you know where the line is, "Huh, I need unsafe, I should find a grown-up". And knowing that the problem areas are flagged gives you confidence everywhere else.

But many ordinary programming tasks are pretty easy, compared to Python you will spend a bit more time with the compiler refusing to compile your program because it's wrong, but on the other hand a bit less time with the program failing at runtime for trivial reasons a compiler would have spotted (compare what happens if you typo a variable name in Rust versus Python!)


Rust is a language that will reward you for embracing it and going deeper as well, which is great.


> But if someone only wants to get a well-paying job and be able to call their sprint tasks done, they probably want Python or Go, because Rust will make them cry

You can be about as sloppy in Rust as you might in Python or Ruby: the language supports dynamic typing ("Any" trait), reference counted objects etc. But that would involve a lot of boilerplate, so your sloppy code would very much stand out as non-idiomatic.


It sounds a bit like gate keeping :). Not to mention, Rust complexity can always be dialled down to a GC language if corresponding sacrifices in performance are accepted. There are a whole lot of domains where you can get away without using a single lifetime specifier anywhere.

I'm currently using it for developing rest api using the excellent warp crate and I wish more people use it for this use case as its quite suited for it.


>If I Web search for a Rust thing, I want to be finding genuine hardcore people talking

So you want the language never to be mainstream?


Rust will never be used as widely as GC'd languages like Java, Go, C#, etc.

It's incredibly well-designed, but you can't design away the inherent complexity of writing safe, secure, fast code. There's no way to make that easy. It will always take a lot of passion or training to really master it. (Yes, I know that some of us here "had an easy time" and it "only took a few weeks", I'm referring to the average person.)

And you know what, that's okay. There are lots of industries where only the most hardcore of programmers use the professional-grade tools. There's nothing wrong with that.

Everyone else will probably continue to use tools like Java, Python, etc. because for most situations, they can get the job done with less training, which means less spent on developer salaries, which is most often more than server costs anyway.

I think the best we can hope for is that Rust is taught in university, like how other industries teach usage of their own professional-grade tools. But even that won't mean that most programmers use Rust, I think.


For Rust to be successful, it doesn't need to beat Java, it just needs to beat C. But that's still mainstream.


>There are lots of industries where only the most hardcore of programmers use the professional-grade tools.

e.g?


> I'm thinking it's best not to make Rust a low-entry-barrier resume keyword.

When that happens, recruiters will instead look for occupational or open source experiences with Rust, which they have done for other mainstream languages for years and arguably it's how they should have done for Rust anyway. So there is no real problem here.


I dont think gatekeeping rust is necessary. The nice thing about rust is that people that are not experts can now attempt to create high performance low level software and the worst case scenario is that its slow due to unnecessary copying.


> I'm moving to Rust, and hope to find similar.

Ridiculous to gatekeep something that you don’t even identify as being part of yet.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: