From the research I did, here's how languages stack up in Lambda runtime (lowest first):
1. Python & JS
2. Go
3. C# & Java
I couldn't find any data on Rust.
The understanding at the time was that Python & JS runtimes are built-in, so the interpreter is "already running" Go is the fastest of compiled languages, but just can't beat the built-in runtimes. C# and Java were poorest as they're spinning up a larger runtime that's more optimized for long-running throughput.
This is very true. Just importing Django + Django Rest Framework + some other minor libraries in Google App Engine (standard) leads to painfully slow response times when a new instance spins up. Like, more than 10s to spin up an instance. Although App Engine seems to be 3-4 times slower than my desktop computer from 2014 on this particular task. I wonder if AWS lambda is better.
> In the real world you don't import 5 seconds worth of dependencies into a lamdba
Laughs in data science.
> a 5 second boot time for a longer-lived service is acceptable.
Not every application can tolerate the occasional 5-second-long request. Just because Python can cold boot "hello world" 3 seconds faster than Go doesn't mean that's going to hold in the real world.
You're mixing arguments here. It's not the occasional 5-second long request, it's "the app doesn't start serving requests for 5 seconds".
Using data science tooling in a lambda seems iffy, especially ones that are not production ready. And good luck getting such libraries in go.
Python cold booting an interpreter 3 seconds faster than Go is a big deal, especially if your target execution time is <50ms and you've got a large volume of invocations, and are not being silly and importing ridiculously heavy dependencies into a lambda for no reason other than to make a strange point about Python being unsuitable for something nobody should be doing.
> You're mixing arguments here. It's not the occasional 5-second long request, it's "the app doesn't start serving requests for 5 seconds".
Lambdas cold-start during requests. So the unlucky request that triggers a cold start eats that cold start.
> Using data science tooling in a lambda seems iffy, especially ones that are not production ready.
Nonsense, there are a lot of lambdas that just load, transform, and shovel data between services using pandas or whathaveyou. Anyway, don't get hung up on data science; it was just an example, but there are packages across the ecosystem that behave poorly at startup (usually it's not any individual package taking 1-2s but rather a whole bunch of them scattered across your dependency tree that take 100+ms).
> And good luck getting such libraries in go.
Go doesn't have all of the specialty libraries that Python has, but it has enough for the use case I described above.
> Python cold booting an interpreter 3 seconds faster than Go is a big deal, especially if your target execution time is <50ms and you've got a large volume of invocations
According to https://mikhail.io/serverless/coldstarts/aws/languages/, Go takes ~450ms on average to cold start which is still up a bit from Python's ~250ms. To your point, if you're just slinging boto calls (and a lot of lambdas do just this!) and you care a lot about latency, then Python is the right tool for the job.
> not being silly and importing ridiculously heavy dependencies into a lambda for no reason other than to make a strange point about Python being unsuitable for something nobody should be doing.
Not every lambda is just slinging API requests--some of them actually have to do things with data. Maybe someone is transforming a bit of audio as part of a pipeline or doing some analysis on a CSV or something else. Latency probably matters to them, but they still have to import things to get their work done. And according to https://mikhail.io/serverless/coldstarts/aws/#does-package-s... (at least for JavaScript) just 35mb of dependencies (which will buy you half of a numpy iirc) causes cold start performance to go from ~250ms to 4000ms.
My rule of thumb (based on some profiling) is that for every 30mb of Python dependency, the equivalent Go binary grows by 1mb, moreover, it all gets loaded at once (as opposed to resolving each unique import to a location on disk, then parsing, compiling, and finally loading it). Lastly, Go programs are more likely to be "lazy"--that is, they only run the things they need in the main() part of the program whereas Python packages are much more likely to do file or network I/O to initialize clients that may or may not be used by the program.
The way I'm using lambda, I compile the lambda build image beforehand which contains the python packages already installed, and the only "time" restraint is that of the lambda spinning up itself.
If you ran e.g. "pip install -r requirements.txt" inside the lambda, then yes it would take time to install the packages.
Installing packages onto the system (“pip install”) is different than the interpreter importing them (loading them when the interpreter hits an “import” statement). Not only is it resolving imports into file paths and loading them into memory, but it’s also executing module-level code which tends to be quite common in Python, so it’s not at all uncommon for imports to take 5s or more.
Meanwhile in Go, dependencies are baked into the executable so there is no resolving of dependencies, and the analog to “module level code” (i.e., package init() functions) are discouraged and thus much less common and where they occur they don’t do as much work compared to the average Python package.
Interesting, I see what you mean, but in my time working with python I've never seen that as an issue. Perhaps in different domains such as big data it might be a problem.
The numbers you link don’t support your ranking, unless you’re specifically ranking by cold start alone.
Even then it doesn’t make sense to group Python and Node but not Go, as Node and Go are significantly closer than Node to Python.
A lot of this was based around the fact that we've seen languages become just so much more performant. This includes Go/Rust/etc, but a lot of Node.js workloads are also sub 100ms, or fast enough that they'd benefit from this pretty well.
I've got bad experiences with go startup (i.e., cold runs). They're much more expensive than I would have expected. If node can indeed run in 40ms (as https://news.ycombinator.com/item?id=25267211 says), then I'm surely going back to JS.
What is your experience? Go is an AOT compiled language so the only thing I could imagine you running into on startup is loading the binary into memory? Theres not a cold-start issue with Go, as its not an optimizing JIT.
My experience is that Go cold start takes around 900ms. Processing (parsing a JSON message, verifying it with one DynamoDB look-up, and storing it in DynamoDB) then takes between 11ms and 105ms. Go does use less memory than node, though, and that also counts in Lambda.
I hadn't expected it either, but it loads node faster. Perhaps via some VM trick?
> The function did nothing except emit ‘hello world’.
A more realistic benchmark would be parsing a 1kb protobuf blob and printing some random key from it.
(this would require importing a non-stdlib parser)
Without knowing how it's implemented, my guess is that they're conserving python/v8 processes, so that they're not cold-starting the interpreter on each lambda execution.
You can't [1] do the same thing for a Go binary, so they have to invoke a binary, which might involve running some scans against it first.
This leads to some pretty counterintuitive conclusions! If you want minimal latency (in Lambda!!), you really should be using JS/Python, I guess.
[1]: OK. Maybe you could. Go has a runtime after all, although it's compiled into the binary! I have never heard of anybody doing this, but I'd love to read something about it. :)
same. I went through the trouble of implementing my function in Rust (Rocket), and it's actually quite slow because (a) startup is slow and (b) async/await is still pretty painful to use so I'm blocking on IO
JS is a great choice for Lambda thanks to great cold performance. I’m seeing runtimes in the 40ms to 100ms range.
Most of the time in Lambda is usually spent waiting for IO, which is slow in any language. If you’re using Lambda for heavy computation, that’s not a great choice.
That's the point- billed per ms, a Lambda that executes in 5ms is 10x cheaper than one that takes 50ms. Billed per 100ms interval, the total cost of the two is the same.
IME Lambda functions are mostly sitting around waiting on I/O, so I don't think it would make much of a difference for those workloads. The important technical factors for those workloads are startup time and I/O capabilities...JS is strong in both of those areas. For simple Lambda functions JS still seems like a great choice, along with Go. Rust would be overkill IMO unless you need to share a codebase or aren't I/O bound or have some other unique requirements.
