Lua is a funky little language and runtime but it is absolutely perfect for scripting in the domain that it was designed for: embedded.
I used to do a lot of game dev and it was incredible for testing different interactions/movements etc at runtime just to get all the inputs right and then implement in the game code. It has saved me probably 1,000s of man-hours in my career. Nowadays I use it in NGINX for high-velocity requests that are just backed by Redis. Lua, LuaJIT, and Redis have made me a lot of money in my career. I'm forever indebted to their incredible creators and community.
I understand that the fact of the matter is that Lua dominates, but I'm curious if there are technical reasons or "right thing/right time" kind of luck, after TCL's star dimmed.
I'm not sure there is an easy answer for the question of why one language became more popular than the other but there some very interesting parallels that can be drawn between Lua and TCL. One thing that both have in common is that both languages were purposefully designed to be able to be embedded with other languages, such as C. But they do this in very different ways.
The problem of embedding one language inside another is how do you make code written in one language call functions and objects defined in the other language? For example, when an object is transferred from the scripting language to the host language, how does the garbage collector keep track of it? This isn't easy to do and in some languages it is quite cumbersome. For instance, in Python the C programmer has to manually increment and decrement the reference counts for the python objects that they interact with.
TCL solves this problem by making everything a string, taking the "everything is a string" mantra farther than any other programming language. Strings are easy to pass from one language to the other.
Lua went the opposite direction. Lua exposes a rich API that lets other languages interact with Lua. The API lets other languages create Lua objects and call Lua functions, as well as send C objects and functions to Lua. Lua is entirely designed around this API and even the core standard library is built using it, ensuring that every Lua feature is accessible to C code as well. One interesting example of this is exception handling. In most languages this is done using try-catch blocks. But that feature isn't easy to use when embedding one language inside another. How would you write try-catch blocks in C? Because of this, Lua has a different way to do exception handling. Instead of try-catch blocks, there is a "pcall" function that invokes a function and returns whether there was an exception or not. Function calls and if-then-else statements are things that are also available to other languages via the Lua API...
local ok, err = pcall(function()
something_that_may_throw_an_exception()
end)
if not ok then
-- handle exception
end
Don't know much about TCL. But, if you are familiar with JavaScript, Lua ended up being semantically a much cleaner, simpler JS without all the WOT?? and committees to please.
It is small, fast, simple, powerful and easy to embed.
It's also very easy to contain in that the standard library is small and most of that can be removed easily. Isolating Lua's allocation arena and controlling Lua's GC is pretty easy.
All the little operators that make things nice like += are missing. Their object implementation is meta-tables, so every single codebase does stuff in fundamentally different paradigms. Operator overloading in such a dynamic language makes things extremely hard to understand. While `"0" == 0` is false, the language will still sloppily convert between floats and ints and will also convert strings to numbers in other places. Worst of all, array indexes are base 1.
mpv supports both Lua and Javascript scripting, with more or less the same API for both. The wiki lists 10 scripts from the community written in Javascript vs a few hundred written in Lua. There are probably network effects in play; people choose Lua because that allows them to mine snippets of code from a larger collection of userscripts. Even so though, I think it's a testament to Lua.
There is barely a comparison to be made. Lua is faster, LuaJIT is much faster, lua is much easier to embed and lua is much more elegant and way less awkward to program in. TCL ends up being funky and lua ends up working.
When there was no competition in the space of already made scripting languages that were easy to embed (except for possibly perl) TCL had a place, but it is a relic of the past now and for good reason.
We use NGINX + Lua [1] to collect real-time diagnostic and performance data from physical machines while they operate. We collect about 150,000 data points (HTTP requests) per second with 2 NGINX servers and 2 Redis servers. We've benchmarked this particular server group up to almost a million req/s before it started to break down (median resp time > 100ms).
We recently implemented a simple streaming server based on Openresty and Redis. Redis is used as buffer. We load tested it up to 2000 mountpoints (128kbit mp3 streams) with 15.000 clients.
The architecture has some nice properties. For example the buffer being not in-process means (besides an obvious performance hit) that the streaming server can be seamlessly redeployed without loosing the buffer.
At one company that served a lot of requests from an Openresty cache, we used nginx's cache for caching full HTTP responses and lua-resty-mlcache for caching other things. lua-resty-mlcache requires one to provide a callback to get the data as a string, then a way to deserialize the string, because the l2 cache in nginx's shared memory zone will store the item as a string. We used msgpack for this part.
Agreed. We've embedded it inside Mudlet (https://github.com/Mudlet/Mudlet) for a decade and people have started their careers in programming thanks to wanting to code for their favourite text game.
Agreed entirely, but of course it can be embedded in more things than people might expect. For example redis, HAProxy, nginx, etc.
I wrote a console-based mail-client which was configured and scripted entirely within lua. That was my solution to the weird not-quite-real configuration options available to mutt. I don't use it any more, and it never became terribly popular, but it was a wonderful fit.
Author here. I'm delighted to see this is still interesting / useful to folks. In case anyone is curious, here's a short origin story for this tutorial.
I learned Lua for a game jam - during the course of the game jam itself. Like many, I found it a bit weird at first. 1-indexing feels wonky (you get used to it, though); now that I use Python all day, I would miss syntax shortcuts like list comprehensions going back to Lua.
When I was a kid, I learned C from a little pocket reference book while building little self-motivated projects. I still think this is the best way to learn - have a project in mind, and just enough fundamental examples to get you there.
I threw together the Lua tutorial mostly in one sitting, spending some time to edit, and going back and forth on how to keep the whole thing runnable while still explaining how external modules worked. Shortly after publication, Adam Bard contacted me with his idea for learnxinyminutes.com. I said great idea and ok'd the inclusion of the tutorial on Adam's site. Adam invited others to write their own similar-style tutorials, and his site grew up quickly.
Lua itself is a truly beautiful language, though I understand this is not a popular opinion. It's beautiful because you can learn the entire language, through and through, quickly. Memory and comprehension of the language itself get out of the way, so the only hard problems left are your decisions about what to do with the language. (You ask "How will I?" vs "How can I?") Instead of being feature rich through added features, it's feature rich through good language design. I expect some will disagree with me, but as one comparison point, consider the complete syntax of Lua:
It's not apples-to-apples - the ECMAScript spec is spelling things out in detail, while the Lua section is an overview - but it's still a thought-provoking contrast.
I'm working on a huge existing game dev project that is done in lua, and I can totally agree with you. The last project I did before this one was in, of all things, Typescript, so when I started to use lua I was quite weirded out by how simple it is... "What about type checking, async/await, list operations, refactoring, test coverage, and so on ?! How can I be sure that whatever I'm doing actually works?" But after just around 2 weeks on the project I started to realize that I basically already knew everything about the language and was already very productive on the project. Sure, I didn't have the same safety as with other languages, but I was churning out features and bug fixes, and constantly overestimating work.
Even though I don't think I would start a big project like this on such a simplistic language, I can't ignore the benefits it has. I guess at end of the day, you have to balance out not just how good and secure the code is, but also how quick it is to make changes to it and to onboard people to it.
> Lua itself is a truly beautiful language, though I understand this is not a popular opinion.
I think most people who actually use Lua come away marveling at its elegance and simplicity. The ones that don’t get stuck at “arrays start at 1?! What a garbage language!” In fact, this is a fairly good way to tell if someone has a well-reasoned opinion about Lua ;) (I will put up Go as a counter-example: “hurr durr no generics” is a meme complaint but it remains one once you actually use the language…)
I've used julia and lua and tend to think if people complain about indexes starting at 1 instead of C's offsets starting at 0, they can't be working on anything too difficult, because it just isn't a big issue in the grand scheme of things.
It's a very well designed language. Authors call this 'orthogonal design'; features like iterators, meta-tables and coroutines are independent and you mix and match those to produce architecture you need. It takes some hard work to design things orthogonally, to identify and isolate usable abstractions, and then put them together in a way that has no rough edges. It seems Lua has pulled it off quite well, with very little accidental complexity.
I'm amazed the language hasn't caught on more. It's perfect for education, as it is simple to learn basics and algorithm design. It's already used in some of most popular games (Roblox, Minecraft mods, WoW...) so there's established playground for children creations. Code flows like English language (which is bad for advanced users but perfect for kids). I'd say it's more suited for education than Python because it is simpler and more transparent.
The things missing are tooling and documentation. There's only single popular IDE. The Programming in Lua book is awesome read (I'd recommend it even to non-Lua developers), but it's not a good language reference. The reference itself is quite useful but sparse on examples. I wish there was clojuredocs.org for Lua, community-sourced examples are amazing idea.
Yeah, based on the low quality of the lua wiki I've started using the reference manual as my source for anything that ships with the language, but the reference manual just doesn't fully specify how a bunch of built-in functions work...
Lua + LuaJIT has been a great combination when it comes to Kong[1] which I believe is the largest open source OpenResty application. Not only we can leverage the underlying NGINX networking primitives for fast non blocking IO, we can also hook into each request and response via Lua to provide APIM functionality and run the whole thing on the piece of art that is LuaJIT[2] created by the legendary Mike Pall.
I have to admit it does have its quirks (Lua lists have a base index of 1 instead of 0), but so far it's been phenomenal. As far as I know, Cloudflare is also leveraging the same stack (OpenResty) for their global CDN.
I grant that it would have gone over 15 minutes, but it's a pity that such a pithy summary doesn't cover coroutines or environments at all.
Coroutines are what elevate Lua beyond "a better Javascript" into something truly great, while first-class environments, although something you won't reach for often, allow for some things which just aren't possible any other way.
OpenResty embeds Lua in nginx, the results are out-of-this-world speed and power. I know OpenResty can be used for complex projects but I always used it for just that little extra simple configuration can't do.
The first few releases of the "Monkey Island" series from LucasArts head a grimy pirate-themed tavern called the "S.C.U.M.M. Bar" (for Script Creation Utility for Maniac Mansion).
feel the need to bring up the wonderful https://learnxinyminutes.com, which has a ton of references just like this for other languages. It actually looks like the lua file there is the exact same one linked!
Lua was originally created to be used by a group of FORTRAN programmers. Lua starts with 1 because FORTRAN starts with 1. FORTRAN starts with 1 because Math starts with 1 (Sum of f(Xi) for i in 1 to N)
Sometimes math starts with 1 and sometimes with 0.
I was once implementing some math algorithm whose description had various subscripted variables. Some ran from 0 to whatever, and some ran from 1 to whatever. I was implementing in C++, with each of these variables becoming an array. I didn't want to write var[i-1] all over the place for the ones that ran from 1 in the book--it was a tricky algorithm and for maximum clarity I wanted my code to read as close to the book as possible.
What I did was overload the () operator to make it a 1-based array access operator. I.e., var(i) == var[i-1].
That worked out reasonably, although it probably would have been better just to make the arrays for the 1-based variables one longer and just ignore var[0]. In other words, if var in the text has subscripts that run from 1 through 10, make the var array in C++ 11 long, and just use var[1] through var[10].
In contrast, indices starting at 0 always baffled me in other languages. In pretty much any real-world scenario, you always start at 1 (it's literally "the first element" as in "1st"). It always seemed to me like language developers for some odd reason decided to apply spatial coordinates to arrays.
0-based array indexes make sense when what you are working with is laid out in memory from an offset. Honestly, if we cared about purity literally everything else (linked lists, etc) should be 1-indexed, but, stuff like composition and abstraction turned out to be more important, so we're stuck with 0-based indexes for everything.
Let's say you want every 5th element of an array, with zero indexing your code can be very elegant. Your indexes are 5 * 0, 5 * 1, 5 * 2. With 1-indexing you have to do (5 * 0)+1, (5 * 1)+1, (5 * 3)+1 etc.
This always seems to pop up in these discussions but I don't think I am terribly convinced by his argument. For example, he doesn't mention that option (c) is better than option (a) if you need to iterate backwards. In that case option (a) becomes option (b), which everyone can agree objectively sucks :)
Yes! Reverse ranges in Python are awkward for this reason. The reverse of `range(0, 5)` (i.e. [0, 1, 2, 3, 4]) isn’t `range(5, 0, -1)`, it’s `range(4, -1, -1)`.
I think it would be better if the end-points were “low” and “high” rather than “start” and “stop”, so the reverse of `range(0, 5)` would simply be `range(0, 5, -1)`.
I think Rust has another good alternative to this, although it might be cheating a little: you can just write (lo..hi).rev(). That way, the common case of going forwards is nice, and the case of going backward isn't much worse.
If you want to treat your table (list) as integer indexed array (i.e. this is just called "array" in many other languages), and use length '#' operator, yes, you need to start at index 1, however, you can use 0 otherwise, or even strings.
Lua is a small language, and tables are the basic building blocks for arrays, dictionaries, classes, modules, ...
1-based indexing is IMHO a small price to pay for such flexible and efficient syntax, with small cognitive load)
I think it's because the creator of the language was originally targeting mathematicians as the primary users. And they consider 1 to be the start of a numerical range. It didn't start to become a "general purpose" programming language until later.
var a : array of integer;
b : array[0..9] of integer;
c : array[1..10] of integer;
begin
writeln(low(a)); // 0
writeln(high(a)); // -1
writeln(low(b)); // 0
writeln(high(b)); // 9
writeln(low(c)); // 1
writeln(high(c)); // 10
end.
As someone who works with lua/nginx seeing a variable declared without `local` makes me immediately nervous as it's creating a global. Maybe start with a `local`?
If you've got decent programming experience, that's all it takes.
I started learning Lua and began working on a game within an hour without needing to reference language documentation much at all. It's a language without many secrets.
Can anyone recommend IDEs for Lua? I've used Visual Studio Community for C# and Eclipse for Java and been really happy with them. Mainly looking for code auto-completion and debugger. Right now I'm doing a small game with C# and Otter2d, but Lua with löve2d seems interesting.
I used Lua back in 2012 to perform call processing for SIP messages. It was one of the first interprovider solutions for video calls. It was our best option over TCL/JS to provide fast processing of text, eventually we allowed users to enable it in Cisco Callmanager and SBCs.
Thanks for the post! It greatly improved my understanding of Lua, which I've followed but never used in a project. I'd love to see more tutorials like this, which assume the reader is a programmer but not in that particular language.
Perfect. Just started using IceSL (https://icesl.loria.fr/) as an alternative to OpenSCAD for designing things to 3D print, and its scripting is all Lua-based.
I really dislike that many languages still allow you to do strings with single or double quotes. Why not just choose one and stick with it? It’s another pedantic thing that allow people argue about the “best way”
Because it lets you use single quotes ("isn't") or double quotes ('error: "%s"') without escaping ('isn\'t', "error: \"%s\"").
A rare case where readability and consistency may not be the same. (Let's not argue about this, it's subjective. I'm aware of the obvious counter- and counter-counter-arguments, such as syntax hilighting and greppability.)
The problem is that a single quote will often times appear in strings as well, and sometimes (like in the case of bash) can't even be escaped. So you end up requiring double quotes either way.
Why not just settle permanently on double quotes? This is what Go and other languages did and I think it's a good idea.
> This is what Go and other languages did and I think it's a good idea.
Well Lua is about 30 years old and Go is, what, 10 years old? I would expect a language that is barely a decade old to improve on past ideas and mistakes.
> The problem is that a single quote will often times appear in strings as well, but they can't be escaped
While having choice of quotes reduces the need for escaping, most languages that provide it also still support escaping. E.g., Ruby has double-quoted strings with escaping, single-quotes without, %Q(...) with, %q(...) without, and two different heredoc forms, with and without escaping (the forms that allow escaping also support interpolation).
I actually like this possibility since it gives you more meaning: E.g. I would always tag localized strings by encapsulating them differently. It's great!
Well that's the first language where I've responded negatively to the comment syntax - and I've worked with tcl.
Also, seriously? Double is more than enough for integers? No. No. No. I'm not going to attack every single line of this article, so lets just say I was triggered.
My daily use languages are Verilog, Python and Tcl.
Edit: I'll actually backtrack to say that this post does a great job of demonstrating things that are non-obvious.
Must have been just the impression I got. When I read about it many years ago, I thought it was very much like JS except with a different syntax. I mean, besides the number thing, it uses prototypes for inheritance, and its "tables" are very much like JS objects in the way of how o.x == o["x"] for example, or how arrays are basically built on them (though augmented with a little syntax-sugar). I don't know of other languages that share these characteristics.
Huh. You're right. Though not several years apart, Lua appeared in 1993 and JS in 1995. I wonder if there's a similar HOPL paper for JS that could confirm Lua's influence.
Not OP, but for me it's not... "blocky" enough? Especially for inline commands it's maybe just too easy to glimpse over it. But otherwise it's also maybe just plain unfamiliar.
I used to do a lot of game dev and it was incredible for testing different interactions/movements etc at runtime just to get all the inputs right and then implement in the game code. It has saved me probably 1,000s of man-hours in my career. Nowadays I use it in NGINX for high-velocity requests that are just backed by Redis. Lua, LuaJIT, and Redis have made me a lot of money in my career. I'm forever indebted to their incredible creators and community.