> knowing the details of heap vs stack, array vs hashmap, and explaining that to someone learning programming for the first time within a language that attempts to hide those details is frustrating
These are implementation artifacts. Knowing how to program does not require any knowledge of them.
> and my old tool looks like a toy.
The great thing about toys is they are very easy to put down for a while and then pick back up again later with very little effort.
> The great thing about toys is they are very easy to put down for a while and then pick back up again later with very little effort.
That matches my experience.
I've written programs in Haskell, and it was very satisfying. But going back to the project after 6 months of work in other languages was very hard. I took some time to remember the concepts, the abstractions and what the cryptic operators meant in their contexts. I've kept away of Rust for this reason: maintenance would be hard if I only dabble episodicly in Rust.
On the other side, my own experience with Lua was not very nice. I contributed (and patched for my needs) Koreader, an ebook reader with 180k lines of Lua. The lack of explicit types and data structures was a strong impediment. And the language has subtle traps... Everything is a table, but not all tables are equals: some tables are sequences (indexed by 1..n) with special operators (like # whose result is undetermined for other tables (which hurts like the famous Undefined Behavior of C++). With Lua, simple questions like "Is this table empty?" or "What is the size of a table?" are too hard for beginners.
So, complex artefacts are hard to go back after a long break, but many toys break easily when you come back after a pause, having forgotten how fragile they were.
I find Rust much better on this front than Haskell. Haskell has a much stronger culture of using library-specific operators and abbreviations for function, variable, and type names, than Rust does. In Rust, you can’t even define a custom operator.
If you understand borrowing and some of the basic traits, you’re a long way there. There aren’t new DSLs to learn or relearn, and the documentation is good at closing any gaps that open up while you’re away from Rust.
> Everything is a table, but not all tables are equals: some tables are sequences (indexed by 1..n) with special operators (like # whose result is undetermined for other tables (which hurts like the famous Undefined Behavior of C++).
IIRC, the right way to check if a table is empty is
next(mytable) == nil
As the sibling comment points to, the documentation of the # operator is complex. It returns a border which is defined as:
a border is any positive integer index present in the table
that is followed by an absent index, plus two limit cases:
zero, when index 1 is absent;
and the maximum value for an integer, when that index is present.
Note that keys that are not positive integers
do not interfere with borders.
When the table has multiple borders, the behavior is undefined.
{} is explicitly defined as a sequence of length 0 by the language spec.
But for any table which has a nil in it between two non-nil values - i.e. non-sequences - there's basically no guarantees on which one of the several possible values # will return.
Yes, that's true. What isn't true is that "some tables are sequences" which have "special operators" whose results are "undetermined for other tables".
A table has an array portion, it has contents or it doesn't, you can check this as I illustrated (I didn't intend it to check if the table is entirely empty, the sibling comment to yours shows how to do that). `#` works on any table.
It is in fact the case that if you start sticking `nil` into the contiguous array portion of the table, you'll have problems: `#` and `ipairs` won't work correctly. That's part of why `false` (and therefore `true`) were added to the language.
It's a quirk you have to know to use Lua effectively, that's all. In years of programming Lua, I've never had a bug which originated from adding `nil` within the array portion.
As the only composite data structure in the entire language, it's reasonable to expect people to learn how they work if they're using Lua. The one correct observation in the post I was replying is that there's no obvious or cheap way to find the number of entries in a table (other than the array portion). Luajit has `table.nkeys`, and that should have been imported into stock Lua.
There are a lot of anecdotes in this thread which amount to "my experience with Lua was as a means to an end: I was using it as a scripting language for something else, and it subverted my expectations in some way or another (mostly by not being Python)". Understandable in a way, but it should take half a day at most to read the entire manual and the online edition of Programming in Lua, and there's no sense in blaming the language because a user wanted to get something done and figured they would skip that part.
The point is that # is basically unusable for non-sequence tables because there are no guarantees on which border it will return. But at the same time, when that happens, you do not get any clear error, either. That is the nasty part here, and it is exacerbated by the lack of any kind of static or even dynamic typing to distinguish sequences from non-sequences - e.g. Python is also a very dynamic language, and has a rich set of built-in data structures, but there's no point at which len() will silently do the wrong thing for any of them. Either a given collection type supports it, in which case you get a meaningful value, or it doesn't, in which case you get an error. In Lua, if you ever get an invalid table as input, you will silently get the wrong value which is nevertheless indexable, and so when you use it you get another wrong value etc, and eventually your computation just produces garbage output with no obvious clues as to where the error was introduced. That is a clear design defect in the language.
As far as personal experiences, I can't speak for OP, but I've learned Lua in early 00s, long before Python. As languages go, I don't think it's the worst dynamically typed language by far - it sure beats JavaScript, and Python could learn some things from it as well. But when it comes to long-term maintainability of large amounts of code, it is subpar.
> The point is that # is basically unusable for non-sequence tables because there are no guarantees on which border it will return.
This isn't true, though! The concept of a "non-sequence" table is not something Lua actually has, and if it means a table without an array portion, then `#mapTable` returns 0, no exceptions.
The ambiguity is if there is a sequence, and you stuck `nil`s into it. So definitely don't do that. Want to call that a trap for the unwary? I don't disagree. If you want to make entirely sure that you iterate all the keys, you use `pairs` or `next`. Not much else to it.
Laser-focusing on one case of malformed input is a strange move. If a Python dict doesn't have a key you expect, looking for it will throw an error, in Lua you just get `nil`. Which is better? No idea, but I know which I prefer, and it's the one that doesn't drag my entire program to a screaming halt.
Trying to create an artifical difference between "sequence tables" and "non-sequence tables" is exactly what I meant by wanting Lua to be Python and being surprised or offended when it isn't. There's just... tables. It's one of my favorite things about the language, in fact, because it makes for a very clean expression of ASTs. Metadata goes in keys, child nodes go in the array portion, everyone's happy. Works a treat for XML and HTML too: attributes are keys, child elements are in the array. In a language like Python, you need an `.attr` dict and a `.child` array, because if you just use a dict, you could have an attribute collision if there's a `child` attribute. It's an entire level of indirection which I don't have to deal with in Lua.
Just don't stick `nil`s in the array portion. It's a mistake. You won't be happy. If you need a conditional branch while iterating, use a `false`. It's a cost-cutting measure that was taken to get a language runtime that fits in 70KiB and has fast arrays which are also dicts. I have several reasons why that's a good idea, you've got one reason why it's bad. I say don't do the bad thing. Simple.
Again, I've never had a single bug from bad `nil` insertion in a table. Used the language for years. I've had nil-related bugs, dynamic-type related bugs, and plenty of logic errors. Just never the one thing that people focus on so diligently anytime Lua comes up. YMMV I suppose.
Maintainability wise, I've seen no difference between Lua and Python, having written plenty of both, up until Python added annotations. A gradual type system for Lua would be an excellent addition, Luau-style but compiling to plain Lua. Teal exists, but it ain't quite it.
I linked directly to the Lua language specification earlier which precisely defines which table is a sequence and which isn't:
"A table with exactly one border is called a sequence. For instance, the table {10, 20, 30, 40, 50} is a sequence, as it has only one border (5). The table {10, 20, 30, nil, 50} has two borders (3 and 5), and therefore it is not a sequence. (The nil at index 4 is called a hole.) The table {nil, 20, 30, nil, nil, 60, nil} has three borders (0, 3, and 6), so it is not a sequence, too. The table {} is a sequence with border 0."
> If a Python dict doesn't have a key you expect, looking for it will throw an error, in Lua you just get `nil`. Which is better? No idea
The one that lets you easily diagnose an error at the point where it happens instead of silently producing incorrect output is better, naturally.
> Metadata goes in keys, child nodes go in the array portion
There is no "array portion", they are all keys, just some are numbers and some are not. It's not at all like XML & XDM, where attributes and child elements are completely different namespaces, so count(foo/bar) and count(foo/@bar) are two different things.
> In a language like Python, you need an `.attr` dict and a `.child` array, because if you just use a dict, you could have an attribute collision if there's a `child` attribute.
Python dicts map exactly to Lua tables. If you want to store data in this manner, you absolutely can:
foo = {"a": x, "b": y, 1: z, 2: ...}
But in Python usually you would instead do:
class Foo(list): ...
foo = Foo([1, 2, ...], a=x, b=y)
and then:
foo.x, foo.y
len(foo) # only counts items, not attributes; None is okay!
OTOH if Foo semantically does not have child items, then you wouldn't derive from list, and then len(foo) would straight up throw an exception. And if your index is out of bounds, you again get an exception rather than None.
> Just don't stick `nil`s in the array portion. It's a mistake. You won't be happy.
It's not like anybody is deliberately writing something like {1, nil, 2}. But tables get filled with dynamically computed data, and sometimes that data happens to be nil (often because you e.g. computed an element by indexing another table, and the key was missing so that operation returned nil).
So now you have to always remember that and guard against it, because it is not an error to construct such a table, either. Which, again, is weird if it is "a mistake".
I agree with this, but my point was that I cannot hide the details from myself, and will bring them up when they are relevant and confuse my poor friend just trying to learn Lua.
Definitely very little effort picking up my toy again after a 6 month break
These are implementation artifacts. Knowing how to program does not require any knowledge of them.
> and my old tool looks like a toy.
The great thing about toys is they are very easy to put down for a while and then pick back up again later with very little effort.