Programming language experts told Andrew Kelley, the creator of the Zig
programming language, that having code which could run at compile time was a
really dumb idea. But he went ahead and implemented it anyway. Years later,
this has proven to be one of the killer features of Zig. In the Zig world, we
call it comptime, from the keyword used to mark code required to run at compile
time or variables to be known at compile time.
Which experts? The "comptime" is just macro-expansion from Scheme/Lisp which has been around for a long time. Aren't C++ templates also "code that runs at compile time"?
D has has compile time function evaluation for 15 or 16 years now. It doesn't have a special keyword, either. It's simple - everywhere the grammar has a const-expression, then compile time function evaluation is in play.
I never liked the original D in the early 2000s, especially the anti-C++ marketing rhetorics. But the post-alexandrescu D is amazing. Just last week I placed a delegate bound to a closure into an immutable struct within a child thread's run function and passed it into the main thread via send to be called and it actually worked! What an amazing language. There aren't many 3rd party libraries, though. And whatever exists is highly questionable.
Fortunately, D now has a C compiler embedded into it (ImportC). ImportC will compile C11 Standard code, and automatically make the interface to that C code available to D.
This means, if a C library is available, D can use it.
(There are some exceptions. While ImportC will handle C preprocessor metaprogramming just fine, those metaprogramming macros won't be available to the D code, other than simple #define manifest constant declarations.)
Could it be because some of it, say, places a delegate, bound to a closure, into an immutable struct within a child thread's run function, and passes that to another thread via a send to be called?
I think so, I mean, if it is a one-off POC, nobody is going to build their uber-for-can-opener startup on top of it. But if billions of closures are actively passed around every day by thousands of projects, and lots of bug fixes and features are deployed continuously, why not build your airbnb-for-doghouses empire using that library today?
One thing I'm curious about is, fundamentally it looks like Zig is going to be the more popular language. I don't know why, but that's just my gut feel. It looks like Zig will get the traction that D never did.
So when I see comments like this that say "D also has X feature", I'm curious about why D didn't take off but Zig (likely) will.
D is consistently ranked, by TIOBE index, in the top 30s of programming languages. Zig is not even ranked in their top 100 nor has even achieved 1.0. Appears to be a number of years away from 1.0, which also means all kinds of things can happen and other languages can get more of a spotlight.
In Zig's near category, there are other languages like Odin, Jai, Vlang, Rust, etc... For example, by the time Zig reaches 1.0, Jai (which is in Beta) might be publicly released, others languages might have added some "killer" features, and/or others in development might have also reached production quality.
Zig has quite a long way to go, and if anything may never achieve the popularity or widespread usage of D. Examples of the difficulty of programming languages rising through the ranks, is Nim and Crystal. Both Nim (2008) after 14 years and Crystal (2014), have not even made it into the top 50.
Stop deluding yourself. If we take out TIOBE, and use the IEEE's Top Programming Languages of 2022 (https://spectrum.ieee.org/top-programming-languages-2022), D is still recognized (even higher than the 30s) while Zig, Nim, or Crystal do not even make their chart. That's the reality. The point is, it's hard to climb the charts, and not to underestimate the positions of established languages or that they will suddenly be abandoned.
> Vlang...
To begin with, you are a known troll (with possibly multiple troll accounts at HN) that has spent over an year engaging in slander, lies, and flames about the language almost any time it's mentioned.
Looks like you are the creator or involved with some other language that can't get as much support and popularity, and have taken to being underhanded. It would be advisable for you to stop being obsessed with Vlang. How many years of your life will you waste on such childish antics? You would be better off focusing on making your programming language better, if it's not already too late and its a failure.
As for pushing this bold face lie that the language is a "scam", that's both ludicrous and easily proven false. Vlang has many hundreds of code examples and projects. Easily found at:
The "scam" is you tricking yourself into thinking that your troll tactics are working. Instead, your continual and unnecessary slander is making what you are even more obvious.
The idea behind compile time evaluation is that expressions which are not syntactically constant expressions can also be evaluated at compile time. A function definition can be marked as being available at compile time. When the body of a function f(x) refers to x, that is not a constant expression syntactically. However, an expression like f(3) can be considered a constant expression, since it passes a constant to function that has been marked as a compile-time function.
D uses `enum` to specify a manifest constant, which evaluates at compile time:
int i = f(3); // evaluate at run time
enum i = f(3); // evaluate at compile time
Note that it's not the function that is specified as being run at compile time, it is the use of the function that determines it. There is no difference between compile time functions and runtime functions. The use that triggers a function to be evaluated at compile time is when the function call is in a const-expression.
Obviously, there is no difference between run-time functions that are qualified for compile-time execution, and compile-time functions.
Then, suppose we have all these requirements
- We want a separate compilation model where the code which is calling f(3) at either compile or run time knows nothing about how f is defined, only how it's declared.
- We want not to include the compiled image of f in the target program, if if is only called at compile time, only if at least one call to it somewhere in the program is staged to run-time.
- We want to make sure that if f is called at compile time, nobody can change its definition to accidentally call for run-time semantics like toggle_gpio(FRONT_LED_2).
and that's how we probably arrive at a declarative mechanism like constexpr that goes on the function, rather than trying to orchestrate thing remotely/indirectly by placing a call to the function into a compile-time context.
D guarantees that a function annotated with `pure` will also work at compile time. But compile time does allow calling some impure functions, as long as the path taken through the function's body is pure. This greatly expands the palette of functions able to be executed at compile time.
Not including functions never called for runtime execution in the runtime is a normal compiler optimization. No user input is necessary.
If the function's semantics change so it is no longer runnable at compile time, the compiler will let you know when you try it. If you want to force the detection, just call it at compile time with dummy declaration.
> that's how we probably arrive at a declarative mechanism like constexpr that goes on the function, rather than trying to orchestrate thing remotely/indirectly by placing a call to the function into a compile-time context.
This issue is raised now and then, and in 16 years of CTFE in D it has never ever been reported as causing an actual problem.
I really like zig and think it is potentially a great language. I also used D a little in the past too, but somehow drifted away but some of the comments here make me want to revisit it
Author here, I cannot vouch for exactly the competency level here, but I was paraphrasing a remark Andrew Kelley made in a video discussing Zig. From what I remember he said that many told him that running code at compilation time wasn't a good idea and you shouldn't do it. My understanding was that these must have been people with some experience with programming language design. Why would Andrew Kelley listen to random opinionated guys on the internet?
I will try to lookup this video when I have time. I was a bit surprised that this remark was a such a turn-off. Had I known, I should have looked up the video to get the exact quote.
It was just meant as a bit of a funny dramatic entry to discussing an unusual language design choice for a statically type language. Sure we are seeing more things like const expressions in C++ today, but at the time this wasn't very common.
I don't think that the precedent matters that much in this situation, as to whether people would dismiss such an idea or not.
People can dismiss language ideas and concepts that "have been known and used for many decades" just the same.
For a trivial example: significant whitespace.
So, e.g. "Lisp has had that since forever" is not a guarantee people would be OK with it, unless they also like Lisp - not to mention they might not even know about it in Lisp.
As for C++, many people consider templates - at least as implemented - a bad idea. So some people knowing that it existed in C++ already, and considering it a bad idea to have in Zig is not contradictory.
Sure but then it's like... for example, if you land rockets and your advertisement says "People told me it couldn't be done!", then fair enough. But if you, say, build an electric car and your advertisement says "People told me it couldn't be done!", then ... who exactly have you been talking to?
The phrase "They said it couldn't be done!" is an evocative proxy for technological novelty. If the experts are not representative, the claim of novelty will be misleading.
yeah, though in this case they didn't say "it can't be done" or "was never done" - just that it is a "really dumb idea". At least that's what the post claims they said.
Which is precisely my point. As one other commenter said: "who have you been talking to?". Executing code at compile-time is a great idea in general, maybe those experts were talking about a very particular problem? It's not clear.
Well, to criticize something people start (and, often, end) on concrete examples they've seen it in practice. And for "compile time code generation" C++ templates are an example they'd be familiar with (perhaps the canonical one, as more people know C++ than Lisp).
FEXPRs aren't comptime and aren't macros--the point of FEPXRs is that you hold off evaluating the arguments until runtime or, perhaps, not at all (think short circuit booleans).
This was actually a schism in Lisp and why FEXPRs died out. CS theory wanted to compile things for speed and FEXPRs very much didn't fit into that. This is why macros took over--they were very much targeted at compile time.
The late John Shutt's thesis and work on the language he called Kernel goes into quite a bit of detail and the history about it:
https://web.cs.wpi.edu/~jshutt/kernel.html
Sadly, Shutt never really "completed" the Kernel language--the language is very interesting but is extremely slow and unwieldy in the descriptions he gave. It generates a lot of garbage and extra "environments"; it resists compilation and memoization; it sometimes seems to go out of its way to deal with things that make life very difficult (circular and infinite lists).
A couple of good passes of optimization might have produced something very interesting.
I was personally attracted to it for very small embedded devices (<16K RAM/16K Flash) as it delgates a lot of macrology to interpretation--sadly I just couldn't get the footprint down far enough given all the garbage and environments the language generates. I suspect it's possible, but it would take someone who knew the landscape intimately to figure out what needed to be dropped.
> hold off evaluating the arguments until runtime or, perhaps, not at all (think short circuit booleans).
> infinite lists
Sounds like an early attempt at Haskell. Haskell is lazy by default so you can write your own "if" as a function and it will be just as short circuiting as the built in if. And infinite lists show up in basic Haskell tutorials.
However Haskell combines this laziness with immutability, which is an easier combination to reason about than laziness with mutable side effects.
Compiler technology for lazy languages is definitely impressive, but it's still the case that if you want to squeeze the last bit of performance out of Haskell code it does usually require manually forcing eager evaluation.
Did they die out, or did they just go into hiding? I can't help but think how R - which is really a Lisp with C-like syntactic sugar on top - always lazily evaluates arguments (and captures the original expression + environment for them that the callee can access), which makes it possible to represent all special forms as plain function calls.
Perl 5 had compile-time execution forever. And this feature is used in everyday's code as that feature is at the core of the implementation of exporting symbols from a module into its importer (See module Exporter).
Example: the BEGIN block is executed during the compile phase as the parser encounters it. The code injected in BEGIN blocks can affect how the rest of the file is parsed.
$ perl -E 'exit; BEGIN { say "Hello" }'
Hello
"perl -c" stops the program after compile, but before execution. In the following code the BEGIN block runs, but not "exit 1".
$ perl -c -E 'exit 1; BEGIN { say "Hello" }'; echo $?
Hello
-e syntax OK
0
If one wishes to be pedantic, constexpr is a C++-like language for generating C++ values at compile time and templates are a quite different language for doing the same thing (and some other things). The pitch of the comptime feature is that you can do all of the things you would do with templates and constexpr (and more things due to some limitations being relaxed and type introspection existing) in the language that is similar to the language you use for runtime code. You don't have to do anything in a weird metalanguage.
C++ has been trying to get compile time introspection and synthesis into the language for a while. There is a proposal going slowly through standardization, but it is not yet know when it will be ready (I believe it missed 2 or more release trains already).
Circle, mentioned elsewhere, is a C++ dialect with fully working compile time metaprogramming.
The big thing is the macro language is restricted to generating additional code when it's evaluated whereas compile time expressions are more about reducing expressions into literals.
Not any code that you can program yourself. It's a bunch of template deduction/instantiation rules specified in the C++ standard, baked into the compiler, which you can steer only to the extent that you can grab the template declaration syntax by whatever horns it it is trying to gore you with.
Just commenting here as the author. Don't call people liars. Being wrong about something and lying are two very different things. I was writing that intro based on recalling Andrew Kelley stating in a video that he had been advised against running code at compile time by several people. I could not recall if he ever said explicitly who that was. It was an off-hand comment.
I choose to par-phrase him, and had no idea that this would get anyone this upset and lead to me being accused of being a liar. How much of experts these people where I cannot vouch for. Maybe they were second rate language designers or maybe Kelley referred to older papers advising against it.
Either way I cannot find the exact video anywhere where he made this remark despite looking through several. Does it really matter? Can you say for a fact that no language designer expert ever told Andrew Kelley this? I am not claim this is the general opinion among designers, only that this is what Andrew got told.
If anyone has the correct quote, I'll be happy to update the article.
As much as I despise P5fRxh5kUvp2th's personal attacks on you (personal attacks have no place in a civilized forum and the uncivil behavior of P5fRxh5kUvp2th should be condemned), I do feel tempted to point out that your responses do not restore any confidence in your article for readers like me. Some problems I see in your counterpoints.
> I was writing that intro based on recalling Andrew Kelley stating in a video
When you are writing for a technical crowd, it is good to be prepared with references and evidence. Unsourced information is likely to be questioned. Unsourced information making dubious claims that go against half a century of computer programming and practice is even more likely to be questioned.
> Can you say for a fact that no language designer expert ever told Andrew Kelley this?
That's not a valid counter-argument. Can you say for a fact that Russell's teapot does not exist? That does not make the existence of Russell's teapot any more likely! The burden of proof lies on you to show that the bold claim you made in your article holds up against scrutiny.
It isn’t a claim about the capabilities or features of Zig or any other language. I don’t see how it makes a big difference to the the reader. We are talking about the exact wording was of those advising Andrew to not do comptime and their exact position. I feel this a bikeshedding.
Making this into a huge topic that requires evidence and citations seems a bit over the top. I would say you guys don’t know how journalism works. You equate article writing with scientific journals and papers. They are not the same thing.
I think what is at the core is getting the language description correct.
>Gee, I dunno. Does it matter if I trust the person telling me something? Well shucks, I guess not.
Or you know, the post is about more than that part, and whether those that dismissed the idea were experts (say SPJ or Hejlsberg) or just some programmers that dabble in compilers and had strong opinions, is irrelevant.
Of course by "stopping reading" one would never get to understand whether that's the case or not - god forbid any otherwise good article/post/book was frontloaded with something inaccurate...
Hey there, from where I am standing it seems that you are looking for a fight online. I recommend trying to find a more productive way to spend that energy.
I have done it in the past and what I learned is that it can give a temporary boost of endorphins, but it's very short-lived and it doesn't really help anyone in the long term, including yourself.
Java's SubstrateVM was doing arbitrary execution of code at compile-time, including allocating, and preserving the objects to runtime in 2013 or earlier. Zig only came out in 2016.
Zig goes a fair bit further than c++ afaik. The ideal would be to be able to call run-time functions indiscriminately at compile-time (e.g. fopen) instead of having to build separate other tools that will generate source code...
Though here be dragons: All side-effects evaluated at compile-time, are not run at compile-time!
So fopen, print and all IO or OS functions should be avoided, or even warned.
It might be useful in certain edge-cases, but in my 20 years of compile-time programming, it's almost ever a bug, not a feature.
Jai allows compile time execution of system calls. We (the D team) considered this, and rejected it because we didn't want users to be worried that there'd be some malicious source code that could delete their file system if they just ran it through the compiler.
I'm wondering if the premise was that people compiling other people's code should be prioritized over "actual" users who develop their software using the compiler, who then would need to resort to external tools to do more complicated things, nullifying the protection anyway; or that it's expected that developers routinely include code they don't trust?
Sure, many people do, and some don't. I suppose the point I'm trying to make is that decisions like that make one consider what are the assumptions and target audience of a project. In this case, as a potential user, if I'm looking for a language that's a deadly tool for doing difficult things, intentional limitations feel almost patronizing.
But there is a clear distinction between the build system and the programming language. There is also usually much less code in the build system and it will be reviewed with the idea in mind that it could do something dangerous at build time. Reviewing every line of source code with compile time effects in mind could be very hard and sometimes the reviewer might easily miss the fact that something will happen at compile time, especially if it is hidden behind some api.
There is no important difference between running compile time code using language primitives, or compiling and then running a program as part of the build. You can review the build system, but you won't find the "evil" stuff in there.
I agree with this, but there may still be times when it is needed.
I'm working on a system where users can run a build in an interpreter that kills the build if it tries to do anything they don't allow. An important and trusted project that needs a special permission will probably be allowed to do that, but random packages? Not at all.
The trouble is, malicious people can be very clever at evading such protections. We didn't want to get into an arms race with them. I'm old enough to remember ActiveX.
The code being compiled, when run, could do anything. As for build time, nearly all the build systems that people use in languages like C, C++, Java, Node, etc can do nearly anything.
Saying the compiler can't run user code and make system calls while compiling is is like plugging a whole the size of a bus with your pinky. You prevented absolutely nothing. You just make devs jump through hoops (external tools in their build) to do the things they need to do and in doing so force them to add dependencies which expand their surface area of attacks.
Maybe a command-line switch to say which files can execute code at compile time but still, having worked on a lisp system that ran code and made system calls at compile time it was a huge time saver. Example:
Jetbrains products have started prompting if you'd like to open a new project in "safe" mode that disables the build tooling until you've had time to evaluate the project.
So at least some IDEs are more cautious about this these days.
However, I think that if an interpreter does the execution, the interpreter can check every "instruction" that is executed. If every instruction is checked, how could that be evaded?
Honest question because I can't see how, and I need to.
The context for this is an interpreter that doesn't allow direct syscalls; only syscalls through the interpreter.
It seems even D has that because dynamic allocation could call `mmap()`, and you mentioned that D allows comptime dynamic allocation. So I'm confused.
Yes, the "classic" Jai example from early in development compiles a program which IIRC just prints out your score like "You got 10 points" every time you run it - but during compilation it finds the score by playing a video game, live, so if you want that program to say "You got 100 points" you need to score 100 points during compilation, once you "die" in the game your compilation finishes.
Whether this is desirable is dubious. But fundamentally it's not any more dangerous than Mara's nightly_crimes! Rust proc macro (which during compilation replaces your running compiler with a compiler that believes it is compiling the standard library and thus allows non-stable "nightly" features even though you aren't running a nightly build, then casually alters the compiler output to say that this was fine and there's nothing to worry about), or any number of other tricks which result from being powerful enough at compile time.
I doubt that D is as powerful as people would want and yet manages to ensure this can't happen. There are Rust people thinking about WASM sandboxing for this, but it's tricky.
I think you meant to reply to me, not Walter Bright.
And yes, this is a trillion dollar problem. I may be just some bloke with an idea, trying to make it happen. However, I have the free time to try, so why not?
Any Python script (or library) can run system calls.
In a language like D you could just execute the same system calls in the executable that was compiled. The assumption that you compile something and then not run it doesn't make this any safer, does it?
Pedantically, you are correct. Pragmatically, D is not getting into an arms race with people writing malicious code that can delete the users' filesystem just by compiling it. People do not expect the compiler to need to be run in a sandbox.
It is in private beta. You can send an email to request access. I'm not sure of the frequency or percentage of requests that are accepted, but I have seen some people get added over time.
It's a language written by Jonathan Blow, meant to replace C++ for game development.
reportedly his game "The Witness" was written in it. He has a series of video's where he explores what the language will look like as he's developing it, and has claimed he'll open source it once he feels it's ready. He wants to maintain control until he feels it's no longer half-baked.
IIRC, he demonstrates things like a music player running during compile time. It's a core tenet of his language.
> He has a series of video's where he explores what the language will look like as he's developing it
Probably the best place to see it in it's current form is his development streams on Twitch: https://www.twitch.tv/j_blow. Right now the most active streamer besides Jon is Raphael, who does a lot of work on the Jai compiler: https://www.twitch.tv/raphael_luba. Sadly, the other people I know who stream their work with it haven't been streaming much lately.
Lisp user here, I couldn’t quiet tell from all the Zig syntax, but is “comptime” here like having the whole language available to you to do whatever you like at at compile time like in Common Lisp?
I've never used lisp, but I'm pretty sure you have the right idea. The only limitation is that it can only act on data that is compile-time known, which is a defined concept in Zig.
At the moment you can't do comptime allocations, so everything has to exist on the stack, but you can, for example, embed a file like a config or dataset, and run it thru a parser to turn it into some data structure, which is something you might otherwise work into the build system, running python/perl scripts to generate temporary code files, but now you can cut out the extra dependency by having that code generation in the same language.
This is probably a complete fabrication. Running code a compile time, even automatically has been extensively studied in CS. Just search for partial evaluation or supercompilation.
Yeah, that's a bit of a turnoff. The rest of the article (until the paywall) was alright.
Zig's use of comptime is fairly revolutionary in a c-like language. Compared to the shenanigans required by c++, the entire language* is (should be) available at comptime. Compared to the backtick/quote/macro magic available in lisp... the awesomeness isn't that it's comptime, the awesomeness is that it's type safe.
But, I'd say it isn't even the use of comptime that's particularly noteworthy, it's the lack of pretty much every feature that isn't comptime. This wouldn't be a good idea without comptime being as robust and featureful as it is.
The reason D didn’t really gain traction over the years was because it required a garbage collector. Recently it’s going the other direction via betterC (a mode where you can ditch the GC), but many parts of the standard library still require the GC to run properly.
You can ensure your D code doesn't use the GC by adding the `@nogc` attribute.
But really, all this concern about the GC is misplaced. It's just another tool available. You can use it in D, or use RAII, or your own allocation system. It's your choice as a D programmer.
It is a shame that until now the software industry has not figure out how to reliably perform automatic memory management or GC. Unlike the auto industry the manual transmission is going to a dinasour route compared to the now default automatic transmission.
I am giving this analogy since Walter has a degree in mechanical engineering and he has correctly made the GC a default in D but not mandatory, but the choice somehow is not popular in the software industry.
Sadly it took too much time, and lost momentum. C++ version
int squares1[4] = {0, 1, 4, 9}; // doing it by hand
std::array squares2 = [] {
std::array<int, 4> a;
for (int i = 0; i < 4; i++)
a[i] = i * i;
return a;
} (); // doing it with CTF
The ability to use CTFE to generate strings that can then be fed back into the compiler is a heavily used feature, not just a golf point. It enables the creation of mini-DSLs.
The thing that makes Zig comptime radically different is that types are comptime values, so you can make functions that take types as arguments and return new types as results - it's like C++ templates on steroids, yet conceptually simpler.
You aren't returning the new type there, looks like you just inline that code. Zig lets you create and return new types that way while your example looks like just regular template substitution.
Edit: So the main difference seems to be that types are first class citizens in Zig, you can write a normal comptime function that returns a type and either use it to do more metaprogramming or everywhere you would declare a type for a function or variable. I am not aware of any static typed languages that lets you do this with normal language syntax.
C++ does it with templates, but in Zig you can do what C++ templates can but with if statements and loops.
Please show me an example where it differs. D's metaprogramming is very good at constructing types. A member of the D community tried very hard to implement type functions, but it turned out to be redundant with D's existing capability.
It reminds me of Idris, where the types can actually run programs (at compile time) as long as they can be proved to terminate, which is more often than I though.
If anything it is only a simpler syntax for C++ template meta-programming, not needed knowledge about SFINAE and tag dispatching tricks, most of which are anyway no longer required with the improvements made across C++17, 20 and 23.
If I run `zig build-obj` on that on my i9 Mac, compilation fails with an assertion error. If I run `zig build-obj -target arm-freestanding-gnu` instead, it compiles since sizeof(long) is 4 and sizeof(long long) is 8.
"Zig's use of comptime is fairly revolutionary in a c-like language."
I might phrase that as "statically-typed language". Most (if not all) of the dynamically-typed languages are technically programs that run at the only "compile time" there is and can do arbitrarily whacky things as a result. It is generally a good idea not to think of them that way. I've removed quite a few top-level Perl statements in my time and enforced rules that a module really should be nothing but declarations of constants and subroutines unless there's an absolute emergency. But dynamically typed languages will generally let you read some files off a disk, query a database schema, and hit an HTTP API to build a "module" if you want to.
And in this case the entire language really is available. The limitations tend to come with the increasingly fragile order of operations the modules impose on each other, rather than technical capability.
I don't say this to slag on Zig. Increasing the range of what static languages can do is a legitimately interesting topic. This is to shed light on the general space of programming langauges.
Yeah, there's certainly antecedents. There's not a lot new in 2022.
Sometimes it's legitimately an innovation just to put it into a standard library from the very beginning. I've discussed on HN a couple of times how useful it was for Go to ship with an "io.Reader" definition for "things that emit bytestreams" in the standard library. It has very little to do with language features; many other languages could theoretically have done it. Many languages have a sort of half-usable "file" abstraction, but you can never quite tell what calls will work on which type of thing. Having a single concrete interface in the standard library from day one of the Go language drove the entire ecosystem in a single coherent direction, though, and in Go, you can pretty much count on that interface and can nest it quite deeply without hitting a corner case.
I expect it may be something like that for Zig... it's not that nobody has ever integrated code generation at the compile step for a static language before, it is merely that I'm not sure anyone's ever pushed it out like from the very beginning at this level. I may be wrong. I can come up with several examples of much more dynamic languages doing it. I'm sure out there in the world you can combine Lisp macros with some static type system for the same language. But that would be a bolt-on, not something that was in the standard lib from day one.
Thinking of languages not just in terms of features, but in terms of what community the standard library affords is probably the next major frontier in language design over the next 20-30 years. Yeah, I hope that we still see some sort of revolutionary language that solves all our problems, but I expect to see a lot of languages that don't necessarily have any new features per se but are just better standard libraries from day one.
Just an FYI, in 2008 PLT Scheme had a language with static typing and macros capable of comptime-like behavior, called Typed Scheme. It lives on today in Racket.
But yeah, dynamically typed languages have been doing this for decades, particularly prevalent in Perl circles.
Virgil [1] has had compile-time initialization since 2006. There is no restriction on the code that can be run at compile-time. It needs no specific designation--it is simply the initializers of components (globals) and their variables. The entire reachable heap is optimized together with the program to produce a binary.
I haven't used Zig (yet, maybe one day), but does anyone know if there's a difference between Haxe macros and Zig comptime? AFAIK Circle also offers something similar for C++ (https://www.circle-lang.org/)
Zig's comptime isn't like "macros" in Haxe or Rust or Scala or Lisp which transform one AST into another AST. It's just code that runs when the code is compiled, rather than when the resulting binary is run.
This makes it less expressive, but it also means it is easier to learn and use. (It's a lot like C++'s constexpr)
The interesting part comes from recognizing that by making constructs like structs first-class-values in comptime, you can achieve most of the common things that templates, macros, conditional compilation, etc give you, but with a much simpler feature (which is just running code at compile time)
Lisp and Scheme macros are just code that happens to execute at runtime, too. They don't necessarily need to transform the AST, though that's what they're often used for.
Like with Zig, you are limited mostly by what is available in the environment at macro expansion time; unlike Zig, you can do things that generate allocations.
> It's just code that runs when the code is compiled, rather than when the resulting binary is run.
That’s exactly what Haxe macros are. It’s regular Haxe code that runs during compile, and it has access to all Haxe language features and the standard library.
The only special thing about them is that they can directly manipulate the AST before it gets passed on to the compiler for final code generation.
In a sense, they’re like “shaders” for the compiler.
Macros in Lisp during compilation often extend the compile-time environment and another purpose is to register information in the Lisp IDE (record source code, record source locations, ...).
> but with a much simpler feature (which is just running code at compile time)
Common Lisp does that with the EVAL-WHEN construct, which allows to run arbitrary code at compile time.
Zig's comptime variables had this fairly bonkers behavior that basically evaluate by themselves regardless of the underlying runtime controlflow[1], I thought it was cool, but seems to be disabled on trunk build.
From testing it seems like it is executing both branches, i.e. if (b) { v+= x; } else { v += y; } is effectively v += x + y. That doesn't make much sense so I can see why it was fixed.
> Zig's use of comptime is fairly revolutionary in a c-like language
Not revolutionary. Before Zig came out, which was 2016, there was Jonathan Blow talking about and arguably repopularizing it. Lots of people (who knows how many developers) and languages were influenced by him too.
2014, Jonathan Blow, who made lots of videos and gave lots of talks, before going on to officially promote Jai (2016). And it appears he was playing around with prototypes much earlier and talking about it (from 2014).
Not very useful to us when Jai has not been released to the public. I would like it to, but currently Zig is more complete than Jai, atleast with respect to it being able to be used by everyone
By the time Zig reaches 1.0, Jai may have too or have been publicly released. And Jai's constraints on getting access to their Beta, appears to have been loosening over time.
> zig's use of comptime is fairly revolutionary in a c-like language.
It's not, it's just fairly insecure. comptime is known forever, in static and dynamic languages. Better languages do know about the security and usage implications and do something about it.
Very little of the language is available at comptime. Allocation is unavailable and the zig developers have declared this a non-feature. That either locks out or requires redundant implementations of a great many things.
If you look at puffoflogic's comment history you can see they are maliciously claiming incorrect things about zig over and over. Here I am refuting them again.
FBA was usable last time I tried zig, but I don't count it because if you knew ahead of time how much you needed to allocate, well then you didn't need an allocator at all. I.e. I define "allocator" to mean "dynamic allocator". Note that the buffer provided to FBA would have to be provided at the very top.
Fixed-size allocators are not composable. You cannot write any kind of comptime routine which calls further routines which need to do allocation, and how much allocation they do is based on arguments to your routine - except by forcing your caller to calculate how much allocation needs to be done. But that leaks your implementation details to your caller, largely invalidating the point of making some kind of comptime routine in the first place. Therefore comptime can be used as a toy, or in C-style language-enforced NIH mode (where you [re]write every single routine you need yourself, or at least know the intimate implementation details of all of them), but not in a serious programming stance.
If I remember right, FixedBufferAllocator requires you to set an upper bound for the amount of memory you allocate -- you don't have to get the number exactly right or anything.
Runtime allocators also have a limit to how much memory you can allocate. It's usually higher, but there's still a limit. Your computer doesn't have an infinite amount of memory.
> if you knew ahead of time how much you needed to allocate, well then you didn't need an allocator at all.
Sure, if it's all code you're writing yourself then it's doable to avoid using an allocator. If you want to reuse existing code that does expect an allocator, however, then being able to use a FBA is handy.
You can just return arbitrarily large arrays or whatever, but this is probably not what you want because it's a separate implementation from the one that takes an allocator. I think if FBA works then you have the building blocks to make one that does not fail by running out of memory e.g. by doing the FBA thing but getting a bigger backing array whenever you need it.
Overall I am not the right guy for this conversation because I am too used to never calling mmap at runtime either.
> it's a separate implementation from the one that takes an allocator
I want to give zig a fair assessment so yes, this is absolutely right. This is why I have elsewhere said that zig is basically two separate but superficially similar languages. You can probably accomplish just about anything in comptime zig, if you're willing to completely reimplement it from the core language up. The limitations I mention come in if you try to write code to be used in either comptime or runtime zig. So comptime zig has no growable arrays, no dictionaries, no string searching, no regexes, etc. etc. until these are reimplemented specifically for comptime zig. Mostly this is just sad, that a decent idea was implemented with such a terrible downside that would be so easy to fix: just add allocators. Just. Add. Allocators. But no, the zig team has explicitly rejected this.
> Just. Add. Allocators. But no, the zig team has explicitly rejected this.
I'm torn on this. To me, comptime is the C preprocessor but sane. Anything you can do above that is bonus. I don't need comptime to be totally Turing complete.
And, I do NOT want macros in the language. Macros are a bottomless well of suck. Lisps/Schemes still tie themselves into knots with macro hygeine. Rust's procmacros are terrible and debugging them is worse--just try figuring out what some intermediate expansion did.
The point of Zig is to NOT be Rust/C++/etc. The point is to be C with a bunch of the sharp edges and corners filed off. It's not for everybody. If you want web services, use Go. ML/AI--Python for ecosystem. Rust for strong safety guarantees.
Zig has some nice things while still remaining relatively small. It, like most modern languages, understands that ecosystem is important. It does a better job than almost everybody at cross compiling. comptime gets you a lot of the benefits of templates while dodging lots of the suckage.
However, at some point, you have to declare "Stop, that isn't going to be part of the language" or you wind up with the ungodly mess that is C++.
You can string format using `std.fmt.comptimePrint`[1]. For example, combined with features like `@compileError`[2], it allows you to make your own error messages (useful for generic functions)
I wrote a Sudoku solver (my go-to "learn a new language" project) in Zig and the compile-time features were extremely useful.
entirely at compile-time, I was able to generate lookup tables for "given this cell index, give me the indexes of the cells in its row, column, and box". and I simply looped over that for each board size I wanted to support, from the standard 9x9 all the way up to 64x64.
another useful feature was support for arbitrary-sized ints [0] and packed structs [1]. for a 9x9 board, a cell can be represented with a 9-bit bitfield for possible values; a uint4 for the actual value; and a single bit indicating whether the value is known. Zig will happily pack that into a struct that occupies only 16 bits, and handle all the bit shifting & masking for you when you access the struct fields.
this meant I was able to represent the state of a 9x9 board in just 162 bytes - a flat array of those packed structs. and the exact same code could operate on the ~28kb of state needed for 64x64.
I am not good enough of a programmer to understand all the meta/compile time programming stuff.
So thank you for this actual concrete example of it being used. But why is that useful? Why regenerate a database every time you compile? Now that you have compiled the program once, can you not just delete that code and reduce your compile time to a fraction of what it was?
You rely on a compiler to produce an executable, right? Well there are many "phases" that the code goes through before becoming machine code. "Compile time metaprogramming" is having control over one of those phases available from the language itself. Exactly how it is implemented is, well, implementation and language specific. But the fact that you can do it is quite useful. You can not only write programs, you can also write programs that transform or generate programs. The full extent of what you're allowed to do is implementation-dependent.
In Lisps, for example, that phase is called the macro-expansion phase (of which there can be layers of) and generally acts as a source-to-source transformation from s-exp to s-exp (possibly with side-effects). Here's an example in a Gambit Scheme REPL:
> when
*** ERROR IN (stdin)@8.1 -- Macro name can't be used as a variable: when
> (when #t #f)
#f
> (pp (lambda () (when #t #f)))
(lambda () (if #t #f))
So you can see the 'when' macro expands to an 'if', and that has to happen _before_ evaluation. So the REPL is in fact a REEPL (Read Expand Eval Print Loop).
It's much more convenient than running an external tool to generate LUTs, which is how you'd do it in C(++). You shouldn't regenerate the database every time you compile, that's what incremental compilation is for. But it is very convenient to integrate code-generation into the build system. What happens when you find a bug in the code/LUTs and you've deleted what generated it?
It's useful due to convenient notation. Otherwise, you'd need to write a separate program that generates the code and call it using a separate Makefile target. (And in that case, you might check in the generated code; this is commonly done for Go packages, so that downstream dependencies don't need to run the code generator tools.)
Adding a compile-time parameter to a function is considerably more convenient than writing a code generator. Zig's compile-time performance might be a problem someday, but there is likely not enough Zig code yet for this to matter. Perhaps they'll add caching if it ever turns out to be an issue.
I think it's supposed to be cached in the same manner other things are cached, but the caching does not work great for us so far (and this may be because we are stuck on stage1 due to heavy use of async).
I have had comptime lookup table generation and similar stuff crash the compiler or take a long time to run. I fell back to external code generators in these cases.
The Go community has a rule of committing both the code generator (for later regeneration and maintenance) and the generated code (to avoid rebuilding it every time).
Hah, I had the same learner project for Zig. I chose a different board memory layout for some increased throughput on my hardware, but generating relevant lookup masks at comptime was super beneficial with that approach too.
On the topic of comptime lookup tables, transcendental discretizations are popular in graphics programming (rescale sin/cos to fit appropriately into some integer type, make an array of them to turn transcendental approximations into array lookups). Those are easy to make flawless and fast in Zig.
Another thing that enables without too much song and dance is compile-time duck typing. You can write a function that easily handles a call like randint([][4]@Vector(12, u8), .{Min(0), Max(13), my_allocator, Shape(.{12345})}) which would allocate that type with an appropriate size and fill it with random integers within the prescribed bounds. It's perhaps a bit more verbose than a truly dynamic language, but comptime type introspection enables that sort of powerful control flow without especially burdensome implementations or any runtime performance cost (ignoring cache effects and whatnot), and the whole thing is statically guaranteed to work (within the bounds of what Zig actually promises).
> the exact same code could operate on the ~28kb of state needed for 64x64.
64x64 seems to need 64 bits for values, ie 8 bytes before we consider the known case, but there are 4096 cells. So that's surely 32kB of state immediately ?
It seems to me that if we don't care to distinguish whether we "know" a value for which there is only one possibility, we can save encoding this, so the 9x9 board only needs 9 bits per cell, the 64x64 board only needs 64-bits.
In this case we can answer "known?" as is_power_of_two() and we can find the known value using e.g. trailing_zeros() which is cute.
Just out of curiosity, in your opinion what's the benefit of using a packed struct and all the generated bit fiddling compared to just a regular struct? Just better memory consumption?
ahh, I might have misunderstood the question - I wasn't sure if they were asking about Zig in general or my solver in particular.
yes, normal structs would have worked fine - like I said, I was learning Zig in the process, so I was going out of my way to experiment with its unique features.
That's pretty cool! What other learn-a-new-language projects do you (or others) have? I like to write a very basic HTTP 1.0 server when trying out a new language.
"In which people who have never used a language with compile-time expressions try Zig and think it's novel"
Not knocking Zig, I think it's swell, but it wasn't near the first language with this feature. D comes to mind, and C++ has it now with "constexpr" and "consteval".
The purpose was to pick interesting features to talk about and select one language for each feature which is a good exponent for that features. For instance I used Smalltalk to cover image based development: https://erikexplores.substack.com/p/what-makes-smalltalk-uni...
Yet Smalltalk was not the first language supporting image based development. I would pick Julia to talk about multiple dispatch. Likewise Zig is picked not because it is the first doing comptime but because that is a central feature of Zig and it is a language which really puts emphasis on it.
I'd be curious for examples. In a weird spot where I'm decently familiar with C++, D, and Zig and it would make for good blogpost material to compare comptime implementations for the same thing across languages IMO.
> Programming language experts told Andrew Kelley, the creator of the Zig programming language, that having code which could run at compile time was a really dumb idea.
Really? I can't believe that! Running code at compile time is as old as Lisp! And it is present in some form or other in some other popular programming languages too. Like constexpr and templates in C++.
Zig tutorials and learning materials and on-boarding experience needs a lot of work. Compare with something like Kotlin, so well designed is its introductory material that one could start doing useful things with Kotlin within a few hours.
Also I find Zig's choice to use abbreviated keywords rather cryptic, for example using 'fn' instead of 'function' only hurts readability I think.
Andrew Kelley said they'll only start writing the actual documentation once they hit 1.0 so they don't need to update it all the time. I personally find parts of the language unintuitive enough that I'll just wait.
I don't get the problem with `fn`, though. Pretty much every language uses abbreviations like that and I've never heard anyone complain about it: enum, char, int, uint, def, ...you get used to it quickly, you need to build a mental model for them either way.
I'm sorry if I misinterpreted or misremembered something you said, but this was my impression from an answer regarding the state of the stdlib docs, that there'll be more effort put into them after 1.0.
If that's not the case (or maybe plans changed?) I stand corrected. It's definitely not my intention to spread BS.
May be 'fn' was a wrong example to highlight, but my general impression was that there was a preponderance of such abbreviations in Zig, as someone who has spent some time in Java shops, where abbreviations are largely avoided in naming things.
I'd be perfectly happy if the answer to TFA's question was "nothing". Zig has the minimalism of something like C89 or Lua. It has a syntax like C. It has compile time execution (as per TFA) like all sorts of languages mentioned in the comments. Etc. There isn't really anything particularly unique about it. Who cares? It feels to me like zig is an example of something that is greater than the sum of its parts.
Really the only thing I don't think I've ever seen before is how zig passes around memory allocators. And that's probably because I'm not a systems programmer by trade, so I'm less familiar with that sort of thing.
> Zig will compile different variants of maximum for each case, where maximum is called with a different set of argument types
How does Zig handle the monomorphization problem for generics in library APIs? In other words, give the `maximum` function from the blog post, if I want to distribute a binary but make that function available to clients of my binary to call with different types, what does Zig do?
Terra has had this for a long time. Anything you wrote in Terra could be invoked at compile time just the same using LLVM's JIT stuff. Made writing game tooling a breeze.
I considered writing about Terra as ab examples of a similar language but I felt the fact that Terra did this at JIT time made it too different to compare. In that regard it is not all that different from Julia. That Zig does this AOT is significant I think.
I'm not sure what you mean. Terra does JIT to do AOT compilation. You run either Lua _or_ native code (JITed) at compilation time to produce Terra code that ultimately gets emitted into the final executable.
Terra is such a an overly recycled term. It was not obvious if this was some kind programming language associated with the Terra block chain and Terraform Labs, which is also overloaded with Terraform, the infrastructure-as-code tool by Hashicorp.
I'm not sure what your point is. Terra is a programming language. In the context of programming languages, I would assume finding the correct Terra would be trivial.
I think Zig is pretty cool. I have played with it a bit. My main complaint I think is that the toolchain is pretty hard to get working right in windows, though its a breeze on linux. The only reason I would care about windows, is that I think it looks like a great language for game development since its fast, pretty easy to write, and interops with C/C++ pretty easily.
A sufficiently expressive (and safe) type system is Turing complete, and many languages have that. They don’t all feel ergonomic, but Haskell and Rust are pretty good. If you’re into it, you should check out Idris (lang) and Idris2. Both very cool languages that support dependent types (types that depend on values).
I can't speak to the quality of Zig's compile-time compilation, other than to comment and say the article is taking liberties with its use of the word unique. As other commenter's have noted, many languages have the feature. Instead I'm going to talk about a bunch of cool and weird compile-time code implemented by people much smarter than I.
If anyone reading this is interested in compile-time computation, you owe it to yourself to read Paul Graham (of HN)'s own work on compile time programming, "On Lisp"[0], or to take compile-time ideas to the next level, Doug Hoyte's "Let Over Lambda"[1] (LOL). Lisp languages have a long and interesting history of compile-time computation via their various macro systems, and given the first-class inclusion in most Lisps, a greater variety of ideas have been explored regarding compile-time computation.
A few interesting examples:
LOL's "Pandoric macro"[2] lets you monkey-patch closure values. It's absolutely bonkers and would almost certainly be pathological to introduce to a codebase, but it's an example of pushing the boundaries of what's possible.
Common Lisp's object system, implemented via macro[3]. To be honest, the entire Common Lisp language is a masterclass when it comes to learning about macros (I can't avoid mentioning Scheme's hygeinic macro system and Lisp-1/Lisp-2.)
A special non-Lisp shout-out goes to Rust's Diesel library for embedding SQL DDL queries into macros[4] which is not something I've personally seen before.
Clojure has a few interesting (and practical) macro libs, particularly core.async, which is an implementation of CSP (similar to Golang's channels AFAIK), it embeds perfectly into the existing language and extends its capabilities even though it's merely a library. Another interesting lib which comes to mind is Meander[5], which uses term-rewriting to provide transparent data transformations. Think declaratively interacting with data structures at compile time, and the library figuring out the best way of turning it into imperative value manipulation code.
Speaking of Lisp, macros and compile-time evaluation, it's amazing that there isn't a standard macro macro-time which evaluates its argument and replaces it with the quoted object resulting from its evaluation. It's so easy to write!
Blatant, arbitrary compile-time evaluation (other than code transformation) is not that common in Lisp programs; which could be why such a macro wouldn't be used a lot.
For complicated, calculated constants, the Common Lisp defconstant has certain semantics that work in that space, where responsibilities are foisted onto the programmer: [A]n implementation may choose to evaluate the value-form [of defconstant] at compile time, load time, or both. Therefore, users must ensure that the initial-value can be evaluated at compile time (regardless of whether or not references to name appear in the file) and that it always evaluates to the same value.
In Common Lisp, arguably, defconstant is the mechanism of choice for blatant compile-time evaluation for actually obtaining a value from a complex expression which is thereafter treated as a constant. And its semantics isn't precise enough for the task of retrieving something from a specific environment. Using defconstant to access a file in the compilation environment won't work if the evaluation is done at load time.
In any case, defconstant looks very similar to these constant expression mechanisms that are cropping up.
