A cursory look at meta-programming in Nim

[ldlework]

http://blog.ldlework.com/2015/05/01/a-cursory-look-at-meta-programming-in-nim/

[hzhou321]

Or, with a text based meta-layer (MyDef):

    import macros

    $(for:A in A,B,C)
        proc $(A)() = echo "$(A)"

    proc execute(order: seq[int], callbacks: seq[proc]) =  
      for i in items(order):                              
        callbacks[i]()

    execute(@[0,0,1,2,1,2], @[A, B, C])

[kazinator → hzhou321]

You might as well use the POSIX shell to write your code:

   cat <<END
   $(for x in A B C; do
       echo "proc $x() = echo \"$x\"";
     done)
   
   proc execute(order: seq[int], callbacks: seq[proc]) =  
     for i in items(order):                              
       callbacks[i]()

   execute(@[0,0,1,2,1,2], @[A, B, C])
   END

[rockboe → kazinator]

It's great that both Nim and Bash are so easy to read and write. The beauty of Nim is it makes writing performant C code as easy as writing Python.

[hzhou321 → kazinator]

It worked!

Now how do I incorporate this into my tool chain (Makefile)?

[kazinator → hzhou321]

Believe it or not, I actually did this years ago, and I have the Makefile somewhere. [... search ...] Found the rules!

  %: %.ct
        cpp -P $(shell env | sed -e "s/.*/-D_ENV_'&'/") $< > $@

  %: %.st
        ( export __FILE__=$< ; echo "cat <<!" ; cat $< ; echo "!" ) | bash > $@

  %: %.mt
        rm -f $@.err
        m4 -D__FILE__=$< $(shell env | sed -e "s/\([^=]*\)=\(.*\)/-D'_ENV_\\1=\\2'/") $< > $@
        if [ -s $@.err ] ; then cat $@.err ; exit 1 ; fi
        rm -f $@.err

Explanation: this gives you three "flavors" of preprocessing. ".ct" files are "C preprocessor templates"; they are put through cpp. ".mt" files are M4 templates: M4 is used. And ".st" are shell templates; they use shell here-doc syntax.

The shell templates do not have to have any "cat <<" or "!"; this wrapping is added by the Makefile. Of course, you can't have a line consisting of ! in these files; you have to escape such a thing if it occurs.

Additionally, for .ct and .mt files, all environment variables are turned into macros in that language, with their names mapped to the the _ENV_* namespace. For .st and .mt files, the __FILE__ macro is established which expands to the file name (the C preprocessor has this built-in).

I used this in an embedded Linux distro that I build from scratch (targetting MIPS embedded hardware). It was used for generating some of the textual materials in the target file system tree. For instance /etc/hosts was generated from a .st template, and populated using a $(for ...) loop.

Yes, I do Lisp and I do stupid. :)

[hzhou321 → kazinator]

Nice!

> Yes, I do Lisp and I do stupid. :)

Why stupid? I think that is straight solutions for simple problems. MyDef is basically the same idea but with complete programming layer to achieve almost anything. I have been using MyDef for all my programming (in all programming languages) for 10 years now, solves all my syntax problems. It doesn't solve language problems therefore I still choose language to suite problems. However it turns out that programming language minus syntax features are much smaller and the remaining criterion often boils down to library availability, dynamic type vs static, or memory management. Syntax differences can be naturally dismissed.

[kazinator → hzhou321]

Looks like your MyDef is bootstrapped using itself; it seems to be written in "MyDefized" perl.

Bravo; it is much more readable than regular Perl.

[hzhou321 → kazinator]

Thanks.

MyDef is not really a programming language, but a meta layer. Sounds like a new concept but it is more like cpp or m4 in the toolchain point of view. As such, it can work with any programming languages. e.g. In the Nim example, every thing written is Nim, except the macro; and MyDef itself is actually in 100% Perl (the bootstrap folder is actually a pure Perl package). However, specific extension/plugin also can be written for specific language to customize and gain special power; I did that for Perl and C.

[HOLYCOWBATMAN → hzhou321]

Hey i can do that in $php too!

    function execute($order, $callbacks){
        foreach($order as $i){
            $callbacks[$i]();
        }
    }

    function abcProcs(){
        foreach(func_get_args() as $f){
            eval("function $f() { echo '$f'; };");
        }
    }

    abcProcs("A","B","C");
    execute([0,0,1,2,1,2], ['A','B','C']);

edit:

you don't have to quote the function names (just to show they are really functions and not closures):

    execute([0,0,1,2,1,2], [A,B,C]);

[rockboe → HOLYCOWBATMAN]

Most scripting languages have miserable performance characteristics and lack native types. The beauty of Nim is it isn't a scripting language, it just looks like it is.

Nim isn't C/C++, but it performs like it is C/C++. Nim isn't Golang, either, but it compiles programs like Go does, down to a self-contained binary with no dependencies. You can even statically link a Nim program against musl libc to ship without even a dependency on glibc.

This puts a completely new spin on writing performant code - just skip the C and write your code in pure Nim. It's a lot like writing Python except you produce comparable results to what you would get from writing your code in C or Go.

[kazinator]

Way too cumbersome, sorry.

TXR Lisp:

  @(do
    (macro-time
      (defun abc-proc (n)
        ^(defun ,n () (pprinl ',n))))

    (defmacro abc-procs (. n)
      ^(progn ,*[mapcar abc-proc n]))

    (abc-procs a b c)

    (defun exec (order callbacks)
      (each ((i order))
        [[callbacks i]]))

    (exec '(0 0 1 2 1 2) '(a b c)))

  $ txr test.txr
  a
  a
  b
  c
  b
  c

Variation on exec:

  (defun exec (order callbacks)
    (mapdo (op [callbacks @1]) order))

[def- → kazinator]

You can do it a bit shorter and easier in Nim as well:

    import macros, strutils
    
    macro abcProcs(n: varargs[expr]): stmt =
      result = newStmtList()
      for c in n.children:
        result.add parseStmt("proc $1 = echo \"$1\"" % $c)
    
    abcProcs("A","B","C")
    
    proc execute(order: openarray[int], callbacks: openarray[proc]) =
      for i in order:
        callbacks[i]()
    
    execute([0,0,1,2,1,2], [A,B,C])

[andybak → kazinator]

Yes but on the plus side there's a lot fewer parentheses.

[kazinator → andybak]

If you're counting tokens, you should count all of them: all symbols and punctuation, as well as any whitespace outside of a string literal that cannot be replaced by a single space character without affecting the syntax.

[andybak → kazinator]

So APL is your favourite language then? :)

You should somehow measure cognitive load. Of course it should be measured on someone fluent in that language - but there should be an adjustment to factor in the cost of becoming fluent. :)

[kazinator → andybak]

APL is about being obsessive with character count. We can turn any language into "APL" by giving the core functions one-letter names and using the lack of whitespace between them in some semantic role like chained application or whatever.

Hey look, "FUBAR". Take the first item, unwrap the list, bind it to function A as the first argument, then reverse! This kind of character-level reduction I'm not interested in at all; It's computer science puberty.

I never said "way too long" but rather "way too cumbersome". What is cumbersome in the Nimrod is the awkward encapsulation. For example, we have to create a special kind of list of statements with a special constructor. This list is a "bag-like" container with an .add method. Yuck!

[ldlework → andybak]

You were down-voted but I secretly agree.

wait - oops. :)

[shasta]

I think the author should post a follow-up after he's tried to do substantial work with this style meta-programming. My guess is that you want to keep templated code to an absolute minimum.

[ldlework → shasta]

You were down-voted but I actually agree. I did conclude the article by saying that templates seem really nice for closing the gap on refactoring code that not even generics could wrap up. So the intention was to convey that templates are really a last resort secret weapon. They just also seem to be pretty easy to understand :)

[kbd]

It's a small thing, but I'm so glad they renamed it to "Nim". Its prior name carries such a negative connotation, while just shortening that gave a name that's cute and simple.

[jwecker → kbd]

Fun fact- Nimrod means mighty hunter. The negative connotation came from early Bugs Bunny cartoons where he applies the name sarcastically to Elmer Fudd.

[evilduck → jwecker]

Kind of. https://en.wikipedia.org/wiki/Nimrod

There's definitely a much older negative connotation than the Bugs Bunny usage.

[jessaustin → evilduck]

You don't like big waterfall projects like the Tower of Babel? You must be one of those microservices people... b^)

[sigzero → jwecker]

Also a very cool future sentinel in the X-Men comics.

[Dewie3 → jwecker]

The voices of American programmers are loud and many.

[platz → Dewie3]

At least we still have Coq

[vezzy-fnord → kbd]

I'm one who actually preferred Nimrod, but it's done by now.

[david-given → vezzy-fnord]

Ah --- I didn't know this was Nimrod! I remember looking at that a few years ago and thinking 'this is awesome but not quite ready yet'.

I need to have another look at Nim. It even looks like it's in Debian!

[Dewie3 → kbd]

`git` also carries negative connotations. But at least that was intentional.