I believe this suffers, as do several other CHIP-8 emulators I've seen, from the early mis-documentation of the SHR and SHL instructions. Forgive me if I am reading this incorrectly; I am not used to Lisp:
Since there is only one reg argument, it seems these are shifting the source register in place. It should put the shifted value of the source register into the target register (8XY6: VX <- VY>>1) This was recently clarified in an exchange on the retrocomputing forum [1].
This is awesome! A couple of days ago, I'd asked an asinine comment[0] about building a NES emulator & someone had suggested to start with a CHIP-8 emulator first. For someone who's looking to write the same in Clojure, I'm super glad that Steve wrote this guide!
This is very neat, but he's right that some won't like the unhygienic variable definitions in the with-chip macro. ;) If I were to do this in Scheme, I'd use a "parameter" called 'current-chip' (think of it like a dynamically scoped variable) and write getter/setter procedures that operate on the current chip. This way everything would remain hygienic. I guess it's just another example of the usual difference in software design between CL and Scheme programmers.
I'm excited to see this blog series develop to the point where there is graphical output.
It is not unhygienic. The with-chip macro is understood as creating a lexical environment in which certain local identifiers are visible. There is no accident there; it is documented. You use the macro to obtain exactly that; as it's raison d'être.
> If I were to do this in Scheme, I'd use a "parameter" called 'current-chip' (think of it like a dynamically scoped variable) and write getter/setter procedures that operate on the current chip.
And then what, use the flimsy hack known as "fluid let" to locally rebind this parameter to a desired chip instance? Let's avoid lack of macro-expansion-time hygiene, by introducing run-time variable mutation?
Another point grandparent is missing: when macros of this type introduce the well-known symbols into a scope, those macros themselves are not "surprised" when user code shadows those same symbols. That is to say, those macros usually provide the symbols to the user code, but do not themselves rely on those bindings (while that is possible, it would be a correctable mistake in the implementation).
Lack of macro hygiene is just peachy keen among Scheme programmers as long as you are explicit about which identifier bindings can leak into, or out of, the macro definition from the macro call site.
Scheme standards before R6RS[0] were a rough consensus on which language features everyone could agree on as useful additions. That's why only syntax-rules made the cut, not because Scheme programmers object to the ability to break hygiene. Implementors disagreed on which mechanism for breaking hygiene was most "correct", so none made the cut and unhygienic macros weren't standardized.
[0] R6RS, in a break from Scheme tradition, adopted a systemd-esque "come up with a standard for these needed features, any standard, and everybody else needs to fall in line whether or not they like it" approach.
>Lack of macro hygiene is just peachy keen among Scheme programmers as long as you are explicit about which identifier bindings can leak into, or out of, the macro definition from the macro call site.
I haven't encountered a single case where I didn't prefer hygiene over an unhygienic solution.
I'm not a lisp master, so please take the following with a grain of salt:
One case where you'd want unhygienic capture is with anaphoric macros, which introduce a binding (usually named `it` as in english). For example, using an anaphoric if called aif, we can transform:
(let ((result (big-long-calculation)))
(if result
(foo result)))
Ah yes, that is a use-case. Though, if I were to implement something like that, I would add an extra set of parens around (big-long-calculation) and specify the name to bind the value to:
Yeah, anaphoric macros are a neat trick, a novelty. They're rarely useful in practice. Additionally, any time one would be useful in practice, you turn your anaphoric construct into a binding construct, and suddenly you have hygiene. To lift an example from @davexunit up-thread:
(aif (foo)
(bar it))
becomes instead
(let-if (it (foo))
(bar it))
A minuscule amount of extra typing to be explicit, and now you not only have hygiene, but you can properly nest such forms and even generalize the form to multiple bindings:
Anaphoric macros are used frequently in the HN source code, so "rarely useful in practice" strikes me as incorrect on both counts. Used well, they make code both shorter and more readable. That's a win-win. The arguments against them, by contrast, seem largely theoretical. Win-wins, a.k.a. having your cake and eating it too, are rare in programming, so those of us who think this way will choose that over theoretical purity any day.
These arguments boil down to little more than personal preference and, above all, habit. Anaphoric macros are the kind of thing you like if you like that kind of thing. http://quoteinvestigator.com/2015/09/09/like-sort/
To clarify, I mean that the percentage of macros which are usefully anaphoric is small, not that any given anaphoric macro isn't useful in practice. A macro like `aif` is useful quite often in practice, but there aren't very many macros like `aif`.
You're right that it's a personal preference. My own preference is to always be explicit (which also explains my relative distaste for Perl...). With my students, I'm constantly emphasizing that "programs must be written for people to read, and only incidentally for machines to execute"[1]. I feel as though explicity goes hand-in-hand with legibility, while I'll admit it's really just a matter of learning what `aif` does, the same way you have to learn what anything else does, I'd argue that a programmer who has never seen `let-if` before would be able to decipher its meaning from the context, whereas a programmer who's never encountered `aif` will be wondering `wtf`.
Oh yes, it's true that there are aren't many different anaphoric macros, though I occasionally run across a useful new one. But the core ones (in Arc: aif, awhen, aand) are useful in many places.
I don't see that famous SICP as applying to this the way you do. Anaphoric macros obviously don't exist to make programs easier to execute—they're there entirely for humans to read, though obviously the humans don't agree about them.
To my mind, wanting a program to be readable by someone who hasn't learned its idioms is like wanting a newspaper to be readable by someone who hasn't learned its words. Some words are easy to guess from context, and that's nice. But others require learning, especially when a text isn't in your native language, and that doesn't make them unreadable.
Well, newspapers are written at a middle-school reading level, or so I'm told.
I disagree that anaphora are for readers. I think they're primarily for writers, as a convenience to save them some typing. The extra time spent reading `let-if` vs. `aif` is negligible, a small fraction of a second at worst.
But, perhaps the disagreement comes from a fundamental difference in what each of us considers "readable". In my mind, readability is measured by the percentage of readers who comprehend what was written without aid. The amount of obviousness that such a goal demands not only requires explicity, but in my opinion it increases the ease of comprehension even for experts (so long as it's not something surprising to them -- e.g., I wouldn't go out of my way to write `let-if` in Arc since (iirc) it already provides `aif`, and that's what's expected by Arc programmers).
On the other hand, I get the feeling that you rather view readability as the maximum of the ease of which each reader could comprehend what was written (not to put words in your mouth, please correct me if I'm wrong). I think that's an equally valid conception of the term, just not one that I share (or even really thought about before, to be honest).
Anaphoric macros certainly don't exist to save typing; they would be worse than useless if that were true, and the cost of typing is negligible in any good Lisp program in any case. I think you might be missing the information they add.
An 'awhen' is easier to read than a 'let-if' because it encodes more information. It communicates something like: "The following form is organized around a single expression of interest. We're going to evaluate that expression and, if it's non-nil, do something with it (where 'it' is literally the name assigned to the value). We needn't name it explicitly, because the expression and what we're going to do with it are both simple and straightforward".
That's a lot of extra information, and it makes 'awhen' semantically much narrower than 'let' (or 'let-if'), which are more general and thus more expensive to read. To read a 'let-if', you must process not only the expression being evaluated but also the name being assigned to it, and there's no implicit guarantee of a single, simple organizing principle to the form. If you get used to them, you'll notice yourself feeling a bit easier when encountering an 'awhen', 'aif' or 'aand' because you instinctively know you won't need to spend as much mental energy to grok it. As sctb put it in a conversation, "When I see an 'aif' or 'aand' form, I have less anticipatory tension than if I see a more general construct like 'let'."
It's a bit like the difference between a simple 'repeat n times' macro and, say, a 'do' loop in Common Lisp, which is much more general and thus takes more energy to read. Consuming less of the reader's mental energy is key for readability—it frees up cognition for understanding the rest of the system.
This all has to do with making code easier to read, not write. Yes it requires a one-time effort to learn what 'awhen' means, and yes the code will be more obscure to anyone who hasn't taken a minute to learn that yet, but those are tiny costs compared to the lightness that's paid back again and again when the construct is used well.
I wouldn't put anaphoric macros in front of a beginning programming student, because learning basic programming is already so hard that any added cost is nontrivial. They're struggling to get up the hill as is, so one shouldn't put any extra tools into their backpack.
Moreover, beginning students work only with simple programs where the value of a tool like anaphoric macros doesn't add up to much. But building complex production systems is another matter. There complexity is the arch-enemy, and any tool that lightens cognitive load is an asset.
I have designed an anaphoric if which chooses the "it" antecedent differently from PG's aif.
Moreover, if the expression identified as "it" is a place, then "it" is an alias for the place, whereby the place is evaluated only once even if multiple occurrences occur.
I disagree that `aif` is semantically any narrower than `let-if`, or that `awhen` is narrower than `let-when`. I'd argue that they have very nearly the same semantics, or at least the same semantic structure, modulo alpha conversion. Furthermore, `it` is implicit, but that does not mean it gets to escape mental processing. It's an extra thing that must be learned and remembered, and a mental binding must be established whenever the form is read anyway.
I also feel confident that pg put the anaphoric macros into Arc primarily to reduce typing, given his introductory article about the design of Arc:
> It would not be far from the truth to say that a hacker about to write a program decides what language to use, at least subconsciously, based on the total number of characters he'll have to type. If this isn't precisely how hackers think, a language designer would do well to act as if it were.
However, I'd be silly to deny that less writing does not lead directly to less reading, and I'd be disingenuous not to also include a later passage:
> But the cost of a long name is not just the cost of typing it. There is also the cost of reading it, and the cost of the space it takes up on your screen.
(Somewhat tangentially, I agree that names in Lisp can sometimes get comically long, but I think there's a happy medium between 20+ character names and 3- character names.)
I personally find the more explicit variable names offered by `let-if` (and the more descriptive macro name itself) to aid in reading. I also disagree that `aif` has a "single, simple organizing principle to the form" whereas `let-if` does not. `let-if` is, as its name implies, a composition of `let` and `if` (just as `mapcar` is a composition of `map` and `car`, and `cadr` is a composition, etc.), and it is structured that way. In fact, even if `aif` &co. are individually fine, they add weight (albeit quite light) to the mental burden of the overall language simply by being extra, whereas `let-if`, as a composition of existing constructs, is not extra.
I'm willing to agree to disagree and chalk it up to preference. In fact, I'm starting to feel a little silly elaborating such a small thing :D I'm just hoping I managed to adequately explain my point of view, and that it at least seems like a sensical place to stand, even if you think it's a worse place to be standing.
And thank you for taking the time to elaborate on your thoughts. A friend of mine with whom I periodically debate religious topics is fond of saying "an unexamined faith is not worth having", and I think the same holds true for any belief, opinion, ideology, etc. At the very least, I had a chance to really think about this stuff in more depth :)
I also don't feel that aif has much value, and doesn't resemble the linguistic anaphora that it is inspired by.
That's why I did a "one up" in my ifa design, to outdo the silly aif macro.
In English, I can say, "if x exceeds 3, then print it". When I say that, it's obvious to everyone that by "it" I do not mean the Boolean value of "x exceeds 3". I mean x. So aif is nothing like the anaphoric "it" we encounter in English.
I tried to make ifa at least superficially try to capture some of that: (ifa (> (whatever) 3) (print it)): prints value of (whatever). Also, suppose (whatever) is a place. We want to be able to say "If (whatever) exceeds 3, increment it": (ifa (> (whatever) 3) (inc it)). And we don't want (whatever) evaluated twice.
The expression (aif whatever expr-containing-it) doesn't buy us much over (iflet (var whatever) expr-containing-var). Both communicate an organization of code around something returned from whatever, which can be nil (don't run the code) or some non-nil object (do run the code, refer to the thing as var). To me also, aif does look mostly like keystroke saving.
My iflet/whenlet/whilet/condlet constructs bind multiple variables. The last one is tested: (iflet ((x whatever) (y another thing)) expr-evaled-if-y-true). This makes the macros helpful in more situations. (Next release will make the y optional if expr doesn't need it.)
E.g.:
(whenlet ((x (get-something arg))
(y (get-some-dependent-thing x other-arg (calculation)))
;; code run if y is true, with access to x too
)
Or:
(whilet ((next (get-next-object stream))
(maybe-foo (if next (get-foo-property next))))
;; process only maybe-foo things from stream
)
We should try towrite macros that have some half decent level of impact: bang for the parenthesis and symbol. Especially if they are intended for enduring public use.
My (soon to be published) book on "Computation At Compile-Time" would not be possible with hygienic macros, because I eval during macro expansion, thus treating the compiler as an interpreter during compile-time. https://github.com/billsix/bug
That's possible with hygienic macros. It's not possible with declarative macros, especially those that don't allow explicit violation of hygiene. syntax-rules is declarative hygienic. However, syntax-case and sc-macro-transformer are imperative hygienic. They're capable of what your examples describe.
> Lack of macro hygiene is just peachy keen among Scheme programmers as long as you are explicit about which identifier bindings can leak into, or out of, the macro definition from the macro call site.
Not all Scheme programmers. Mostly Chicken users, in my experience :p
The rest of us demand hygiene.
> Scheme standards before R6RS[0] were a rough consensus on which language features everyone could agree on as useful additions. That's why only syntax-rules made the cut, not because Scheme programmers object to the ability to break hygiene.
The reason only syntax-rules made the cut is precisely because the only thing the Scheme committee could come to a consensus on, as far as macros are concerned, was the high-level, hygienic syntax-rules system. In fact, the very reason Scheme didn't have a standard macro system for many years was specifically because of the problems of traditional procedural macros being unhygienic.
> [0] R6RS, in a break from Scheme tradition, adopted a systemd-esque "come up with a standard for these needed features, any standard, and everybody else needs to fall in line whether or not they like it" approach.
Err, no. R6RS adopted a 60% vote for the ratification of the standard rather than the 100% consensus approach used previously. This change was largely viewed as a welcome one by the community, because R5RS was delayed for nearly a decade because of one person (R. Kent Dybvig, iirc) holding out on some things everyone else wanted (especially records). Furthermore, the change to a vote was enacted by the Scheme community at large, and the editors of the R6RS were all well-known and respected members of the Scheme community before they assumed editorship. There's nothing systemd-esque nor "fall in line whether you like it or not" about it. The entire process was very democratic.
The main reason R6RS hasn't seen wide adoption has more to do with its size than anything else. However, keep in mind that the Scheme Steering Committee had explicitly tasked the R6RS committee with creating a standard that was both backwards-compatible with IEEE Scheme and made it easier for Scheme implementations to be interoperable. At the time the R6RS process started, Scheme's biggest problem was determined to be the lack of portability available to Scheme programs due to the minimalism of R5RS, which made it inadequate for many real-world programs and lead to each implementation having its own incompatible extensions — as such, programs tended to run on one particular implementation and not any others. This made it especially difficult to write useful libraries for Scheme. While R6RS is far from perfect, the R6RS committee really did an admirable job in achieving that goal, all things considered.
>R6RS committee really did an admirable job in achieving that goal, all things considered.
That's true. Although I didn't like the specific decisions they came to myself...
>the problems of traditional procedural macros being unhygienic.
But that wasn't even a problem by the time R5RS rolled around. syntactic closures had been around since 1988, and syntax-case had been around since 1992. R5RS released in 1998. The reason R5RS didn't have hygienic procedural macros was not because such a system hadn't been devised, ir was because the SSC couldn't agree on which one to use.
>Not all Scheme programmers. Mostly Chicken users, in my experience :p
>The rest of us demand hygiene.
That was a regrettable regression from CHICKEN 4, IMHO. I mean, we still have implicit renaming, which is hygienic, but ir is O(n). Frankly, Felix &co should have implemented syntactic closures. Or even syntax-case: as much as I dislike it personally (I agree with Shinn on this one), it's not badly designed, and it would be a heck of a lot better than what we got.
But what can you do? Gambit and Guile got this one right (well, they use syntax-case, but close enough), and debugging (which sucks in CHICKEN, even with Feathers, which is a massive improvement: I can't even evaluate arbitrary expressions on exception. Really?). But despite Andy Wingo's heroic efforts, Guile is still relatively slow, and Gambit has an even worse library selection than Guile's, which isn't that great (although, credit to the Guile devs, the apis on said libraries are really nice).
> That's true. Although I didn't like the specific decisions they came to myself...
Eh, I liked some, didn't like others. If I'd been knowledgeable enough at the time the ratification vote came around, I would've probably voted "no", but I also think I would've had only a few reservations (I was still a newb back then so I didn't vote). The rate of attrition during the drafting process was rather high, which leads me to suspect there were some politics going on internally among the editors, though I don't know for sure, but in retrospect it sure feels like some kind of foreshadowing.
But, it's pretty much the same for R7RS-small: some things I like, others I don't. However, the few quibbles I have with R7RS-small are relatively minor (coincidentally, one of those quibbles has to do with parameters), so I cast a "yes". R7RS-large, though, is shaping up to be something I find really quite distasteful as a Schemer, possibly even moreso than R6RS. I may wind up abstaining regardless, as I've been working on my own Lisp dialect because I've grown kinda sick of the Scheme community in general (no offense — I just feel as though the disagreement and the in-fighting in the community will never end, and I'm tired of it), and after learning CL, EuLisp, ISLISP, and Le-Lisp, my ideas of what it means to "be Lisp" have changed a bit (not that I find Scheme bad in the least, quite the opposite, but I've developed an appreciation for the more traditional dialects, and I've discovered that my ideal is somewhere in the middle).
Hell, if I were around for R5RS, I would've had some qualms even then (for example, I wouldn't have let it ratify until it had records — I would've let Dybvig filibuster until the end of time, and we'd still be using R4RS :p ).
> The reason R5RS didn't have hygienic procedural macros was not because such a system hadn't been devised, ir was because the SSC couldn't agree on which one to use.
Sure, but they did agree on syntax-rules. Furthermore, it wasn't clear at the time what the appropriate system to standardize was — turned out that ER macros can be implemented with synclos (and thus synclos are at least as powerful as ER macros), and synclos can be implemented with syntax-case (and thus syntax-case is at least as powerful as synclos), but nobody could figure out how to go in the reverse direction (and thus it's possible that syntax-case is strictly more powerful than synclos). There was hesitation to standardize a low-level macro system when it was possible that it was the "wrong" one (the point of having the low-level system in the first place was to provide something fundamental, and if synclos turned out to be strictly less powerful than syntax-case, they would not have been fundamental). To my knowledge, it's still an open question whether or not synclos can implement syntax-case.
> we still have implicit renaming, which is hygienic, but ir is O(n).
IR macros are my favorite low-level system by far. Their implementation in chicken (which I once ported to Chibi) is O(n) in the average case, but actually O(n^2) in the general case: since the macro walks the form to perform the renaming (twice!), you can theoretically wind up with lots of nested macro calls, in which case the expansion grows superlinear since it winds up walking both macro forms and their expansions. I spent a lot of time trying to come up with an implementation of IR macros in terms of synclos that didn't suffer from such problems but failed. I still think it's possible, but it might need a small change or addition to the synclo API.
> Or even syntax-case: as much as I dislike it personally (I agree with Shinn on this one)
I know we've had this discussion before, but please remind me why it is you don't like syntax-case. It's not perfect, but then again neither are ER macros or synclos. It essentially behaves like a mix of syntax-rules and IR macros, which is pretty much the ideal behavior in my opinion, but I'll be damned if its API isn't complicated. I think it's the right idea, but the interface is in dire need of simplification. (I do remember one objection was the introduction of new reader syntax, but that's really not essential to the system — we can discuss getting rid of it when we're coming up with that new API ;) ).
> Gambit and Guile got this one right (well, they use syntax-case, but close enough), and debugging (which sucks in CHICKEN, even with Feathers, which is a massive improvement: I can't even evaluate arbitrary expressions on exception. Really?).
Gambit's debugger is a dream. Is Guile's similar? I haven't used Guile much since 2.0, but that's going to change soon.
> But despite Andy Wingo's heroic efforts, Guile is still relatively slow
Yeah, Andy is amazing. He's done (and continues to do) great work on Guile. To be frank, Guile was a piece of shit until Andy took over, a laughable implementation at best. But since 2.0 it's been a serious contender, and 2.2 is shaping up to be a great release.
> Gambit has an even worse library selection than Guile's
Ugh, tell me about it. Marc tried to rectify that by introducing the snow package manager, but it never really took off. Shinn made a v2 of snow and included it with Chibi. I've got my fingers crossed that it gains traction this time.
> I could bear the slowness, but the libraries...
That feeling right there is what prompted R6RS to grow so much ;)
>R7RS-large, though, is shaping up to be something I find really quite distasteful as a Schemer, possibly even moreso than R6RS
Why so? It seems alright to me, but I haven't been a Schemer as long as you have.
>and after learning CL, EuLisp, ISLISP, and Le-Lisp, my ideas of what it means to "be Lisp" have changed a bit (not that I find Scheme bad in the least, quite the opposite, but I've developed an appreciation for the more traditional dialects, and I've discovered that my ideal is somewhere in the middle).
I've been feeling the same. When looking at how dynamic CL is, I start to understand why CL proponents say Scheme is closer to ALGOL than Lisp.
I'll miss the cleanness of Scheme if I go, but I can't escape the feeling that it might be time to move on.
OTOH, there are things I want to do that CL can't do. For example, dynamically bound funtions: the idea you could have a function that shadows another function (maybe even a core function), and, for instance, provides diagnostic information about all calls to said function for the dynamic extent of the binding form. Imagine a macro like:
(report-usage cdr (foo))
Which would show every usage of cdr within foo, and what args it was called with.
AFAIK, you can't do this in CL. You can in PicoLisp, but that's because PicoLisp is essentially Lisp 1.5 reborn, with all the benefits and disadvantages thereof. Like slowness.
>I know we've had this discussion before, but please remind me why it is you don't like syntax-case.
The API is both syntactically and semantically overcomplicated. It violates the standard macro abstraction (forms aren't just lists that you can CDR down, they're now "datums"), for little benefit, IMHO, and in general seems satisfied to heap complexity upon itself.
Because of the semantic complexity, I think any new API that stands a chance at fixing the system would have to radically alter the internals.
>Gambit's debugger is a dream. Is Guile's similar? I haven't used Guile much since 2.0, but that's going to change soon.
Both seem to be semi-inspired by CL's error system. So yes, I think. I haven't had a ton of time with either, though...
>IR macros are my favorite low-level system by far. Their implementation in chicken (which I once ported to Chibi)
That was you? I remember that!
>Shinn made a v2 of snow and included it with Chibi. I've got my fingers crossed that it gains traction this time.
I hope so, too. Chibi is a really nice implementation.
>That feeling right there is what prompted R6RS to grow so much ;)
But R6RS grew in the wrong ways. IMHO, it didn't so much add new libraries to do new things as add new libraries to do old things (although previously unstandardized) in very complicated, by theoretically "better" ways.
> I've been feeling the same. When looking at how dynamic CL is, I start to understand why CL proponents say Scheme is closer to ALGOL than Lisp. // I'll miss the cleanness of Scheme if I go, but I can't escape the feeling that it might be time to move on. // OTOH, there are things I want to do that CL can't do.
I'm in the process of designing a new dialect, sort of a "spiritual successor" to EuLisp (in a technical sense, not a political one). Essentially, I want a dialect that (1) embraces modern PL theory without abandoning tradition wholesale, and (2) incorporates advances made since the finalization of CL without being so minimal as Scheme. I've talked to the editor of the EuLisp (draft) standard (Julian Padget) and bounced some ideas and inquiries off of him. One thing I asked was whether there was anything he wished they would've wound up including in the final draft, and anything he thinks would've been good to have in retrospect; the things he mentioned were all already on my list B) So, I think I'm on the right track in terms of "spiritual succession", though not being from Europe I'd feel like too much of a sham to call what I come up with EuLisp, especially given that it's not EuLisp as it stands today. I've also been in contact with several folks who were involved with Le-Lisp (and later ISLISP) at INRIA (in the course of my questioning, I managed to initiate an open-sourcing of the original Le-Lisp and LLM3 implementations; last I heard, approval from INRIA was acquired and they were in the process of getting it to build on modern systems; I should ping them and see how it's going :) ). Anyway, I'd welcome collaboration if you happen to be interested. I'm drafting a "standard" (Nouvelle Académie Nord-Carolinien de l'Informatique (NANCI) X3J1337 "Programming Language Evelin" (Extensible, Vertible, and Expressive List and [artificial] Intelligence Notation)) and a reference implementation, both of which I intend to post for public review/comment when it's ready (a few months away still, at the very least).
> For example, dynamically bound funtions: the idea you could have a function that shadows another function (maybe even a core function), and, for instance, provides diagnostic information about all calls to said function for the dynamic extent of the binding form.
You can probably cover some of the use-cases with before/after methods, but I don't think they'd cover all of them.
> It violates the standard macro abstraction (forms aren't just lists that you can CDR down, they're now "datums")
Syntactic closures are also a new datatype that don't use the list abstraction. The nice thing about syntax-case is that you can actually write a metamacro that gives you an API akin to the synclo API and hides the dirty dealings with `syntax` values under the covers. It's actually pretty straightforward to implement IR macros on top of syntax-case.
> Chibi is a really nice implementation.
Agreed, and the next release is shaping up to be very nice.
> But R6RS grew in the wrong ways. IMHO, it didn't so much add new libraries to do new things as add new libraries to do old things (although previously unstandardized) in very complicated, by theoretically "better" ways.
I totally agree. Many of the additions abandoned tradition and ignored existing SRFIs. In the very few cases where I felt that breaking compatibility with relevant SRFIs was justified (records, for example), they came up with something worse than the SRFI rather than better. R7RS at least restored tradition and adopted some SRFIs, but I was disappointed that those very few cases where breaking compatibility was justified (records, parameters, etc.) wound up missed opportunities, as the SRFIs were adopted wholesale. Still, the situation is better than it was in R6RS.
By the same token, some things that R6RS introduced that were novel and filled a role not otherwise covered by an SRFI (libraries, for example) were themselves ignored and abandoned by the R7RS committee, for seemingly no reason other than "R6RS bad". I think the R7RS libraries are nicer to use than the R6RS ones anyway, but it did irk some people. R7RS should have been two steps back and two or three steps forward, but it wound up being two steps back and one step forward. I don't hate it; I think it's one of the better standards Scheme has had. Nonetheless, I see quite a few missed opportunities there, which makes me feel a little wistful ;)
> If I were to do this in Scheme, I'd use a "parameter" called 'current-chip' (think of it like a dynamically scoped variable) and write getter/setter procedures that operate on the current chip.
Do you mean to create global functions with those generic, unqualified names? The unhygienic solution at least keeps them nicely inside the lexical scope of the "with" construct. That actually looks more hygienic, for another, more traditional, meaning of hygienic.
I find it confusing that something is magically introducing variable bindings. I see the convenience it adds, for sure, but my personal style would be to opt for a more verbose approach that didn't do this.
You're joking, right? It's not a trivial matter. I appreciate both design methods (I associate myself with plenty of CL programmers), but I can't hide that I think Scheme made the better design decisions.
> You're joking, right? It's not a trivial matter.
It is a matter of context. There is a whole blog post full of hard work and you comment about an unhygienic macro. Author said:
This is nice and readable. Some folks will dislike the fact that it introduces new variable bindings that are “hidden” in the macro definition, but I like the concision you get from it, especially for a small project like this.
This is a small project, the code is deliberately using macros in a way that is regarded as poor style (even in CL), but this is not important here. There is no need to start a flame about single namespaces.
You could also have a symbol-macrolet so that "memory" expands as a call, i.e. "(memory)", but then you would have the same syntax as the above poster wanted to avoid ;-) (except that accesseors already provide setf functions)
HOLY SHIT I SAID I LIKED THE POST. I have talked to the author on IRC. I was just playing off of what he wrote about certain people not liking the with-chip macro. The amount of downvotes I'm getting is ridiculous.
I know both Lisp and Scheme pretty well. Scheme's macros are hygienic, Lisp's macros however are much easier to handle. The hygiene problem can be solved by using (gensym) for local identifiers. Example:
To completely solve the hygiene problem on an academic level, you have to use gensyms for all references generated by the macro, including all function symbols.
Lisp macros don't bother doing this because the Lisp-2 separation provides adequate hygiene. Programs which choose to lexically shadow standard functions like cons or list in the function namespace are considered to be sticking a fork in the proverbial toaster, and so that is not handled.
That would't work as well in Scheme because the mere presence of a variable called list in the user's code would interefere with a (list ...) function call generated by a macro.
Also providing hygiene are packages. If macro-generated code references private symbols in the macro's own package, everything is cool. Packages take care of the problem for nonstandard functions. You just don't use another package's symbols for local functions and variables, and that's that.
How are Lisp's macros easier to handle than syntax-rules? On the one hand, you have a function that receives an s-expression, manually deconstructs and transforms the s-expression while (hopefully) carefully, manually aliasing any symbols to avoid inadvertent capture, and constructs a replacement s-expression. On the other hand, you have a pattern-matching facility that automatically takes care of deconstruction and hygiene for you. Yes, syntax-rules is strictly less powerful than defmacro, but in practice it covers the vast majority of use cases very well.
I'd also like to mention that avoiding variable capture is only one part of macro hygiene. The other part is referential transparency in the face of shadowing. Common Lisp partly provides this because names in the standard package can't be redefined, but it's still wide open when it comes to other names. Consider this (admittedly contrived, for simplicity) example:
(defun square (x)
(* x x))
(defmacro hypotenuse (x y)
`(sqrt (+ (square ,x) (square ,y))))
...
(let* ((square (make-square s))
(diagonal (hypotenuse s s))) ; ERROR! tried to apply
...) ; #'* to a square!
In the expansion of `hypotenuse`, `square` refers to the local variable bound on the line above it, while the macro author clearly intended it to refer to the "global" variable up top. This is a situation that's fairly easy to run into, and horribly confusing when it occurs (especially if the user of the macro didn't write it). Hygiene ensures that the reference to `square` in `hypotenuse` will always refer to the binding of `square` that was in scope at the time the macro was defined, thus avoiding the problem. On its own, `gensym` is necessary but not sufficient for hygiene.
Really, an ERROR? Could you show the actual message that was printed? You can't, because you did not even bother to try.
Here is an actual test:
(defun square (x)
(* x x))
(defmacro hypotenuse (x y)
`(sqrt (+ (square ,x) (square ,y))))
;; whatever
(defun make-square (s) (list :square s))
(defun foo (s)
(let* ((square (make-square s))
(diagonal (hypotenuse s s)))
diagonal))
Now, let's compile (SBCL):
; caught STYLE-WARNING:
; The variable SQUARE is defined but never used.
Not really what you expected, right?
(foo 10)
=> 14.142136
It looks like it works.
You are trying to apply Scheme's semantics to CL. You forgot to take into account CL namespaces, which consist of (1) function/classes/type/variable namespaces and (2) namespaces defined by packages (or custom mappings).
> namespace n. 1. bindings whose denotations are restricted to a particular kind. ``The bindings of names to tags is the tag namespace.'' 2. any mapping whose domain is a set of names. ``A package defines a namespace.''
If you defined a local function named square, that could have been the beginning of a point. However, I don't buy the idea that inside a single package, someone is going to shadow a function that belongs to the same package, in a macro (if this happens, it is more likely to be done on purpose).
As for shadowing function bindings from other packages, this is even more improbable.
Please check your facts and avoid posting made-up error messages that can easily be mistaken for actual ones.
I'm not trying to be critical of CL. I thoroughly enjoy working in CL. I'm just trying to clear up some confusion about what it means for a macro to be hygienic, that's all.
> And how can I believe you? this is just too late.
Huh?
> You were spreading confusion by using a dialect of Lisp known only by you, that looked exactly like CL, without giving proper warnings.
It's a hypothetical example meant to illustrate an aspect of macro hygiene. Again, it's not a criticism of CL. Also, it's not "a dialect of Lisp known only by me". It happens to be EuLisp, which is what I've been working with lately, and I chose it because it was designed as a mix between CL and Scheme (and Le-Lisp, RIP) and I figured that, as such, it would be accessible to both Lispers and Schemers. Apparently, I was wrong.
> Fine, but this would have been the same if you used a nebulous "Scheme" without saying it so.
The specific language was irrelevant to my point. Only the semantics of defmacro and gensym were relevant.
> > And how can I believe you? this is just too late.
> Huh?
See it from my point of view: for all I know, you could as well have made an error, then pretended you were not really writing in CL but a nebulous Lisp; hours laters, after having looked around in the archives, you come back and say "It was actually EuLisp". I am not really saying you actually did this, but I am not ready to believe all you say blindly either. Clarifying things up front would have been way more credible.
> Also, it's not "a dialect of Lisp known only by me". It happens to be EuLisp [...]. I figured that, as such, it would be accessible to both Lispers and Schemers. Apparently, I was wrong.
EuLisp is accessible to both Lispers and Schemers, but you can't blame readers for not guessing what you have in mind when you aren't precise enough in your writings.
> The specific language was irrelevant to my point. Only the semantics of defmacro and gensym were relevant.
Except your example depends on other things than defmacro and gensym, and those things depend on the actual language you use, which makes the language not irrelevant. Simply saying "Take this EuLisp code...", or "assuming a Lisp-1 dialect..." would have prevented the confusion that would necessarily arise. I wonder why you did not just say that.
I apologize for causing so much confusion. My comment wasn't clear at all and I failed to communicate precisely what I intended. Thank you for your critique. I will take it to heart and try to improve my written communication in the future.
On that last point, you give the semantics of defmacro as defining "a function that receives an s-expression, manually deconstructs and transforms the s-expression".
You then exemplify that with (defmacro hypotenuse (x y) ...) which features automatic destructuring that pulls out x and y. The macro needs to do no further manual destructuring on these arguments.
You're right. I was thinking about Scheme when I wrote part of that comment, EuLisp in another part, Common Lisp in another, Le-Lisp in yet another, and my writing wound up being pretty muddled. I apologize for the confusion. I need to work on improving the clarity of my communication. Part of the problem is that I view comments on the Web very casually, a conversation, but that's not really the right way to approach it due to the differences in format. Anyway, thanks.
P.S., I was playing with TXR a bit last week after you mentioned it in another thread on here somewhere. After I got to the web site, I recognized it and recalled playing with it a tiny bit quite some time ago (at least a year ago, possibly much longer). Anyway, it's really neat. If I get some spare time sometime soon, I'm going to try and pick it up a bit more because it seems like the kind of thing that will come in very handy at the command line.
The ideas in TXR seem really novel to me. Did you come up with that form of pattern language, or did you nab it from somewhere else? The closest thing I can think of is SNOBOL, but TXR is much nicer to use than SNOBOL.
The idea for the TXR extraction language came from simply wanting to do the inverse of what "here document" interpolation does: have some text with variables, and have that match a similar text, binding pieces to the variables. I had a passing familiarity with pattern matching (as in logic programming) without much actual experience using or coding that sort of thing. I wasn't aware of how SNOBOL works, and also wasn't aware of PEGs. I just started implmenting and adding ideas as they occurred to me.
The pattern language is basically equivalent to PEGs, which can be seen from the examples of arithmetic expression and JSON parsing.
When I added functions, and hence recursion, I knew that the matching would increase in theoretical power from just regular languages to context-free. But only when I used it to implement some parsing it hit me that the code is basically just grammar rules that execute. After that when I encountered PEGs in some paper, the material was obvious.
Someone told me some years ago that it reminded them of SNOBOL in some ways. That made me curious, so that I researched SNOBOL a bit.
The directives which capture or assert a chracter or line position might have been influenced by SNOBOL. :)
Your example looks like Common Lisp. The square symbol is bound in the variable namespace by let*; the reference in the macro expansion is for a square in the function namespace.
The dual namespace saves most Common Lisp macro-using code from this problem.
In large programs, you'd want to use packages. Those also prevent the problem:
yourpackage:function and mypackage:function are different symbols, so no interference.
Also, ANSI CL makes it undefined behavior to rebind a standard function; i.e. (labels ((list () ...)) ...) can behave arbitrarily, and implementations are permitted to signal an error like "cannot rebind cl:list".
My example is intended to be nebulously "Lisp". I did mean to mention in my comment that Common Lisp's packages help here (I'm usually frustrated in these discussions when people are always like "Lisp2 Lisp2 blah blah" but that's only part of the story, imo packages play a bigger role), so thanks for pointing that out. I did, however, mention that CL doesn't let you rebind standard names, though.
My point wasn't "Common Lisp (or defmacro) bad, Scheme good", or that you can't write hygienic macros in Common Lisp[1], though it appears that I've come off that way. My point is just that `gensym` alone isn't enough for hygiene (in all cases).
[1]: Another thing that doesn't seem to be understood well enough in these discussions is that hygiene is a property of macros, not macro systems.
Formally speaking, CL lets anything happen when you rebind standard function names; that's what undefined behavior means. It's a good idea to diagnose that, and not difficult to do.
> that doesn't seem to be understood well enough in these discussions is that hygiene is a property of macros, not macro systems
Can you cite a comment which doesn't seem to understand this?
And it is wrong; of course macros can be hygienic or not; but macro systems which crank out automatically hygienic macros are also themselves called hygienic. Everyone understands what that means.
(Or else countless of Lipers, including notable experts, have been wrong for years in how they discuss macros and macro systems.)
> Can you cite a comment which doesn't seem to understand this?
I don't have any on hand, but I've encountered this misunderstanding quite a few times over the years in discussions of macro hygiene. To be clear, I didn't mean to imply that either you or GP don't understand this. It's just something I've noticed before and it seemed like a good time to bring it up.
> And it is wrong; of course macros can be hygienic or not; but macro systems which crank out automatically hygienic macros are also themselves called hygienic. Everyone understands what that means.
I'm not saying that macro systems can't be hygienic. I'm saying that hygiene is a property of macros. It's not wrong to say that a system like syntax-rules is hygienic, considering that it only allows for the definition of hygienic macros. IMO, the line gets a little blurrier for systems like syntax-case, synclos, or explicit renaming. They're also commonly called hygienic because they provide special facilities for defining hygienic macros, but synclos and ER macros aren't really that different from defmacro+gensym when it comes down to semantics.
I suspect you're just allowing yourself to be overwhelmed by the ambiguity in "hygienic macro system", which can be parsed as ((hygienic macro) system) or (hygienic (macro system)).
The former parse attributes the macros as being hygienic, and the system as having to do with that kind of macro (an not itself being called hygienic).
Thing is, every macro system is a ((hygienic macro) system), because you can write hygienic macros with any system. The latter parse makes more sense to me, as it relates that it's a system designed to enable hygienic macros beyond what a direct transformer would provide.
In chicken, you have to (r) all your functions (the rename function is passed to the function transformer, but it's called r, or rename, by convention) if you're writing an er macro, which is what you do if you want expansion to be fast.
I don't know why, but it slipped my mind while I was writing the above. I think I was just too focused on my goal and wound up screwing it all up anyway. Oh, well, c'est la vie.
I find parameters an unpleasant hack. I can't understand why RnRS doesn't just add dynamic variables back in (as lispm pointed out to me, they were in R0RS).
What do you gain by having fluid variables rather than parameters? As I understand it, they serve the same purpose, and the difference between them is just an implementation detail.
Firstly, parameters are functions. That means that if the function you're using didn't plan for use with parameters (maybe the code predates R7RS, maybe that value was never supposed to be changed), you're out of luck: have fun with your globals.
Secondly, parameters must be global. You can't, say, have a function that depends on which parameters were bound in a parent function. This is because parameters aren't true dynamic scoping: they're using lexical scoping to emulate dynamic scoping. All well and good, but why not just implement dynamic scoping?
Thirdly, Parameters are fiddly to get right. The path to RnRS parameters is paved with the bones of many a construct (like fluid-let) that tried and failed to safely handle parameters, whether it be because of multithreading, or because of those two famous bugbears of Scheme programming, call/cc and dynamic-wind. The reason for this is that all these constructs, at their heart, are changing the value of a global for the dynamic extent of a function. In scheme, as you no doubt know, dynamic extents aren't guaranteed contiguous.
OTOH, true dynamic variables are a lot simpler to get right in the prescence of continuations: a continuation is a copy of the stack, in low level terms. Dynamic variables are bound to the stack, so dynamic variables and continuations should Just Work. The same is true with threading.
In short, parameters are a construct less flexable than true dynamic variables and a more complicated one to implement, to boot.
I disagree that dynamic variables are more complicated to implement, though. As you mentioned, parameters are fiddly to get right. Dynamic variables, on the other hand, only need to be defined in what I'll call a "fluid environment", which is basically the same as a lexical environment, but it gets passed to `apply` whenever a procedure is applied, the same way the lexical environment gets passed to `eval` whenever it's applied. Under the hood, all the usual operations on environments work without change, because the data structure is the same, though you'll need to expose an API to the extra environment for it to be useful.
Anyway, I don't think the implementation of parameters as procedures (even global ones) necessarily prohibits them from working with call/cc. Consider an implementation like this:
(define (make-parameter value)
(case-lambda
(() value)
((x f)
(let ((old value))
(dynamic-wind
(lambda () (set! value x))
f
(lambda () (set! value old)))))))
(define-syntax parameterize
(syntax-rules ()
((parameterize ((param val)) body ...)
(param val (lambda () body ...)))
((parameterize ((param val)
(params vals)
...)
body ...)
(param val (lambda ()
(parameterize ((params vals)
...)
body ...))))))
Yes, it's still a kludge compared to real fluid variables, but I think it's safe in the face of continuations (I only did minimal testing, so I don't guarantee it). This kind of thing is exactly what dynamic-wind is for :)
>implementation of parameters as procedures (even global ones) necessarily prohibits them from working with call/cc.
It appears I didn't write clearly: the fact that they are procedures doesn't make them not work with call/cc. As I understand it, the procedures mechanism actually makes it easier to make work, which is why Feely wrote the SRFI that way, as opposed to earlier mechanisms like fluid-let, which did use variables.
The procedures are really just a way to control access to the variables. If you use normal global variables, people can treat them like lexical variables which breaks the semantics of dynamic scope. With the procedures, updates are forced to go through the dynamic-wind mechanism and forced to "pop" on exit from the dynamic extent. It's just a way to leverage the call stack to implement the semantics, but I agree it's much nicer (and cleaner) to do it the "real way" handing the fluid environment to `apply`.
I think the use of procedures is fine, but not ideal. At the very least, I think the parameters should be abstracted to hide the fact that they're procedures. This is trivial to do now that there's a real record facility, and the extra abstraction would leave the door open for implementors to use an alternative implementation strategy. A carefully-designed API could even make it so that implementors could use procedures+records or the "real way" with clients being none the wiser.
One thing I do like is that parameters are lexically distinct from regular variables, so it's always obvious whether some particular variable is lexical or fluid.
>This is trivial to do now that there's a real record facility
Last I heard, there's still no way for a function to masquerade as a variable. Has this changed?
>One thing I do like is that parameters are lexically distinct from regular variables, so it's always obvious whether some particular variable is lexical or fluid.
See one of my other replies for a usecase of dynamically bound functions (intercepting calls to a function within a certain dynamic extent) that makes me wish otherwise.
[1] https://groups.yahoo.com/neo/groups/rcacosmac/conversations/...