Ruby is the only language where extensively deep magic feels okay to me, for some reason.
I don't like it in python, I don't like it in Java.
But e.g. in Rails, sure I bump my head sometimes, but overall I like how magical Rails feels because it lets me go so fast.
Maybe it's because Ruby alone is willing to sacrifice so much speed (though not THAT much vs python tbh) and is willing to go all-in on it, that they enable magic to be so deeply magical that it can deliver adequate value to compensate for being more inscrutable?
Whereas other languages' metaprogramming systems keep you a little more leashed.
Ruby's metaprogramming allows you to create really nice, ergonomic abstractions. I can write `has_one :posts` in a User model class in Rails and a ton of useful functionality pops into existence.
On the other hand, that deep magic metaprogramming can be really hard to follow if you need to understand how it works. Tracing back through (or even debugging) a metaprogramming-heavy codebase is a nightmare.
I'd argue that deep magic metaprogramming is great for when you have abstractions you almost never need to dig into. Rails is great because it's relatively rare that I need to go spelunking in the rails codebase itself (and thus understand the deep magic). Instead, I can rely on a huge pile of documentation, stack overflow answers, conference talks, etc to figure out how to use rails' abstractions (like `has_one :posts`) without needing to understand their implementation.
On the other hand, the average production codebase should minimize their use of metaprogramming. When I don't understand how Joe's `Whatsit` interface works, I'm much more likely to need to dig into Joe's code to understand how to use that abstraction. If I have to understand Joe's deep magic every time I do that, it's a net loss.
for debugging, I can't recommend pry (and pry-byebug for older pre-Ruby 3 codebases) heavily enough. being able to do show-source post.author and have it automatically unwind all of the metaprogramming and magic is really helpful
Coming from C/C++, and then C#, F# I am annoyed by the fact that debugging is not a first class experience in these languages.
This lack of easy debugging capabilities has cultivated a whole group of developers whose primary way of debugging is 'let me out in a print statement'.
People do like to blame this on JS, but it's been true as long as I've been writing software. "Don't build on unmaintained libraries" was a lesson I learned hard in Ruby, long before the javascript ecosystem was a thing.
RubyJard looks cool, but the repo is marked as "Archived" and the last commit predates Ruby 3. And, in fact, it looks like there are issues with Ruby 3.2 and RubyJard.
Most software cannot really ever be "done", if only because no software operates in a vacuum. Nearly all software relies on a language, a standard library, an OS, direct dependencies, indirect dependencies, etc. The only way your software can be "done" is if it relies on absolutely nothing, and very little useful software does that.
Damn, you're actually right about the archived stamp. I'm confused because he actually did the pry version bump when I asked for it, and he appears to have rolled it back and yanked the gem build from the server. I'm at a bit of a loss, tbh - this is a real Mandela Effect moment for me.
Well, that sucks. Still, I urge people to clone a copy, update the pry dependency and give it a try because I've never run into any issues and it's at the center of my debug game. I'd be devastated to lose it.
I've worked with Ruby for over fifteen years off and on and the last seven exclusively -- in a large codebase with a fair amount of metaprogramming. We've onboarded engineers, junior and senior, who previously have never used Ruby and it's been interesting to see who does and doesn't have a hard time with it. It doesn't seem to match overall experience and skill level. It's more of a sense that people with patience and a kind of outcome oriented approach (over fixating on why something is done a certain way) will have an easier time unwinding the complexity.
One thing I've noticed, and this is something that seems to bother non-Ruby engineers, is the people that flourish in a complex Ruby application prefer to read code over documentation.
> It's more of a sense that people with patience and a kind of outcome oriented approach (over fixating on why something is done a certain way) will have an easier time unwinding the complexity.
Makes sense. In some ways there's less magic to ruby, because it's not using macros, templates, reflection, generics etc. In other languages you're stepping out of "normal" code into "special" code to make the magic work.
I loved it.. until something broke and searching a method name didn’t work because it was using dynamic method names. A full day of unpacking the Spree Commerce shipping system put me off a bit.
Yeah sure - it absolutely has drawbacks. I too have spent time going "what the heck is this actually doing". But man when you're not debugging one of those wonky things, Rails just feels like programming with a rocket strapped to your chair.
Is it the most maintainable code? No. Is it the easiest to understand? It depends tbh. Rails provides so much out-of-the-box stuff that lots of rails apps end up looking sufficiently similar, especially if they're small-to-medium-sized. But yes you can end up writing some dreadful spaghetti without a bit of discipline, but imo that's true of python and (node) js as well.
But damn is it SO fast to get up and going. And given product-market fit is why most projects and products fail, it's hard to argue against using Rails for me, since if I'm even still here to complain about it later, that's already a win.
I'm not super proficient with Ruby/Rails but we use it for our backend at work, and this reflects a large portion of the time I've spent using it. People who use Rails every day take for granted just how much magic there is and how confusing it can be for someone less experienced with it.
Which is great in theory, until you realize that there are 200 pages of convention, and there is just enough room for interpretation that different companies have slightly different takes on it.
I think here, different people have different preferences.
I prefer convention over configuration because once learned it just works.
For me in Rails when I look at an URL, I kinda know what is the file that handles that (the controller). And that is what I appreciate.
Of course, different needs require different solutions. I also worked with Sinatra which does not have this. But I choose it exactly because it is light and it allows me to group my logic in a different way.
I like metaprogramming for gems, libs, DSLs, etc. stuff that rarely changes, where the maintainers knows where all the bodies are buried. I rarely use it for business logic for the reason above.
That’s what irb is for. But yeah it’s not always obvious how to reproduce a prod issue. Frankly it tends to be a lot of tribal knowledge, so if you’re working at a big Ruby shop don’t be shy about asking the more knowledgeable devs questions.
If you're just using it for quick scripting, which is where startup time mostly matters, and you only use the stdlib and no external gems, you can use --disable-gems and it starts in half the time as python for me.
Well, both are quite slow for what it’s worth, compared to most other programming languages.
Which is still plenty fast for CRUD web applications even at large scales, don’t get me wrong, I’m just pointing out that JS for example often has a 10x multiplier between it and Ruby/Python.
I don't know if thats true. I've been working on Rails apps for over 10 years (some poorly written.) I worked on one particular Rails app for 7 years.
I've also worked on Java, Go, Elixir, and Haskell codebases.
I find that anytime we went "Slow" was because of decisions that were made around the data model, which can happen in any framework or language.
The thing that I love about working in Ruby is Rubyists tend to write fantastic tests (which double as documentation of what ever business logic), and the culture around "convention over configuration."
So even when we decided to port one of those Rails apps to Elixir/Phoenix, it wasn't an absolutely awful rewrite.
That isn’t my experience. Using a good statically typed language means that refactors will be fast. Using Ruby/Rails or similar means that you may not be confident after you’ve introduced important changes in your code base. So you might be more conservative? Take more time, do more testing (than was necessary if you used less meta programming and a more statically typed language)
The deeper you go in ruby, the more you regret metaprogramming.
It's fine for those 2-3 cases, and should be banned everywhere else.
In rails is highly abused. What rails achieved and it's good at can be achieved with less metaprogramming, but of course nobody is going to build another rails, since rails is already there.
Rust’s macro system is the first I’ve seen with the same potential. But of course, an equal to Rails has yet to surface; and even if one does, being a compiled language still counteracts a lot of its ergonomics.
I'm sure there are rare cases where these techniques are useful. Like creating developer tools or making your own object persistence layer or winning a code golf contest. But if you're an app developer for goodness sake just write a few extra lines of code. Do whatever it is you're doing the verbose and clear way, not the slightly shorter and super obtuse way.
Stuffing method definitions into classes at runtime, monkey patching, dynamically generating method calls with .send. These will all be very puzzling for any future developer that works on your code, senior or junior. And come with bunches of technical pitfalls. Writing clear and maintainable code is a higher calling for us than reducing LOC and showcasing neat tricks. Even if you call yourself a Rubyist. Speaking from experience.
Arguably reducing LOC helps maintainability, as long as it's done in a clear manner. That's one of the main points of abstraction, and all programming languages offer various abstraction facilities, unless it's a language like brainfuck.
Depends how it’s done. And you can overabstract or do the wrong abstractions thinking you gained something from your cleverness but quite often all you’ve done is you obfuscated, reduced the maintainability for others and you got stuck with a steaming pile. I’ve seen quite a few fellows do this deliberatley in order to become harder to replace and it works somewhat. But said codebases don’t age too well
You don't need to be a python zealot to use the term, and abhor the practice of, monkey patching. In fact rubyists should hate it even more. There's a special, nearly unattainable state of pure ethereal rage one can only reach after 6 hours debugging some incomprehensible failure only to finally realise it's due to someone who has suddenly become your mortal enemy monkey patched ".to_json" a week ago.
"Your honey, I admit to killing him, and I'm sorry, but in my defense, he monkey patched a core method"
Ruby meta programming is awesomely powerful, but also one of the main reasons I never want to work in another’s Ruby codebase again. There is always some developer who read some article like this and invents their own DSL to solve a problem that didn’t need one. It’s pure pain to debug it.
Exhibit #1: rake's DSL syntax. It allows "neat" syntax abominations like
rule :name, [:param] => [:dep1, 'dep2'] do |t|
where every argument except the name can either be missing, single (value) or multiple (array). Sure, it has the "advantage" that it's syntactically valid Ruby code, but it then requires some 70 lines of awful code to actually parse that data into a usable construct: https://github.com/ruby/rake/blob/7b50e9dc37abc57fd365c16cb1...
I always have to look up the variations of that parameter passing when writing rake tasks, not sure that is such a good sign that it's a great design in the first place.
If you have some group that all understands and checks each other from the beginning, Ruby is probably great.
However it was every startups default choice for a while, and those code bases get wrecked and have no oversight until way later. A lot of bad ideas becomes foundational. This scenario is far more common than good Ruby code bases. It’s just easier to rule out Ruby when looking for new jobs.
I agree, but I wonder how people feel about Racket in this context, where the "language oriented programming" approach is common and the idea that you solve problems by creating a DSL is the norm.
I feel the same. For my own projects where it's only me working on the code it can be really nice, powerful and big productivity boost. But if you start having multiple people on the codebase it's not going to be fun to debug.
As most other commenters have similarly said, this blog post should be a single word: "Don't".
The book "Eloquent Ruby" has several chapters on how to use "method_missing". None of those chapters say the only valid use of method_missing: "Never use it."
Lots of languages have deep flaws. Javascript, Python, Java, Perl, C obviously, all let you do some pretty horrific stuff if you really want to. Ruby is the only language where the horror is embraced by the ecosystem and its users.
There's a reason most engineers are adamant you shouldn't use this. It's why learning Ruby as your first language is probably a bad idea, since it normalizes what every other language has agreed is bad practice.
When i was around 20 i learned Ruby as basically my first language and it has been a joyride ever since. I had a chance to develop and co-develop things in other languages but for me there is none matching speed, joy and effectivity of Ruby.
Everyone's mileage may vary but i definitelly do recommend learnign Ruby as a first leanguage. What i also recommend starting developers is to not follow dogmatic advices blindly.
I don’t know Ruby but I’ve been a huge fan of Lisp macros (Clojure).
As with all abstraction mechanisms, you avoid them until you need them. When you do need them, they become a force multiplier.
A lot of macros can be avoided with data driven programming, which is likely one of the strongest techniques in terms of cost/benefit.
A light form of meta programming is source code generation (often used in tandem with data driven). It lets you have macro-like power, but the guts are spilled out for you to modify further or reason about easily. In some languages it’s the only thing you can do at that abstraction level.
In any case, meta programming is very powerful. But its quality and utility hinges on you to avoid it until you exhaust all of the lower level techniques. Else you end up with the worst kind of wrong abstraction.
Do! When the need arises. Language designers wouldn't include such facilities if there was never a need. Ruby is hardly alone in this. I can't imagine a Lisper ever saying never use macros. C++ has templates, Rust has macros, Python has magic methods, metaclasses and decorators. Javascript, Java, Perl, C let you do some pretty horrific stuff because sometimes you need the the flexibility. If you look at enough library and framework code, you will see those things in use.
Don't! Be dogmatic about programming. There are always use cases.
Yeah no. There are times when it's useful. I think most of the times you don't need it but you coming in and saying don't use it sounds a bit pretentious.
Ruby does not have to do what other languages are doing. There is no good or bad.
Do you have any stories? Tried using it and it backfired? Were bitten by a nasty bug in a gem?
We have hundreds of thousands of lines of ruby code spanning many services / monoliths. Even now I find it somewhat annoying to open a controller / component that is basically an empty class def but somehow executes a bunch of complex stuff via mixins, monkey patches etc, and you have to figure out how.
We are turning to https://sorbet.org/ to reign in the madness. I'm keen to know if others are doing the same, and how they are finding it (pros and cons)
I have used Sorbet in my web framework. The method signatures can be tedious to type out sometimes, but it helps a lot when refactoring code. I feel more confident that my changes will work. It's not as flexible as TypeScript, but it's pretty good and has saved me from mistakes many times. The language server helps a lot with autocompletion so I don't have to keep every little detail in my head all the time. Sometimes I have to structure the code a little bit different just to make Sorbet happy which is annoying, but I've also been able to replace huge parts of the codebase quite easily.
Do you also not allow loops, because they are the devil’s lettuce?
Metaprogramming is just a higher level of abstraction that lets you express things that would otherwise be tedious or repetitive. It’s not that different than using a loop to avoid having to write the same thing over and over again. Avoiding it at all costs is dogmatic and kind of silly.
The single most difficult thing in software development is the fact that the system doesn't exist, it is defined by mystical incantations, and we work on that invisible system by changing the instructions to build it.
Anything that makes the understanding or "building" that system mentally is no-go.
Flashback to one of my favorite debugging stories where an entire java code-base was meta-programmed into existance at runtime and would constantly throw errors to lines of files that didn't exist.
Broad proscriptions such as this almost always set my teeth in edge. Sure, meta programming can be a foot gun. But it can also be a way to cleanly achieve specific goals. Metaprogramming should by no means be the first tool reached for, but throwing out the baby with the bath water is an overcorrection.
It also means you are losing out on a significant portion of what makes Ruby Ruby.
If you're going to give rails the credit for the successes, you should probably give it the blame for the failures, too.
I've worked for large companies and startups, and seen rails several times, along with .net, java, scala, php, node and other backbends.
The only thing that mattered in any of their technical successes was the quality of the engineers working on them.
There were certainly minor differences- irritations like compile times or dynamic typing- but the trade offs were 90% of the time personal preference, when compared to the difference made by the strengths of the actual people writing code.
That >75% value created by companies which used Ruby did not create an equivalent debt in another column.
If Ruby-first companies that aren't AirBnb (for example) fail, it's not because AirBnb succeeded. (Perhaps unless your company was called Couch Surfing.)
To protest on the grounds that for some teams, Ruby might be the wrong choice is to willfully ignore the key detail about the whole outlier-degree success thing.
> Flashback to one of my favorite debugging stories where an entire java code-base was meta-programmed into existance at runtime and would constantly throw errors to lines of files that didn't exist.
If an error is thrown from Lombok code, you’re misusing Lombok. It gets way more hate than it actually deserves. Spring does 100000000x more in terms of making code based confusing to navigate and confusing errors yet receives very little criticism from the Java community. If we want to talk about misuing metaprogramming and code generation, Spring is the biggest offender IMO.
> If an error is thrown from Lombok code, you’re misusing Lombok.
I mean, that's fair, but if you're using it in a team based environment, eventually someone's going to misuse it and you'll have to figure out what happened. I think how bad things go when you misuse it is a valid consideration, even if it shouldn't dominate everything else.
> It gets way more hate than it actually deserves. Spring does 100000000x more in terms of making code based confusing to navigate and confusing errors yet receives very little criticism from the Java community.
I mean, I don't think you're wrong, but I'm also not in the java community. The few times I've forayed into there, it seemed more like a dynamically-typed (or more accurately, stringly-typed using only the names of config) annotation language rather than java. And if I wanted string-based typing and confusing semantics, I'd just write my whole project in bash.
Lombok does kind of hacky things with Java byte code and classloaders. Definition of "undefined behavior" of JDK.
The ideas, however, are solid. They are just implemented better in other frameworks such as https://immutables.github.io/, using officially supposed JVM tools like annotation processors.
I learned how fun metaprogramming is but also realized that it is a double edged sword, so if I want the codebase to be maintainable then I'd stay away from metaprogramming as much as possible, it's a hell to debug, but also very powerful when you want to construct your own DSL etc.
Is this what passes for an article about metaprogramming in ruby?
Using send is extremely common especially when mocking private methods in rspec. I guess I am speaking from a rails lense, but what other lense is there for ruby development?
I am forever stuck in a world where "articles" aren't something you read in less than a minute.
Metaprogramming Ruby by Paolo Perrotta is an awesome in-depth resource in comparison. It’s a bit outdated but the base of metaprogramming magic in Ruby hasn’t changed much in 3.0+
It occurs to me that metaprogramming looks awfully familiar in JavaScript.
function process_item(item, action)
{
item[action]();
}
(Also similarly forgetting the guard statement).
Probably not great code, but still.
Since there are no classes in JS and everything is an object, you technically have complete control over such 'classes' and 'methods'. This is probably why people keep trying on turning JS into a different and saner language, or better yet, are using another language entirely like TS. Which is a shame, because it's kind of fun to dynamically generate new functions on the fly and having complete freedom to do anything you want.
I don't like it in python, I don't like it in Java.
But e.g. in Rails, sure I bump my head sometimes, but overall I like how magical Rails feels because it lets me go so fast.
Maybe it's because Ruby alone is willing to sacrifice so much speed (though not THAT much vs python tbh) and is willing to go all-in on it, that they enable magic to be so deeply magical that it can deliver adequate value to compensate for being more inscrutable?
Whereas other languages' metaprogramming systems keep you a little more leashed.