I've started to use Ruby for mathematical and scientific computing using IRuby. I am not in need for high performance computing yet so Ruby is still sufficient. Oh boy what a joy it is to use.
Totally valid point, but I do think magic is more a property of the clever (usually reflection-based magic) things people make with Ruby, less so the language. The draw towards magic is actually because of the logically-consistent object model Ruby is built around that I find to be less magical and edge-case-filled than other script-y languages.
If you just use it as a glue language to call out to other things, prepare data, and iterate over results: it's pretty clear and concise.
(I do personally like the reflection-magic in a lot of Ruby apps, so I might be overlooking something that feels normal to me, but is some very weird behaviour to anyone else!)
How is pythons object model less "logically consistent"? Multiple inheritance is dangerous but I'm not sure if its logically inconsistent. And it doesnt have the issue that ruby has of procs not being objects
Ruby borrows a lot from Smalltalk. Everything is an object, including things that are normally keywords elsewhere like Class or Module. There are no public properties on objects, only methods, which is technically only « messages » and object can receive.
Ruby does well with metaprogramming magic because most of the languages own keywords and concepts can be rewritten at runtime. They’re implemented AS Ruby code.
I would say it’s logically consistent because there isn’t as many « special » things. Python’s def keyword is special. You can’t make your own, nor can you hook into it. In Ruby, that’s just a message being passed to the current scope’s object. You can do anything you want to it. Ruby plays by a pretty consistent rule book.
I'm sure there are minor exceptions or arguable cases but that's also the case with Ruby. Most infamously with Proc.
Classes are objects in python, usually instances of class called type or a subclass of type(usefulfor controlling class creation), this is called the metaclass.
A constructor call MyObject() is basically just the __call__ method of the metaclass.
Modules are also objects usually of type types.ModuleClass or a subclass although really you can stick any object in sys.modules dictionary and have it act like a module and import it.
Sure it doesn't use messages instead attributes whether field or methods both go through the same lookup method __getattribute__ which goes up the class inheritance chain in an OO manner. There's also __getattr__ for more method_missing like access instead of overriding every object attribute access.
A function or lambda is an instance of function class although I don't think you can't modify or override the function class I think you can hook it to log it.
You can also do all sorts of stuff by hooking module import and rewriting the ast. Or more easily you can decorate/wrap the functions in another function object.
It's true that you can't modify c based classes but you can do a lot of magic in python, people usually just don't because unnecessary magic is frowned on. But sometimes it's useful like the way pytest rewrites
assert a==[1,2,3]
to print a nice error message with both variables
https://github.com/pytest-dev/pytest/blob/main/src/_pytest/a...
Basically the python function is created as a regular python function but then passed to a jit decorator
@something.jit
def some_func():
....
Which takes the source code and uses it as a dsl for something else compiles it say something to run in the GPU and assigns it the name of the original function. Technically you haven't hooked creation of of the func just replaced it with a wrapper or in this case a related function that doesn't call the original but it does the job.
Same. I've been using Ruby+Rails for the past year as my company's entire back-end is Ruby. I found it to be ugly and almost intentionally designed to be difficult to read. The tools for Ruby (RubyMine, Ruby LSP for VSCode) are slow and unreliable. It also seems that debugging Ruby works sometimes, but not always, which is ridiculous. RubyMine has its own wacky debugger support, which breaks whenever there's a new Ruby release. VSCode tries to use rdbg, but frequently stops working.
At least for me - Rails solidified my distaste, but Ruby gets tarnished by it.
It's like seeing a c++ codebase riddled with macros. Is c++ directly responsible for the madness? Probably not, but the tooling allowed it.
And Ruby takes a double whammy on this front because Rails was really what drove the popularity. Hard to describe the frustration of hitting a very large Rails codebase and literally not even being able to find the definition for a class because it's got a fairly generic name and is getting auto-loaded in the bowels of the codebase by someone who thought they were clever like 7 years ago.
It's like a special version of DLL hell. Or all the pains of global window/self vars coming from script tags in JS, but at least that gave some breadcrumbs, and the ecosystem is generally moving away from it even if ESM is still painful in a lot of ways.
Either way - I don't like hidden things. I'd much rather see the sausage get made then try to save a couple lines of typing, and the older I get, the more I judge languages on the simple metric of "How well does text-based search work for finding relevant code?". Rails performs really badly on that metric.
Ruby except with Python style imports where you had better god damn explain in explicit detail where every symbol in the file came from would pretty much be my ideal. Python from blah import * is banned in pretty much every codebase but in Ruby that, or worse autoloading, is all you have.
In Ruby, you always use the global name (with caps) that normally matches the library with possibly some nesting. (exceptions exists, but its also possible to add globals in Python)
Unless you are talking about include, but thats for mixins, which are snippets of reusable coee you can add to you class.
It doesn't feel at all as dirty as `from blah import *`.
My problem also. Loved Ruby's object model, but it seems the metaprogramming gives too much freedom to run free and wild. I had a hard time understanding understanding how some gems even achieve, as they crisscross across files and other gems.
I agree. And it's impossible to follow. No type hints, missing syntax, generated identifiers all over the place, etc. Awful awful language. I'd rather write Perl.
1. It leaves out parentheses in places where they really should be there for clarity. Ok it's not nearly as bad as e.g. OCaml, but it's still a downside.
2. Basically Ruby code seems to favour magically generating methods and variables based on other strings, which makes them impossible to search for.
Other languages sometimes do that too, e.g. ill-advised __dict__ Python tricks, or with macros in C++ or Rust. But it's definitely worse in Ruby, and it's a common complaint (search for "magic" in these comments).
I literally have given up following Gitlab's code before. That never happens in better languages, e.g. I can easily follow VSCode's codebase (Typescript) or gitlab-runner (Go).
Really? That's one of the things I like about Python: how its for loops consistently consume an iterator, and that's it. Anything that looks like an iterator can be looped across or used in a comprehension without special casing.
It's so great for writing DSLs. This magic might turn some people off but the result of this meta-programming makes for some very expressive and concise code.
> It's so great for writing DSLs. This magic might turn some people off but the result of this meta-programming makes for some very expressive and concise code.
The problem with DSLs is that you now have a different language for each project you have to maintain.
Each time you come across some little DSL that shortens the original writers work by a few lines (at most), you have to read it carefully, and in depth, to understand what was intended.
I'm turned off by Ruby for this reason alone. If I want the ability to throw little DSLs all over my code, Lisp does it better and more readably.
I keep hearing about the dev satisfaction with ruby, but I'm starting to work with it and I haven't yet felt it at all - I see many footgun design decisions like clashing classes not being an error, and blocks feel like a more cumbersome/limited version of lambdas/first class functions.
I'm not saying this to attack the language, just pointing my current mental picture to ask: what's the magic everybody loves? I know there is something there because it's an almost unanimous view among ruby devs, so I want to get it.
You’re probably talking about the ability to reopen (or “monkey patch”) classes (which is why defining the same class again can’t be an error). That’s a powerful tool if you use it carefully, especially for debugging or hacking on someone else’s code.
> blocks feel like a more cumbersome/limited version of lambdas
Yes, that’s basically what they are. You could totally write JS-like code and pass even multiple lambdas to methods. However, the most common use case — passing a single important lambda (e.g. to a “map” or “each” method) — gets special treatment and its own language feature.
> what's the magic everybody loves
For me it’s the “batteries included” factor. Ruby can do a lot out of the box and most of the interfaces are well designed with tons of useful methods with descriptive names. It feels like someone cares about my experience and is trying to anticipate what I might actually need.
Also the object oriented mechanisms are easy to understand and don’t feel weirdly bolted on like in Python.
Reopening classes is a powerful tool, but it would be a lot safer if the syntax was limited to class_eval and didn't let you define the same class twice.
As someone who's worked with Ruby for 12 years here are some insights:
Ruby makes message passing first class. That just changes how you think about programs. In exchange you give up passing functions so our anonymous functions are our blocks (actually just another object that can receive messages). So you don't `list(something)` you `something.list` and that lets you change what `.list` does depending on who `something` is very easily.
Ruby's defining feature is that the line between language author and program author is razer thin. You can completely change the language yourself by extending core classes. This is why we have `1.day` in Rails even though its not supported in the main language. DHH (author of Rails) could add that without consulting Matz (author of Ruby). So lots of stuff gets prioritized to make developers happy because its easy to add.
In Ruby the messages you receive are passed up the chain of ancestors. Your ancestors are a linked list of classes going back to a root class like `Object`. You can modify this linked list at will with mixins and inheritance with complete control (should I go before or after this other receiver).
Ruby's REPL and debugging experience is amazing. I often keep one tab with an `irb` or `rails console` open to sketch things while developing the code for it elsewhere. I'm also always inside a debugger to figure things out. When I'm in Rust or Python I'm met with a very different environment.
I get the debugging experience, as I've already found it pleasant and painless compared to the difficulty of setting debugging environments in other languages. But for the messages first approach and thinness, what's the real world benefit in practice?
You can intercept and modify messages on the fly. This helps a lot if you want to update legacy code, but don't want to break other classes that use the same call. It also allows for modularization more easily. I have a project that receives data from a bunch of sources, so I have a class for each source (this is simplified). To make it much easier to add a new one, which happens pretty regularly, the code that calls the processor checks all the classes in a specific folder for a certain function (`.respond_to?`) and then gets the right one for the data type. This means that to add a new source I have to change 0 lines of legacy code, just drop the new class file in the right place and it works.
Yeah like many things Python's overnight success was the result of a decade of work. Python started involving scientists (and creating science-oriented SIGs) in the mid 90s. Numpy's original ancestor was Numeric, first released in 1995.
If i'm reading this correctly for web applications setting RUBY_THREAD_DEFAULT_QUANTUM_MS=10 or maybe even RUBY_THREAD_DEFAULT_QUANTUM_MS=1 could be more optimal then the current default of 100, allowing for more throughput at the cost of potentially slower single shot response times?
I think you have that backwards; it should decrease latency and decrease throughput.
I don't imagine the overhead will be too bad though, so it may decrease latency while keeping throughput essentially the same.
Of course you'll want to benchmark on your specific load. For example, at my day job it'd make no difference because we don't use threads, each Passenger worker has just one thread and handles one request at a time.
I actually had a less than pleasant surprise when debugging sidekiq process with a higher than usual threads count, and noticing 100ms-multiples in my traces. 100ms before switch indeed seems to be too high for an app with a potential to misbehave.
IIRC it’s complicated because it can depend on the relative run times, and how fast the IO OPs are. By lowering the quantum you’re increasing the switching overhead (because there’s more handoffs), but it might allow the io thread to complete much sooner and stop contending with the cpu thread.
Python’s “new gil” (it’s some 15 years old...) uses a similar scheme, with a much lower timeout (5ms? 10?), but it still suffers from this sort of contentions, and things get worse as the number of CPU threads increase because they can hand the gil off to one another bypassing the io thread.
Ben Sheldon, I really appreciate how you wrote this article. It feels like it was written in a very direct way that is more easily consumable for people who's attention tends to drift on long-form articles. I feel like you covered a complex topic succinctly, and I appreciate that. Helped me get a better understanding of thread contention. Thanks.
Intresting that this problem (IO-bound threads should have priority over CPU-bound threads) already solved at OS level (most OSes will give priority boost to thread that was unblocked because of end of IO operation in hope that thread will soon block with another operation).
Loved this writeup, because queues are the most important and general concept in reasoning about performance. When you realize this, you start seeing them everywhere. Locks, async IO... everything is just interacting queues.
Yes, but I think it's a revelation to many people that things like this map to queueing. Literally _everything_ related to performance is queueing, but we use different words for different scenarios, like "contention."
> Literally _everything_ related to performance is queueing
Well, _scaling_ is queuing. But other things: cache hit rates, data packing efficiency, efficient use of hardware intrinsics, etc, don't relate to queuing.
* An arrival rate of requests
* A processing time for each request
* A corresponding queue length when a new request comes in before the current one is done being processed.
So anything that affects processing time affects queueing. Cache hit rates, data packing, etc., all affect processing time, but processing time is in the context of a queue.
Your whole system is a queue. Requests come in, are processed, and some output is given.
Algorithmic complexity affects the processing time of each request / task. Knowing the latency distribution is crucial in understanding the queueing characteristics of the system, and the algorithmic complexity will greatly affect that distribution.
Most mutexes aren't fair; it's not strictly equivalent to queueing. That said, I agree with your larger point that I don't understand what the author finds revelatory about this.
Maybe not but even spinlocks logically contend this way. For example the Go and Abseil mutex libraries, which are similar, account cycles spun toward contention statistics.
reply