Hacker News new | past | comments | ask | show | jobs | submit login
A new futex API (lwn.net)
92 points by signa11 on Aug 25, 2023 | hide | past | favorite | 49 comments



I really wish it were reasonable to provide monthly subscriptions that weren't several dollars per month, because I really can't support all the things that I would like to support. LWN is consistently a source of rabbit holes that I can't help but descend into, and I both begrudge them and love them for that, but $9/mo is just... a bit high. It competes with too many "I like it but don't feel a strong need for it in my life" things. But I don't blame them for choosing that level, processing fees are no joke.

This is not intended as a comment on LWN's subscription setup or the article or anything actually relevant. I'm just feeling a bit depressed at the state of funding-of-good-things. It sucks that it's so hard.


They have a $5/month "starving hacker" subscription level. Morally you ought not to use it if you can afford more, but I expect they'd rather you use it than give them no money at.

Personally, I pay for "project leader" level ($16.00/month) so I can give them a bit more... But I'm not up to "maniacal supporer" ($50.00/month)!


"Morally"? According to what moral framework it's wrong to accept the lower price when someone offers you the exact same product at two different prices?


^ This guy gets his groceries from food banks.


Clearly if no such moral framework existed, LWN would not offer both prices.


When they clearly tell you so, and it's a good cause. Why overcomplicate things?


Do you always take what a seller says at face value?

It seems to me that introducing morality into the discussion is overcomplicating it. I don't feel I have any moral obligations with the people I trade with, beyond not deceiving them.


When they tell you "the price is low if A is the case, and high if A is not the case" and you say "I'll take the lower price" and A is not the case then you are deceiving them.


When I said "deceive" I was thinking more along the lines of currency being counterfeit or product not being up to spec. Anything not directly related to the transaction is the private business of each party. I don't ask the people I buy things from to show me their accounting books to see if the price they ask for is fair, so why should I accept any questions regarding how much I'm able to afford?


If a hotel you are planning on staying at for a business trip is offering free rooms for people affected by a natural disaster, do you take the free room?


A hotel has a finite number of rooms. An online publication has no practical limit on the number of subscriptions it can sell, unless it sells them at below cost.


There are also a finite number of people needing to stay there. So imagine you know there are a lot of open rooms, do you still lie and take one?


I assume you're saying that in this hypothetical the hotel has so many rooms that it could host all of those people. In that case, no I probably wouldn't. Not because I think it's immoral, but because I don't know what repercussions I could face if I'm caught in my lie. If I knew for sure that there could be no repercussions then, yes, I probably would lie. I see literally no reason not to do so.


Apples to oranges.


The moral framework of those see the subscription price as a donation for a good cause.


I guess you're right, it's not about morals. It is about common decency (alongside things such as not littering), though.


The no joke is to have a service public on internet: this is several tens of percent on top of the actual service.


Would be nice to be able to centrally manage subscriptions through your bank. Or perhaps better yet, to roll them into taxes, which would avoid some issues going through banks would cause, and offer a nice symmetry to how one-time purchases work.

To me, all those subscriptions have a management overhead that's often greater than their sticker price. Here, $30/month is actually better than $3/month, because there's only so many $30 things that will fit into my "I like it but it's not a strit necessity" monthly budget. But that same budget will fit 10x as many $3/month subscriptions, and the amount is small enough that it's easy to ignore, and it's easy to lose track of all such subscriptions (and then tracking them down and re-evaluating whether you need them feels like more effort per subscription than that $3 or $1 or whatever).

The bigger overhead, however, is relationships. Each subscription is a business relationship, and one I do not want to have. This is where the tax angle, and aforementioned symmetry with one-time purchases, would come into play: for regular one-time payments for goods and services, I do not need to establish ongoing relationships with the providers, because government does it for me. It rolls them all into a neat package by means of generic consumer protection laws, which allow me to not care or remember about any particular vendor if all is right, and if something goes wrong, all I need is to have a standard proof of purchase (receipt or invoice), and a set of standard keywords to put into an e-mail ("warranty", "non-compliant goods", "GDPR", etc.), and things magically right themselves most of the time. Worst case, I need to also remember which form to fill to get courts involved.

That's such a huge win, that most people don't even see it. Whenever you go to buy your daily bread, or get a haircut, you do not need to stick to the same vendor, or sign a contract with them, or otherwise care about ongoing relationship. You can, if you want, but you do not have to, and that's despite the vendor very much wanting you to. The relationship exists, but it's managed for you, centrally, by your local regulator - making purchases O(1) with respect to number of vendors. Whether you shop at one place or a thousand, the overhead is the same.

I would like that to extend to recurring services too. There is absolutely no need for a random SaaS to force me into having a relationship with them. I would prefer to have a standardized process of billing and resolving disputes that lets avoid having to study everyone's individual ToS, or dealing with vendors' dark pattern bullshit. I want to focus on making use of the service, not on who provides it. That's the symmetry rolling subscriptions into tax system would provide.


> Would be nice to be able to centrally manage subscriptions through your bank.

In Australia we’re getting a new system called “PayTo” which will be replacing all current direct debits. One of the advantages of this new system is that you can do precisely this-manage direct debits from within your bank account. Other benefits include making it easier to setup than the current direct-debit-tire-fire.

Obviously doesn’t help in all scenarios, as many subscriptions are just withdrawals, but it’s nice that it’s at least possible.


We have PBS here in Denmark too, which sounds very much like PayTo. However, what you'll quickly find is, unless it's rolled into Stripe etc., that international businesses are not going to pay the premium of using PayTo on a subset of their userbase. Both implementation and billing through such a system is typically higher than continuing to bill debit cards directly.


There was, at one point, a very positive, non-ad related, Google project targeting microtransactions for supporting content on the net without having to create full business relationships etc. - Google Contributor.

Like many google things touching payments, it unfortunately started in such limited area that it couldn't have any future other than cancellation for lack of interest, because most interested people couldn't use it...


> Whenever you go to buy your daily bread, or get a haircut, you do not need to stick to the same vendor, or sign a contract with them, or otherwise care about ongoing relationship. You can, if you want, but you do not have to, and that's despite the vendor very much wanting you to. The relationship exists, but it's managed for you, centrally, by your local regulator - making purchases O(1) with respect to number of vendors. Whether you shop at one place or a thousand, the overhead is the same.

This seems to miss out on the difference between commodity and non-commodity goods and services.

LWN, or the channels I watch on YT, or the people I support on Patreon, are not commodity services. They are not interchangeable in any real sense.


> This seems to miss out on the difference between commodity and non-commodity goods and services.

It doesn't, it applies to non-commodities too. Wanting a particular good or service != wanting an ongoing relationship with a vendor.

Even with YouTube producers, when I like some particular ones and support them on Patreon, that still doesn't mean I necessarily want a generic relationship with them as a vendor/brand - I may be interested in their channel, and everything they do around it, but that doesn't mean I want to hear about their other channel, or know about other unrelated businesses they run.

That doesn't also mean I never want such relationship either. I don't want it by default, and I want to control how far I go, not be tricked or dragged into what I do not want.


The problem is that you're mixing two things:

> Whenever you go to buy your daily bread, or get a haircut, you do not need to stick to the same vendor

This is absolutely not possible with non-commodities. If you want that YT producer's stuff, you have to go to them. You cannot switch to another "vendor".

> or sign a contract with them, or otherwise care about ongoing relationship

Well, that's certainly true, but I'm not sure what it really means given that you cannot switch suppliers.


What you say applies in the limit. A lot of stuff is only sort-of non-commodity, i.e. you could always do without, and eventually substitute by something entirely different.

However, my point is focused more on the relationship aspect. Even with a 100% unique, non-commodity creator, to whom I have to go to get my content fix, there's a difference between two scenarios:

- I come if and when I want, consume content and/or support them in exchange for specific content, or as generic patronage - but otherwise, they do not force themselves into my mental space.

- Like above, except I get spammed by the creator, especially with marketing material outside of what I care about; I have to remember to manage recurring payments 'lest I'll be unknowingly spending money on things I don't want; subjecting myself to the above is a de-facto condition to access the content I want.

The ideal non-commodity recurring relation for me is almost purely transactional - I pay and get what I want in exchange, and nothing more, and otherwise can safely forget the vendor exists.


I joined a number of patreon things and i liked how it just billed me the one thing a month and it was pretty easy to stop it all on the one website, but i recently tried joining a new one and it asked for my payment info again and had a different pay date and i just noped out of it. kinda a pity, i liked the way it used to be.


futex APIs are both prevalent and somewhat unfortunate: Since the task enqueue happens in the kernel, you have no control over how it does it. This makes something like "wait if not equal" or requeue either not portable (few non-linux api's support it) or inefficient to adopt (too many syscalls from intermediate locks or eager wakes when using). It also means custom queueing (i.e. readers in one list, writers in another, same addr key) isn't possible.

NetBSD and Windows seemed to have made a more perf-flexible API here: Each thread has an implicit AutoResetEvent which it can wait on while any other thread can set it. All task queueing can then be done in userspace, in a lock-free fashion too for Windows. This avoids that extra syscall at the end for futex based locks/rwlocks and employs natural/efficient requeue. NetBSD has a slight edge in that you can set the AutoResetEvent of multiple threads using a single syscall.


I’m pleased to see FUTEX2_SIZE_U64, but saddened that it isn’t actually implemented. It has always seemed like a very useful primitive to have.


+99999999, it's seriously annoying to not be able to have a lock "superimposed" on a pointer (as done in lock-free datastructures, and particularly relevant for "intermediate" data structures where some operations can proceed lock-free but some block [e.g. hash table resizing])


> Of course, the job does not end with these patches; there is still a lot of functionality provided by futex(), including priority inheritance, that is not available with the new API.

This is always unclear to me. If there's work being done to streamline the API, why not just provide a single API that always does priority inheritance?


Why has there been so little movement on futex? It's been a decade since FUTEX_SWAP was proposed and we still don't have it.


is there precedence for linux adding new APIs which wrap old APIs, don't implement some features of the old APIs, and add their own new functionality on top?

It seems like a recipe for disaster, but also maybe it's the only realistic way you can go to maintain backwards compatibility.


I'm sure why it's a "recipe for disaster"? The old API is very crufty and now that the problem is stable and well understood, the kernel developers would prefer to start with a cleaner API. Not least because adding (eg) NUMA support to the existing API would be piling hacks on hacks. Take a look at the old API and wonder where you would pass the NUMA struct pointer:

https://man7.org/linux/man-pages/man2/futex.2.html

Another special feature of futexes is that no one ever used them directly (not least because they are subtle to use correctly). Instead a few libraries used them and implemented standard APIs on top (pthread etc). Libraries like glibc and other libcs, and probably a few languages that have an aversion to using glibc. So the specific use cases of the old API are well known and scoped.


> probably a few languages that have an aversion to using glibc.

You don't need an aversion to the whole glibc to consider that "Just eat some archaic data structure" is too expensive for a language that cares about performance.

Rust's Mutex is defined over futex on Linux and some other Unix systems, srw locks on Windows because if you write it in terms of the pthread structure or the equivalent for Windows it's slower and much bigger.


SBCL can use futexes directly.


I seem to recall V4L2 has a similar backstory? https://en.wikipedia.org/wiki/Video4Linux#History


disappointed you can’t provide a onehot integer to do single bit futexes


wasnt futex one of the biggst security holes ever found in modern Linux?


No.




It's a local exploit, which are not uncommon in Linux.


How much of a performance benefit does the average application get from "futex"? Are there some workloads that are more futexable than others?


This question as stated make very little sense as futexes are mostly used to implement lock functionality. As such, they typically have overhead but provide correctness in a multi-threaded environment.


They make locks faster. That’s how a typical developer will interact with them, anyway.


A spinlock can be conditionally faster than a lock with uses futex, but it doesn't scale. Futex is really meant to provide an efficient way to block and unblock threads. One could use FUTEX_LOCK_PI to have it actually implement a lock, but that's a separate argument/edge-case.


This has varied over time depending on kernel implementations, CPU core count, number of threads, etc.

Some relevant reading: https://matklad.github.io/2020/01/04/mutexes-are-faster-than...


Wow, I was genuinely curious about this, but I see curiosity is met with downvotes now.


I don’t think it was particularly fair. The reason is probably that the question was poorly formed: it’s like asking how much of a performance benefit an application gets from "read" (not exactly like that; you could implement locking without futexes, it would just typically be worse, potentially by a lot, and doesn’t really work in as many cases). It also isn’t really the topic which should really be the new api rather than current futexes, though that sort of thinking doesn’t really stop anyone.

A better question to ask might have been for examples of cases improved by the new api. (Or rather by the new features that can be added thanks to the new api). An example improvement is that by having smaller futexes (8-bit instead of 32-bit) you can fit more in a cache line which could matter for some applications, I guess.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: