Hydra not populating with cross compile builds is the bane of my existence.
I'm using `clang` from `pkgs.pkgsCross.musl64.llvmPackages_latest.stdenv` to cross-compile Rust binaries from ARM macos to `x86_64-unknown-linux-musl`. It _works_, but every time I update my `flake.nix` it rebuilds *the entire LLVM toolchain*. On an M2 air, that takes something like 4 hours. It's incredibly frustrating and makes me wary of updating my dependencies or my flake file.
The alternative is to switch to dockerized builds but:
1) That adds a fairly heavyweight requirement to the build process
2) All the headache of writing dockerfiles with careful cache layering
Not sure if this applies to your situation, but I believe you can avoid a full rebuild by modularizing the flake.nix derivations into stages (calling a separate *.nix for each stage in my case). That is how it appears to be working for me on a project (I am building a cc toolchain without pkgscross).
I pass the output of each stage of a toolchain as a dependency to the next stage. By chaining the stages, changes made to a single stage only require a rebuild of each succeeding stage. The final stage is the default of the flake, so you can easy get the complete package.
In addition, I can debug along the toolchain by entering a single stage env with nix develop <stage>
Not sure if this is the most optimal way, but it appears to work in modularizing the rebuild.(using 23.11)
I've been super interested in this field since finding out about it from the `sled` simulation guide [0] (which outlines how FoundationDB does what they do).
Currently bringing a similar kind of testing in to our workplace by writing our services to run on top of `madsim` [1]. This lets us continue writing async/await-style services in tokio but then (in tests) replace them with a deterministic executor that patches all sources of non-determinism (including dependencies that call out to the OS). It's pretty seamless.
The author of this article isn't joking when they say that the startup cost of this effort is monumental. Dealing with every possible source of non-determinism, re-writing services to be testable/sans-IO [2], etc. takes a lot of engineering effort.
Once the system is in place though, it's hard to describe just how confident you feel in your code. Combined with tools like quickcheck [3], you can test hundreds of thousands of subtle failure cases in I/O, event ordering, timeouts, dropped packets, filesystem failures, etc.
This kind of testing is an incredibly powerful tool to have in your toolbelt, if you have the patience and fortitude to invest in it.
As for Antithesis itself, it looks very very cool. Bringing the deterministic testing down the stack to below the OS is awesome. Should make it possible to test entire systems without wiring up a harness manually every time. Can’t wait to try it out!
> you can test hundreds of thousands of subtle failure cases in I/O, event ordering, timeouts, dropped packets, filesystem failures, etc.
As cool as all this is, I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised. So much complexity I've seen in such systems boils down to calling a "function" is: async, depends on the OS, is executed at some point or never, always returns a bunch of strings that need to be parsed to re-enter the static type system, which comes with its own set of failure modes. This makes the seemingly simple task of abstracting logic into a named component, aka a function, extremely complex. You don't need to test for any of the subtle failures you mentioned if you leave the logic inside the same process and just call a function. I know monoliths aren't always a good idea or fit, at the same time I'm highly septical whether the current prevalence of service based software architectures is justified and pays off.
> I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised.
You can't get away from distributed computing, unless you get away from computing. A modern computer isn't a single unit, it's a system of computers talking to each other. Even if you go back a long time, you'll find many computers or proto-computers talking to each other, but with a lot stricter timings, as the computers are less flexible.
If you save a file to a disk, you're really asking the OS (somehow) to send a message to the computer on the storage device, asking it to store your data, and it will respond with success or failure and it might also write the data. (sometimes it will tell your os success and then proceed to throw the data away, which is always fun)
That said, keeping things together where it makes sense, is definitely a good thing.
I see your point. Even multithreading can be seen as a form of distributed programming. At the same time, in my experience these parts can often be isolated. You trust your DB to handle such issues, and I'm very happy we are getting a new era of DBs like Tigerbetle, FoundationDB and sled that are designed to survive Jepsen. But how many teams are building DBs? That point is a bit ironic, given I'm currently building an in-memory DB at work. But it's a completely different level of complexity. And your example with writing a file, that too is a somewhat solved problem, use ZFS. I'd argue there are many situations where the fault tolerant distributed requirements can be served by existing abstractions.
TigerBeetle is actually another customer of ours. You might ask why, given that they have their own, very sophisticated simulation testing. The answer is that they're so fanatical about correctness, they wanted a "red team" for their own fault simulator, in case a bug in their tests might hide a bug in their database!
I gotta say, that is some next-level commitment to writing a good database.
Sure! I mentioned a few orthogonal concepts that go well together, and each of the following examples has a different combination that they employ:
- the company that developed Madsim (RisingWave) [0] [1] is tries hardest to eliminate non-determinism with the broadest scope (stubbing out syscalls, etc.)
- sled [2] itself has an interesting combo of deterministic tests combined with quickcheck+failpoints test case auto-discovery
- Dropbox [3] uses a similar approach but they talk about it a bit more abstractly.
Sans-IO is more documented in Python [4], but str0m [5] and quinn-proto [6] are the best examples in Rust I’m aware of. Note that sans-IO is orthogonal to deterministic test frameworks, but it composes well with them.
With the disclaimer that anything I comment on this site is my opinion alone, and does not reflect the company I work at —— I do work at a rust shop that has utilized these techniques on some projects.
TigerBeetle is an amazing example and I’ve looked at it before! They are really the best example of this approach outside of FoundationDB I think.
That's the nicest counter-example to my example, thanks for that! I wasn't familiar with take_while() (looks like that's Rust, and looks like Python has a similar itertools.takewhile() ), TIL a neat FP version of break. My example was quite trivial, it's not always so obvious how to break up a convoluted loop, but it should always be possible.
A buddy of mine got hit by one of these recently. He got a message from a friend asking him to try a demo of new video game. His friend is a video game developer so this didn't seem suspicious. The "video game" had a landing page and everything. Turns out that his friend's account had already been hacked and the "video game" was a stealer like this. TL;DR: he lost his account and that same hacker tried to get me with the same scheme.
Discord support has been completely unhelpful, because he didn't have 2FA enabled before and the hacker added it.
"oops I got hacked when I hacked and/or tried to hack you" is a pretty old trick... I would be cautious about this friend.
At least it was just discord. I'd treat this as a valuable lesson on the virtue of 2FA especially if they have a habit of running untrusted executables (especially with admin permissions...)
I don't think you should feel safe just because you have 2FA enabled. Local malware can wait until the next time you have to provide your second factor, and then use it to disable 2FA, etc.
My main takeaway from looking at some of the repositories is that they are deathly afraid of being run in a VM, because they think that means someone is trying to reverse engineer them. (Which I suppose makes sense; test untrusted software in a VM, if it doesn't do anything evil, then run it outside of the VM.)
At the end of the day, running local exes is about trust. Having 2FA enabled reduces the attack surface you have exposed, even if it doesn't eliminate it as you point out.
I think they just mean that recovering the account post-compromise was not possible because they didn't have 2fa already setup to authorize them as the true owner of the account, not that 2fa would've prevented the issue
Where is the state for the side effects stored? Say I have an AWS Lambda that I want to make idempotent. Lambdas don’t have local storage that persists across runs (unless you mount EBS volumes or something) so I presume state can be stored in a DB?
I just did a comparison between almost every hashing algorithm I could find on crates.io. On my machine t1ha2 (under the t1ha crate) beat the pants off of every other algorithm. By like an order of magnitude. Others in the lead were blake3 (from the blake3 crate) and metrohash. Worth taking a look at those if you’re going for hash speed.
I don’t have the exact numbers on me right now but I can share them tomorrow (along with the benchmark code) if you’re interested.
The PR I have lets you provide the algorithm as the caller, although I did benchmark against fxhash and I think it would be a good idea to suggest `ahash`. I'm certainly interested.
Not going to quote the whole paragraph? The paragraph that goes on to say that there's an increased risk of non-Hodgkin lymphoma and other issues?
> The German Federal Institute for Risk Assessment toxicology review in 2013 found that with regard to positive correlations between exposure to glyphosate formulations and risk of various cancers, including non-Hodgkin lymphoma, "the available data is contradictory and far from being convincing".[11] A meta-analysis published in 2014 identified an increased risk of NHL in workers exposed to glyphosate formulations.[12] In March 2015, the World Health Organization's International Agency for Research on Cancer (IARC) classified glyphosate as "probably carcinogenic in humans" (category 2A) based on epidemiological studies, animal studies, and in vitro studies.[8][13][14][15] In contrast, the European Food Safety Authority concluded in November 2015 that "the substance is unlikely to be genotoxic (i.e. damaging to DNA) or to pose a carcinogenic threat to humans", later clarifying that while carcinogenic glyphosate-containing formulations may exist, studies "that look solely at the active substance glyphosate do not show this effect."[16][17] In 2017, the European Chemicals Agency (ECHA) classified glyphosate as causing serious eye damage and as toxic to aquatic life, but did not find evidence implicating it as a carcinogen, a mutagen, toxic to reproduction, nor toxic to specific organs.[18]
Personally I trust a jury of 6-12 laypeople or the *European Chemicals Agency* over a company with a vested interest in keeping their product on the market.
> Not going to quote the whole paragraph? The paragraph that goes on to say that there's an increased risk of non-Hodgkin lymphoma and other issues?
I'm glad that you quoted the whole paragraph, but this seems to put a damper on your conclusion
>"the available data is contradictory and far from being convincing"
>Personally I trust a jury of 6-12 laypeople or the European Chemicals Agency over a company with a vested interest in keeping their product on the market.
Their conclusion is:
>the European Chemicals Agency (ECHA) classified glyphosate as causing serious eye damage and as toxic to aquatic life, but did not find evidence implicating it as a carcinogen
I haven't run ZFS + LUKS but I'm running ZFS Native Encryption.
There is an outstanding issue [0] that I've encountered on two separate machines running in this setup. The issue occurs when using ZFS Native Encryption + (NVMe?) SSDs. The issue appears to be snapshots getting corrupted (or maybe failing to create?) occasionally. It happens roughly 1/1000 snapshots in my case. I take a lot of snapshots so this occurs weekly.
I haven't lost any data yet but this issue is annoying because I get spooky alerts from ZFS warning me about "potential data corruption". To clear them I have to manually intervene by deleting the corrupt snapshots, restart the machine, then scrub.
What's most annoying is that this breaks ZFS send/recv until I intervene. send/recv was the whole reason I went with native encryption instead of LUKS in the first place.
Again, no data corruption so far (if you don't count the snapshots that get lost, which I don't, because I have so many of them). Just very annoying and tedious.
Interestingly it also seems to mix your content with content your friends tend to view. Or at least it does so tentatively, to see if you share the same interest. When I friended a pilot friend of mine on TikTok, I immediately started seeing pilot/plane content. This is after having had an account since forever and not once having seen pilot content on my For You page before. It's an interesting way to broaden your horizons, but it does turn a little common denominator, depending on what your friends like.
Similarly (and I have no way of proving this, but I believe it 100%), TikTok seems to alter your feed to incorporate videos from other people watching TikTok near you. My friends and I have made a game of this where we cast TikTok to a TV, watch videos, and try to guess which of us the video was intended for.
I've never once had a "oh wow that's a neat content discovery method" moment from Facebook/Instagram/YouTube but it has happened multiple times with TikTok. I'm not saying TikTok is the end-all-be-all of social media, but it's a good window into what social media could be, if they stopped chasing ad revenue and anger-driven content.
Oh it definitely keeps track of you social circle and shows similar content. Bedtime browsing on the same phone often yields some "I've seen this before" reactions.
