I'm going to take this opportunity to plug my favourite open source project - the Nix package manager[1].
It can work as a universal homebrew replacement (works on MacOS, Linux, WSL and can be easily ported to most BSD variants), comes with a huge collection of packages[2] and produces its own reproducible source builds. Like homebrew, it's a hybrid source and binary based package manager (if you haven't done anything to modify the build, it will likely be downloaded from a cache of pre-built binaries[3]). Unlike something like homebrew-cask, it will never download the pre-built .dmg file from the developer's website - with the obvious exception of proprietary software.
It can also work as a great AUR/ports replacement on Linux systems. Fedora doesn't provide FFmpeg or an up-to-date version of a package you need? No problem, just get it from Nix! All the advantages of a rolling release distro, without actually having to use one.
Due to its functional nature, it comes with a wealth of advantages over homebrew and other traditional package managers[4]. Once you get past the learning curve, creating your own packages or modifying existing ones is a breeze. It can create disposable development environments with dependencies of whatever project you're working on, without having to install them in your system or user profile! Check out the Nix manual[5] for more information.
It's so flexible that people have built a Linux distribution where your entire system configuration is a Nix derivation (package) - with atomic upgrades, rollbacks, reproducible configuration and much more! [6]
Essentially, all the benefits touted above apply here, but it is worth noting that Guix is a younger project. The author was originally a Nix dev, but found the DSL to be too awkward to use in practice, and opted to use Scheme through and through. Yes, Emacs bindings are available.
Also, Guix can now produce Dockerfiles, if that floats your boat:
The main thing that probably turns people off to Guix is that GuixSD specifically rejects any non-free software. While a noble goal, it leaves most people's hardware unusable.
The main allure to Nix/Guix is to be used for the internals of an OS. For example, NixOS is the most stable and maintainable system I have ever used, even though the documentation is absolute shit.
You don't need to fork the entire OS, though - just provide the relevant non-free packages on top of the regular OS.
Guix devs themselves have hinted that "adding impurity to a free OS is much easier than removing impurity from a non-free OS", so I suspect they may not be categorically opposed to the idea of non-free packages.
Do you use NixOS as a desktop OS? If so, what advantages do you find over something like Arch? I understand the repoducability from configuration files are often touted as a selling point, but I'm not reinstalling my OS regularly so I don't really see the advantage.
When I upgrade or (un)install a package, I get a completely clean fresh installed system. If upgrading my video driver broke my system, I can just pick the most recent version of my install from the bootloader. When I want to start writing code, I can use nix-shell to create an isolated environment with all the dependencies I need, and get to work, with one command.
At any moment, I know that my entire OS install is completely clean, devoid of any cruft, even if I have been installing and removing obscure software for months. I don't have a mess in /etc, in /usr/share, anywhere.
> I'm not reinstalling my OS regularly so I don't really see the advantage.
With NixOS, I'm never reinstalling my OS, because every update leaves me with a fresh new system, guaranteed.
Personally I run Arch on my desktops but NixOS on my servers. I mostly use Arch because of all the relatively uncommon stuff I occasionally use from the AUR (somehow I doubt STM32CubeMX or ESPtool will enter Nixpkgs any time soon), but I've occasionally ended up cursing that I couldn't do a Nix-style rollback when an update broke something important.
It's not Haskell, strictly speaking. There are some weird global state mutations in NixOS (I.e. the pkgs object) that aren't as intuitive as you'd like. Also, the novel file structure makes composing packages that aren't already built to do so quite difficult. Trying to get eclimd to work with emacs and eclipse was a giant pain.
I expect that your issue is more with the fact that there is a mega state object that contains almost everything - not that it is somehow global, which it isn't. You can't access it unless it is passed to you.
I breezed through your explanation with great interest and became more curious than ever to try Nix out at some point - I've heard tidbits about it over the past few months and found it to be a fascinating project.
Then I read some of the other comments in this subthread, about Nix being Haskell and Guix being Scheme, and how the internals don't conform to the principle of least surprise within their development domains - and became sad.
On the one hand, there are going to be teething problems, that's fine.
On the other hand, if I'm learning a new package manager, I ONLY want to learn package management. My brain gets really confused if I don't stay highly domain-specific when learning new things. So if you present me with "you need to learn B to grasp and work with A" you'll just get a duck trying repeatedly to jump up over the curb and failing. Looks hilarious, maybe, but very impractical.
I don't know Haskell or Scheme yet, which puts a great dampener on me using Nix for a bit.
As another practical example, emacs and vi do greatly interest me (particularly emacs' flexibility), but I'm not interested in having to learn Lisp or memorize a Nethack labyrinth of single-character keyboard commands (and the scopes for what works with what and where).
I suspect I might pick up emacs at some point after I've played with Lisp/Scheme/et al for a while, but that's not right now.
TL;DR: It's a strain for me to learn new things (basically do anything that requires focus and can't be done with passive autonomy) if my mental stack already has anything in it. AKA, I have a monumentally stupid variant of ADHD that gives me learning difficulties.
To be clear, Nix isn't haskell. It has its own domain-specific language. The documentation is a total mess, and months into using NixOS, I still don't really understand it.
That being said, NixOS is the best damn OS I have ever used. Period.
Installing NixOS is a total breeze. It's magical.
1. Boot live CD
2. Mount partitions
3. Write a simple configuration.nix
4. nixos-install
5. Boot a complete system, and log in with the user you defined in your configuration.nix (with a hashed password and everything).
nix-env lets you install packages in your home directory (without root).
nix-shell lets you create a special environment with specific packages, etc. and run a shell in that environment.
nixos-rebuild [switch test boot] (as root) creates a new system according to your modified configuration.nix. If you use switch, it (re)starts updated/new services all on its own. If you broke your system, just reboot, and pick the second most recent boot entry, and you will be back where you started.
TL;DR NixOS is well worth the effort. The benefits far far outweigh the headaches, and there is only room for improvement. You will start with a system that you cannot break!
While I agree with most of your comment and am writing this on my NixOS desktop, installing is a breeze only if you don't want to encrypt your root partition or don't want to use it on a macbook or something that mature OS'es usually do quite nicely out of the box.
I have my bookmarked set of gists to guide me through these things and it took me days to get it right the first time.
Also nixpkgs are a bit of a mess, you usually get a version of a package someone has bothered creating a PR for. This might be very old or bleeding edge. For specific versions you can be SoL. It is of course fairly easy to have your own .nix of the package and PR's are usually accepted quickly but the repo usually doesn't keep previous versions.
And then there is a whole world of pain regarding language specific packages. Python is especially painful for me.
It took me ages to find the page which hints at nix-shelling projects but this is a pita and requires a completely new workspace setup that is not compatible with people that don't use Nix.
Regardless I love NixOS, but to say this is perfect, not even close.
Please stop saying the documentation is "total mess" and "absolute shit" because this is far from the truth. There is comprehensive documentation available for Nix[1] and NixOS[2]. There is an online search for packages [3] and NixOS configuration options [4]. Sure the documentation will be incomplete in places but that doesn't make it "absolute shit", I have found it to be very helpful generally.
Comprehensive is not the same as user friendly, or well paced, or organized in a way that will assist new people learning.
I've seen plenty of projects that were exceptionally well documented. Every method and class had inline docstring documentation, every module has a separate module docstring block. 100% PEP 257 compliance was enforced as part of the CI tool chain... helpful but only after I was sufficiently knowledgeable about the way the project was being used to get any benefit from it. The the sheer volume of documentation meant that many people had written extremely terse explanations, often devoid of any of the contextual information that would significantly improve someone's understanding of how the code in question was be used.
Point is... The volume of documentation or the percentage of documented features, has no direct bearing on its usefulness to any particular user.
I was personally donating monthly to Nix foundation for past few years, the maximum amounts that don't trigger tax concerns, but finally pulled my funding after the team continued to bury their heads in sand about state of docs for newcomers.
The main site docs are incomplete and reference style. The Wiki which had some use cases also had a banner saying it was all deprecated.
So you get newcomers who can't orient themselves, and even advocates like thomastjeffery in this thread who love it but months in still think the docs are wildly unhelpful.
Interestingly, guys who understand it very well are no longer capable of seeing the problem. They forgot what it's like to have no idea how any of it works, so can't describe the happy path between ignorance and the lightbulb coming on -- or even see that path isn't lit.
For now, the only way to walk that path is either absolute focus on learning it w/ no time for a day job, or have your hand held by someone enlightened (which is expensive for your MVPs compared to fixing the docs).
Like I said, after months of using NixOS, I do not understand how to write a simple Nix expression.
Where is this explained?
I am speaking from experience when I say the documentation is "absolute shit". Most of the useful documentation I have read comes from the deprecated wiki.
When I first installed NixOS, I had to read the source code for simple things like defining a user or getting samba set up, and still had to rely on the deprecated outdated wiki.
Three current state of documentation for NixOS is spread between the manual, deprecated wiki, and the source code itself. The manual barely shows how to get a simple system working, but as soon as you reach an edge case - which Nix itself is fantastically suited to handle - you have to start digging.
In time, documentation for NixOS will be mature and usable, but right now it is severely lacking, moreso than any other aspect.
I guess not. I'm just still running Slackware, so I haven't needed to grapple with it just yet - and I'm holding out hoping to find that one other distribution that also doesn't use systemd so I can use that instead. :P
I've heard good things about Void Linux, I'll likely have a look at that at some point.
On the flip side of the coin, I've kind of accepted that everyone who uses Linux has to put up with systemd now, so I'll accept my fate eventually. It's just that I value constructive and positive communication, I don't like the systemd team's blatant arrogance, and I consider it a corporate abuse of power that nothing has been done to curb or deal with it. (I hope I never have to deal with Red Hat in a business setting. Feathers, sparks, fur... everything would fly, and it wouldn't be pretty.)
If you don't want to memorize single-character keyboard commands, why does vi greatly interest you? How do you plan to put it into use without those commands?
- people talk about how rapidly they can perform complex edits with few keystrokes, which is something I most definitely can't do with Geany (a Notepad++ clone that made me wish Notepad++ worked on Linux); and
- it's a standard utility that you can generally expect to be available in some form on literally every UNIX (especially server environments).
So, basically I'm being pragmatic. I'm at the point where I'm wondering about building my own text editor (something I've wondered about for several years, actually), but that immediately introduces a software-installation problem if I want to be able to bring that editor to other machines.
I have a small modicum of knowledge about how to get by in vi, but it's extremely barebones - exiting, copy/pasting (I think the wrong way, heh), not much else.
I definitely could learn significantly more, but I just have a massively hard time fielding the idea of "okay let's play with the text editor for a few hours". Sure, once I was actually doing it I'd probably find it fun, I just can't convince my brain's broken scheduler that it's not actually a waste of time :D
As an aside: I'm wondering about building my own editor because I want something that can jump around text similarly to the Canon Cat and which has syntax highlighting and autoformatting similar to QBasic/QuickBASIC - and I'm frankly petrified by the idea of adopting a text editor that uses Blink for rendering. Blink (aka Chromium) is like Flash meets Java for UI design nowadays, it's almost like crack, and I want to do everything I can to stay as far away as possible from it.
This is not because the systems using it are not well-designed or -built, but because I fundamentally disagree with the philosophy and idea of using a Web rendering engine for tasks like communication or text editing. HTML5/CSS/JS/WASM is the world's biggest Rube Goldberg "safely run arbitrary code from remote servers" in existence; it wreaks enough havoc on the Internet, I don't want it leaking into my desktop too.
My main complaint is the W3C: they produce specifications that are too narrowly scoped and at too frequent a rate for the associated implementations to be operationally cohesive. The code for each feature in isolation might be excellently architected and written, but as a whole the amount of memory and CPU churn undeniably associated with rendering engines is significantly higher than would be found in platform-native (or even cross-platform) solutions.
You can definitely argue "well that's how things are now, the Web will improve as time goes by"; I'd argue that no, things have always been a mess in the Internet community, and nowadays the driving forces are all companies with advertising and/or consumer-consumption interests. This in itself isn't necessarily wrong, but it means that the focus is "is it good enough? okay, ship it" and there's no conservative holding back until what gets released is of extremely high quality.
It looks like you can search the nix package list but not browse it? Based on clicking on the first few packages listed, most Nix packages are apparently for Linux. Is there some way to see which packages are available for Macs?
Signing does help! Apple can revoke the certificate the instant the malware is discovered, blocking all future installs. Whereas Handbrake is going to rely on XProtect[1] which entails Apple refreshing all Macs in the universe at some delay.
While I'm at it, distributing through the Mac App Store would have added further protection against this attack on a couple levels. 1) It's harder to compromise with a MITM attack in the first place. 2) macOS wouldn't install an update signed by another party like this.
[1] "We have been informed that the process to update the definitions for OSX's XProtect feature started this morning, so this should start rolling out to machines automatically soon if not already."
Not exactly. Signing would have minimized the overall damage by letting Gatekeeper halt all installs from pretty much the moment the malware was discovered. And distributing through the Mac App Store would have prevented this MITM attack in the first place.
$99 for Apple, $300+ for Windows. It adds up. I suspect Handbrake gets plenty of donations, but most projects don't. Heaven help you if you want to build a signed driver for Windows.
"The HandBrake and HandBrake Documentation projects are not accepting monetary donations. Please instead consider donating to the VideoLAN non-profit organization and the Blender Foundation."
Yeah, fair point. I guess I was erroneously looking at that from the standpoint of the financial scale I'm working with right now (don't have a job, etc) and didn't catch myself. (Hmm. I'll keep an eye on that.)
If distributing a commercial (or enterprise) application then yeah it's peanuts (or a rounding error). Good point.
Irrefutable point though - it's one thing to want something for free as a consumer, but quite a different story as a creator or developer. I totally get the rationale behind charging to keep quality high.
Charging also creates a contract and with that contract be able to enforce a minimum standard. That helps greatly too.
% codesign -vv HandBrake.app
HandBrake.app: code object is not signed at all
In architecture: x86_64
However, code signing only goes so far. In there past, malware has been spread in signed application bundles as well [1]. The only good solution is sandboxing. Unfortunately, virtually nobody sandboxes macOS apps outside the App Store (where it is a requirement). These days I think twice before installing/purchasing an application outside the Mac App store.
God dammit. I downloaded this a few days ago and sure enough, I'm infected. What are reasonable mitigation steps to prevent this in the future? I noticed handbrake said it must "install additional codecs" which is mighty odd, but I didn't think much of it at the time.
Is there a security product on OSX that would have prevented this?
Is there a security product on OSX that would have prevented this?
Little Flocker. Unfortunately, it was sold to F-Secure [1].
I'm infected
Personally, I would wipe the whole system (rather than just removing the malware as in the steps that they describe) and change all credentials (passwords, SSH keys, etc.).
Ironically-- the reason I didn't have little flocker installed... I had just wiped my machine. When I reinstalled (the reason I redownloaded HB), flocker was on longer available.
One possibility is that the credentials of one of the developers is obtained. Then this only adds marginal security, since the page with hashes could have been updated as well.
I downloaded it 3-4 days ago and I don't appear to be infected as far as I can tell but this gives me serious concerns about if I might be in some other way. Sigh.
What's the alternative? Using different passwords for every account is not practical without some way to store them, and using the same password makes you even more vulnerable. We could write them all down in a big book or something, but that leaves us vulnerable to physical theft and loss.
Theorycrafting without a real idea here but how about a physical device with private keys and challenge-response authentication from a site. To login you have to press a button to sign a specific message as a specific person.
Then they can't silently gain access to your account persistently. They can gain access until you logout but then they have to wait for you to login again. Or they can change the key but then you'd notice that you can't login.
To protect against password compromise in general, use 2 factor auth.
For this to really work, you need a way to detect compromise of the password. This way the 2nd factor holds you over until you've managed to rotate all important enough passwords.
Alternatively, separate trusted hardware with an interactive (i.e. challenge-response) protocol. Something like a TPM or yubikey.
Two options come to mind, neither perfect, but options.
1. Use a password pattern. Something like 8 random digits that you memorize and then part of the domain name or business name. Such as "goo" for google. Put them in whatever order you like. Now you've memorized one pattern but use a unique password everywhere.
2. Use a predictable algorithm instead of a password. Their are web based services for this. You enter the domain name and then "encode" it to a password. That is typically not reversible.
These fall down some when you have to change a password or when a system has requirements that don't match your password (like requiring a number or symbol). Other users will mention other limitations as well.
You made me curious so I tried this app. Unfortunately the UI is horendous. So far I tried the macOS, iOS, .jar and web version. Each version generates different passwords for the same site. This app has the most confusing UI I have seen in a long time, with each version being wildly different from the other. I still have no idea how to have multiple users for the same site, or even if setting the user makes a difference at all (seems not as it does not change the password). But most important: this app generates different passwords for the same input on different platforms. Kind of beats the purpose of having it in the first place.
That is an interesting idea. echo URL + password | sha/md5
There are 'stateless' password managers that work that way. It does not really protect against malware. If your user account is compromised by malware, what holds them from reading out your password and applying the same procedure to obtain password for interesting sites? You'll still be updating your password everywhere.
What you want is a second factor that uses a challenge-response mechanism with user interaction (e.g. U2F Yubikeys that require a finger press to start the challenge-response).
Even if they have plaintext password (which is often not case), this is just shasum. Feel free to guess which password (and which exactly scheme) I used to generate my password for news.ycombinator.com, if (of course it's now not like that :) it is:
You have now (after hacking whole hackernews db) access to my password ONLY for the hackersnews. Which was the original goal of the method: to use different passwords at different sites, which if compromised (password), do not reveal scheme used to generate it for different sites.
That's not the point. The malware would have access to your complete machine, possibly with root privileges, what holds them from reading your master password with a keylogger when you type it in?
It does not provide more security against trojans than a password manager.
I was replying solely and only to the acusation that after revealing plain text password on one site (which was generated using said scheme) you disclose every one.
This is simply not true.
In addition (but I didnt address that), there is no single keychain/password store to steal by the trojan. I can use it anywhere, using only my head as a 'storage' machine.
You now have 3 responses as of my comment and they're all worthless because they completely miss the point of Password Managers. PMs are not elegant, they are not ideal, their limits are very well known. They exist purely as a hack around the objective real world reality that authentication in general in the modern world completely sucks and depends on passwords. And that that reality is something we've got to live with and deal with for the foreseeable future.
Yes, we shouldn't have passwords for authentication at all, ever, period, and shouldn't have had them for at least a decade now. All the tech has long existed to switch to hardware backed public key cryptographic auth, and even to do so in a way that is far more user friendly then endless passwords and more secure at the same time, a sort of win-win that is quite unusual. We should all have HSMs (be it in the form of tokens, cards, compliant hardware in phones, or whatever) holding our private keys, and websites should only have our public certs. Entire classes of issues (like everything associated with password databases) would be perfectly and completely eliminated forever. There would be no dependence on any 3rd party services. Users would only need to remember at most a couple of PINs and that's it.
Back here on Earth though I can count every single site I've used in my life that had certificate based authentication and still have plenty of fingers left. Maybe U2F will make a bit more of a dent, but for a long time to come passwords will be the core of a lot of people's most critical personal security (like most of their money) and online identities. To have any value passwords need to be unique, and need to be fully random, and also have to deal with layers of extra crap that have been piled on top like password policies, arbitrary allowed characters (universal UTF? hahahaha), arbitrary minimum/maximum lengths, "security" questions (which should of course just be random strings lest they undermine the point of having a password but probably have their own special character restrictions), etc. Humans cannot remember all this garbage for more then a handful of sites. Password Managers are a practical solution to this mess of the "worst one except for all the other ones" variety. They effectively replicate (badly, but that's not their fault) some of what an actual decent key system would offer by default. They make attack scaling harder.
In short they're the best match to the most typical threat models most people face. Any good efforts to replace passwords period with keys is to be applauded, and if successful would eventually (years down the line) make password managers obsolete by making passwords themselves obsolete. But in the mean time password managers matter and people talking down at them due to issues that don't actually match general threat models are doing a terrible disservice to the public.
Password managers aren't implicated in the compromise of the OS X keychain or your browser... this is FUD.
Your browser's native store is probably unencrypted [1], and the OS X keychain password can be snatched with a clone of the native password prompt. Neither is true for any respected password manager. They keep you safe.
The OS X keychain is a password manager, right? Just as your browser. Your browsers native store is probably encrypted, see your own source.
How would a respected password manager guard against an infected machine?
Take keepass as an example, they state: The actual problem here is running specialized spyware (as the same user and with the same rights, like KeeFarce assumes). If you are doing this, everything is over. An application cannot protect itself in such a case; all modern PC operating systems (Windows, Linux, ...)http://keepass.info/help/kb/sec_issues.html#keefarce
Also, at the bottom of the page: Neither KeePass nor any other password manager can magically run securely in a spyware-infected, insecure environment. Users still are responsible for the security of their PC.
If your system is compromised you're fucked, it's game over, they could install a key logger that records your 1Password master password or just replace the 1Password binary with an infected one.
I don't understand how I'm supposed to verify the checksum if I've already installed (and run) the HandBrake.app ... and long since deleted the .dmg installer file ????
I think the main concern here is the state of GUI apps in macOS and Windows. Popular apps in these platforms are mostly closed-source, even for personal side projects. For the few open source GUI apps, no package manager provides support for building GUI apps from source.
I wish package managers would make it easier to build GUI apps from source, or even provide their own binary packages for GUI apps. I really feel reluctant to install most GUI apps on macOS and Windows because I can't trust that the build/distribution platforms for these apps are properly secured.
Being open source isn't going to stop someone replacing a download on a hacked server.
Also, the package manager you're after is brew, specifically 'brew cask', which installs GUI apps.
EDIT: By the way, building from source doesn't stop someone replacing the source package the package manager uses. Sure you can look at the code in the package, but are you going to? It's more likely you'd check the code on Github, if at all.
The cask description is misleading though. Cask deals with applications which are too complex to compile (/shrug) but provide an .app or .pkg, or with things that only provide a compiled .app or .pkg (not open source)
brew has GUI applications. Easiest one I can think of (because I use it) is Pidgin.
I'm mostly worried about the security of the build/distribution platform, which is why I would would prefer a reputable package manager to provide binaries built and distributed using their own infrastructure. If this isn't feasible, better support for building from source would at least ensure that the binaries are correct, provided that your own computer isn't compromised.
Homebrew Cask currently can't address this kind of situation, since the binary is still built and distributed from non-reputable sources.
The correctness of the source code itself is surely a problem, but it's better than having to trust random binaries built and distributed from non-reputable sources.
Usually package managers on linux distros, to use an example for comparison, tend to check checksums of downloads for security purposes during any installation. For MacOS users, I guess I understand they want to use software not blessed by Apple, then isn't homebrew or whatever supposed to do the same thing?
Handbrake for Ubuntu is distributed through the official Handbrake PPA, and this means there is still more transparency: there is a public record of the source code that went in the build process, and the binaries were automatically built from that on the Launchpad build servers.
There's a potential downside too: If a trojan gets into the PPA sources, it will be automatically downloaded and installed by users.
Yep, but it's not just the official source, it's also the guy's computer who uploads the source to the PPA.
Just something to keep in mind when adding a PPA: you're tying your machine's integrity to the integrity of the PPA and its keyholder(s), and this trust is tested each time you update your packages.
Most Mac users don't use anything like Homebrew. Typically you just download the software as a .zip file or equivalent, unpack it, and run it.
Apple does support code signatures for apps downloaded this way through their Developer ID program, but you have to pay $99/year to be a member of their developer program in order to do so.
In theory, you could still sign downloads using a code signing certificate you got elsewhere, but system wouldn't check it for you, and few people would bother to do it manually.
>In theory, you could still sign downloads using a code signing certificate you got elsewhere, but system wouldn't check it for you, and few people would bother to do it manually.
Hence, we have occurrences like the OP. The point is checking checksums or (better) signatures is the most common way to mitigate it. Otherwise, this will continue to happen. Dare I say that is the entire reason repo managers of all stripes do this in the first place.
You can't turn Gatekeeper off anymore, at least not without using the command line. You can bypass it on an app-by-app basis by just right-clicking an app and selecting "Open" there, which is what you'd have to do if you're downloading apps like Handbrake.
Even for homebrew users, installing GUI apps via homebrew usually means you'll need full Xcode installed. I, for one, only have Xcode command line tools installed and never use homebrew to install GUI apps.
Actually, Homebrew discourages building GUI apps. It's in the process of moving all GUI apps to Homebrew Cask, which doesn't support building from source. The exception is for GUI apps targeted for Linux, which can't be converted to a macOS-style appliation bundle.
Yikes. Missed this by 1 day. I updated Handbrake to 1.0.7 on 1st May to compress a bunch of videos. Was a little surprised to see it wasn't signed but after scanning it with ClamXav I figured I was safe and installed it on every Mac in the house so I could crank through my project faster.
If I understand correctly even if I had in fact downloaded the compromised version ClamXav wouldn't have detected the malware?
This kind of stuff is extremely worrying and really strengthens Apple's case for signed application binaries across the board.
Are package managers like Homebrew and MacPorts not also susceptible to this kind of binary poisoning?
I can't believe this. I literally downloaded handbrake like 45 minutes ago! Luckily I got the proper version, but boy oh boy, it was a close call. I think I'll reinstall claXmav on all my macs.
If you look at the history for handbrake in cask, it looks like it was first updated to 1.0.7 with the correct hash, then later the hash was changed with no version bump, and then reverted. That's crazy! Why didn't alarm bells go off when the hash changed? https://github.com/caskroom/homebrew-cask/commits/master/Cas...
Sadly Homebrew is terrible for this, the developer guide for how to get the hash for the files you're downloading is literally to just test run the download with an empty hash in the brew file, and it will calculate the hash from the file on the website. In this case Homebrew would not help, as galad87 mentions here https://news.ycombinator.com/reply?id=14282116&goto=item%3Fi....
And, what's your proposed alternative for calculating a hash?
Btw, we treat hash changes very seriously in homebrew-core; they are never merged without a confirmation from the upstream. Unfortunately Cask apparently doesn't live up to the same standard, but Homebrew Cask is not really Homebrew.
This is highly unpopular here, but I think we need something like the iOS/Android permission system on the desktop. Windows Metro apps do this, but they are highly maligned here.
I should be able to install a transcoding app like Handbrake without giving it all the keys to the kingdom and allowing it to do whatever it pleases.
This is highly unpopular here, but I think we need something like the iOS/Android permission system on the desktop. Windows Metro apps do this, but they are highly maligned here.
I expect it’s the walled garden strategy that is unpopular with a techie crowd, not the idea that an OS should have more nuanced permissions than (to a first approximation) “any installer is either crippled or root”.
A lot of us have been concerned about the very coarse security model in mainstream OSes for decades, but there’s a huge amount of momentum behind the Windows, macOS and Linux ecosystems now. Any new platform with a fundamentally better security and application management system would have the usual chicken-and-egg problem in terms of users and software base, and without something else to drive it (such as inventing a whole new type of device with Android and iOS on smartphones) that’s a tough barrier to get over.
I think this is what Flatpak is trying to solve - each application is run in its own sandbox and requests its own priviledges in a manifest file.
It is currently in a pretty early stage of development, but it's usable and looks pretty great. Maybe this time creating a new package manager will make the package manager wars on Linux quieter :)
The problem is that both AppArmor and SELinux[1] run applications unconfined by default.
Also, sandboxing GUI applications is more work than just slapping AppArmor or SELinux on top. First, users still expect to be able to open arbitrary files from their home directory (which would then be added on a case-by-case basis to the application's sandbox). Second, X11 does not provide UI isolation, so a sandboxed application can still read all keystrokes (and thus credentials). The Flatpak people realize this and are working towards the goal of eventually sandboxing GUI applications (on top of Wayland).
Apple actually nailed this pretty well. All applications in the App Store are sandboxed. If you open a file in such an application, the 'Open'-dialog runs out of process and links the opened file in the application's sandbox.
Unfortunately, there has been quite an organized campaign by a subset of macOS app developers to discredit the App store and especially sandboxing. Sure, it would be nice if macOS supported more fine-grained permissions and Apple has been slacking there. But for the end-user sandboxed App Store applications are much more secure than downloading unsandboxed applications from random developer sites. As this and previous incidents have shown.
I know essentially nothing of the windows system API, but it would have to be implemented at that level.
The Microsoft hate would suggest they are restricting the permission system to Metro to encourage development to move towards Metro. It could also be that the underlying metro API is much easier to build a permission system for than the old windows one.
From what I've read though, these kinds of permission systems tend to only be a barrier. Sandboxes seem to always have some exploits. It would certainly be an improvement though.
With regards to file-systems, a simple linux based file-ownership model would do essentially the same.
Windows already supports sandboxing old api applications, through something called AppContainer:
> Isolation is the primary goal of an AppContainer execution environment. By isolating an application from unneeded resources and other applications, opportunities for malicious manipulation are minimized. Granting access based upon least-privilege prevents applications and users from accessing resources beyond their rights.
I'm not sure why this is not really used. Maybe requires some recoding from the original developer, maybe it's too inconvenient and creates frustration for users.
Even with file systems permissions, there are a lot of problems. For example, a transcoding app like Handbrake should be allowed access to users files, but ideally it would only be able to access VIDEO files. Any attempt at reading any other kind of file (source code, text files, password stores) should be forbidden. This requires a very expressive permissioning system.
Along with the fact that Apple updated the built-in sorta-antivirus in MacOS to detect it. But it only detects an SHA1 hash on the original DMG. If someone rebuilds the DMG or puts the malware with another app and builds a DMG, it'll bypass the MacOS sorta-antivirus.
The legitimate version of Handbrake is not signed, at least not in the standard Apple way. I know a cert costs some money, but well, this is why they exist and why the latest versions of MacOS have hard-to-override warnings.
Not necessarily. For issues like the one discussed in this thread, a simple ssh-style "trust initially" would have sufficed, and would have prevented the malicious installer from running.
Note that I'm not proposing this as a replacement for the current cert system (which you pay into), but as a replacement for unsigned executables.
What about creating different users on a MacOS system to do different things? Wouldn't this mitigate exploits like this?
Why shouldn't I create a "Tommy Transcoder" user on my system? That user would have the Handbrake app in his own Application folder. I assume that Handbrake will run correctly without needing to be installed in the system /Applications?
I already do this for a few items of software. Maybe it should be SOP to do this for most/all software?
Or what about installing most apps into virtual machines and using VMWare to run them?
I do recognize that such an approach couldn't be used universally. E.g. VMWare itself must run on the native machine, and with elevated privileges.
I'm interested in "defense in depth". No single technique can defend against all possible exploits.
Didn't this also happen somewhat recently? How can this be prevented? The window could be reduced by actively monitoring mirrors? Could BitTorrent help mitigate this because the torrent file validates data and isn't under the control of the parties?
Hosting on GitHub solves one problem, but creates another.
Chrome for example has a sort of a bloom filter which is used to check all downloaded executables. This will raise a nasty warning if the thing you downloaded is not a "popular" download.
For obvious reason, this check is disabled for a bunch of sites, like github, sf, ...
I know for a fact that some malware authors host their stuff on GitHub exactly to bypass this Chrome check.
Do you have any more info about this? I've downloaded random executables from a lot of unpopular (and in some cases, newly created by me) sites and never seen anything pop up in chrome warning me not to download it.
> Chrome also does checks on executable files (like ".exe" and ".msi" files). If the executable doesn't match a whitelist, Chrome checks with Google for more information, such as whether the website you're accessing hosts a high number of malicious downloads.
At the time I looked at the implementation in the Chrome source code, but I remember that it took me a while to locate it.
> The Download Mirror Server is going to be completely rebuilt from scratch.
Am I alone in thinking that this is irresponsible? Why not move releases to github?
Why aren't you going to start signing macOS binaries? I find this offensive. Thanks for potentially compromising users because you couldn't be arsed to pay for a certificate.
Please feel free to donate the funds necessary to cover the Apple ID signup for multiple years as well as the potentially necessary Apple hardware to install the SDK on and sign the app (since Apple artificially restricts code signing to only run on their own hardware) as well as any necessary training hours to the team to setup said environment so that they can sign the software they're giving to you for free.
It's $100 for the signing and $500 for a Mac Mini. Any professional software developer would make that kind of money in a few days. And I would bet that an open source project as big as this gets at least that amount in donations yearly. It's completely irresponsible to not sign releases after something like this happens.
And if they did get $600 in donations in the last year... they should dedicate all of that to the Mac build? Even if they've already spent those funds on hosting and other expenses?
A user of a free product telling the developer of said free product how the developer should be spending the money that said user isn't giving them takes a special kind of entitlement.
A developer should feel a little responsible about the product if it has a large amount of users... It's simply gaining bad reputation for the developer with such news because of saving $100 a year.
If the developers are all students or something and can't pay for that, please let me know if there's a page asking for it and I'll pay or any of the other 1000 users who can.
Hold on… your premise doesn't make sense. Apple's SDK only officially runs on Mac - but that applies to the entire build process, not just codesigning. Anyone who is already producing a Mac build either has access to a Mac build host, which they can also use to sign, or is using third party tools to cross-compile from another OS.
In the latter case, there is also a third party tool capable of signing binaries:
It's worth noting that you need access to a Mac you can trust to setup your Apple developer credentials on as well as iSign and then export your credentials and certificate from each time you get a new certificate. It's likely you won't be able to do this in the Apple store. While you or I may have a colleague or trusted friend we could do this with, many developers will not.
It does say that in the readme, but I don't think it's actually necessary: you can get certificates and provisioning profiles through the web portal without going through Xcode. (For certificates, it has you upload a standard CSR file.) Though developers may not know that…
Admittedly, if you don't have a Mac build machine, (a) you can't test your builds, and (b) you can't use the official SDK (including, e.g., system header files) without violating the terms of service. Not that many people care about that, but if you don't mind ToS violations you may as well just install pirated macOS in a VM (which is easy enough in practice). Perhaps Apple deserves blame for not having a legal way to run a macOS VM on non-Apple hardware; certainly it makes life harder for open-source developers that want to play by the rules. Still, these obstacles have nothing to do with signing.
A user of a free product telling the developer of said free product how the developer should be spending the money that said user isn't giving them takes a special kind of entitlement.
There is a very simple solution. Get a dev program membership. Sign and sandbox the application and make it available in the Mac App store for $1.99. People who want a signed, sandboxed Handbrake pay the equivalent of a cup of coffee. People who want to play Russian roulette can download Handbrake for free on the web page.
Given that Handbrake is extremely popular, that small fee will probably cover the $99 per year plus a Mac Pro (or two) within no time.
>It's $100 for the signing and $500 for a Mac Mini. Any professional software developer would make that kind of money in a few days
Sorry but you are living in a bubble. There are thousands of software professionals who don't make that kind of money in India, South America and many other places.
Explain that to me slowly: if you're enjoying the work of someone from a place where the economy is vastly different and worse than where you live, that person should feel bad about your loss or inconvenience?
Handbrake is a very popular package. It's probably downloaded about a million times a year. It would be very easy for the author to pay for a code-signing certificate with either donations from users or unobtrusive, inoffensive amounts of advertising.
Best practices call for releases of executable code to be signed for the reasons outlined here.
Or you know, don't be a entitled jerk of an user who can't be bothered to support Free (as in Speech, NOT as in beer) Software.
Come on, if every developer that uses any kind of Free Software decided to donate every hour the dollar amount of one HOUR of their work for the projects that THEY need, no one would be in this mess.
Or you know, don't be a entitled jerk of an user who can't be bothered to support Free (as in Speech, NOT as in beer) Software.
(Shrug) I've authored a large amount of software that's free as in both. I sign my releases for my users' protection, and also to avoid looking like a careless dilettante. I expect others who are serious about their craft to do the same.
If that makes me an "entitled jerk," well... meh. There's no way to respond to an accusation like that except to own it.
While its great that you have a spare $100 a year and the money to invest in Apple hardware to digitally sign Mac builds, not every project will have those resources. Projects that don't have the resources will often publish hashes so that users will be able to check to ensure their download is legitimate before using it. Handbrake does this: https://handbrake.fr/checksums.php
For free, the author could simply GPG sign all releases for all platforms and suggest users verify them. Then, the decision to run untrusted code is the user's decision and bypasses platform monetization.
Then, if suffient and continual contributions of money and expertise permit, official codesigning per platform could be added as another layer (defense-in-depth).
Wow. Waste of bits (your comment). Please take your political agenda some where else. This is the 3rd time this author has been compromised. For $99 it seems a bit negligence on his part. Donations are not excepted so...
"The HandBrake and HandBrake Documentation projects are not accepting monetary donations. Please instead consider donating to the VideoLAN non-profit organization and the Blender Foundation."
It's cheaper than that. I've let my Apple developer subscription lapse but I can still sign packages until my certificate expires. You can probably also get a reasonable second-hand Mac mini for $200.
So you are looking at an up-front cost of $300 then around $25 per year.
Travis CI is free for open source projects, perhaps they could better invest their time into working with that then rebuilding.
I'm assuming the price for hosting the mirrors isn't free, and probably more than $100/year. If they moved to github releases they could recoup some money.
You still need access to Apple hardware. From my other comment on a similar tool: "It's worth noting that you need access to a Mac you can trust to setup your Apple developer credentials on as well as iSign and then export your credentials and certificate from each time you get a new certificate. It's likely you won't be able to do this in the Apple store. While you or I may have a colleague or trusted friend we could do this with, many developers will not."
Thanks for correcting me. There were actually two comments by two different users (the links above appear identical since they have been shortened, but the full URLs are unique). I should have looked more deeply into the matter before posting the assertion and links - my apologies.
While users could help financially, a project should not delegate the responsibility of security down to users until they explicitly donate for that especially for a project of this amount of user base.
You're welcome to compile, sign and release your own Mac binaries, and even charge a fee for them if you want in order to cover the costs. It's OSS after all. Or even make a polite request or ticket on the problem. But don't tell OSS devs what to do in a snarky voice.
At the same time, when your project takes off past a certain point, don't you gain some amount of responsibility if you know that you have become an attack vector?
If you don't want to take on that responsibility, then you could just quit hosting binary builds, and focus on the source. Then it becomes "someone else's problem" to deal with hosting binaries, and providing security.
Good point; though approaching the problem in an indignant and righteous fashion doesn't achieve anything; offering to help with a solution is more in line with the (ideal) OSS ethos.
Sigh.. This could be somewhat repaired by making a beta-release, distributing to devs and testers. Once confirmed good, rename file and release via IPFS. The key here, is if multiple devs did this, the hashsum would prove the file being shared.
Any one client that's been hacked or infected would show up as an improper hash and easily spotted.
[1] https://news.ycombinator.com/item?id=11234589
[2] https://news.ycombinator.com/item?id=12403768