Macports, meanwhile, continues to work quite well because it 1. places packages in /opt and 2. has always installed via sudo.
Macports just has a much more robust philosophy of building its own universe largely separate from OS X, and only rarely impacted by system updates. The downside is that it takes forever to install the first few packages. Homebrew offers a much faster initial install because it leans on OS X libraries for various things. But then it is much more fragile when your system updates. Macports really only depends on a slice of Xcode; from time to time updates will fail to install until you open Xcode and accept whatever new license Apple has bundled with it. But you don't get key components changing out from underneath your packages due to an Apple System Update.
My bias here is that I'm a longtime happy Macports user who has tried to use homebrew repeatedly, encountered lots of failed recipes, and gone back to Macports, shaking my head at all the hype around homebrew and the fact that is has somehow become a defacto developer default despite some really sloppy, poorly thought through practices. (And I know that's harsh but for a long time they trashed Macports right in their tagline — "Is MacPorts driving you to drink?" — which just seemed gratuitous.)
The reason I switched to HomeBrew, after attempting to use MacPorts for several years, was that the formulas all actually worked and resulted in installations that weren't horribly broken and/or reliant on 10 different additions to my PATH and environment variables. Homebrew may have some questionable policies, but at least it works well and has recipes that successfully compile. It also tells you why recipes aren't compiling.
> reliant on 10 different additions to my PATH and environment variables
MacPorts adds exactly two directories to the $PATH: export PATH=/opt/local/bin:/opt/local/sbin:$PATH
No new environment variables are added. The only required environment variable tweak is the PATH one above.
(In a corner case, where you have modified your MANPATH in your bash profile, which almost no one does, you may want to add another variable for man pages, but normally this works fine out of the box without an env var. The other optional env var is for versions of OS X older than 10.5.)
(FWIW, I've had the opposite experience — MacPorts rarely fails, every time I need to switch to Homebrew, several times now, I get a fail right out of the gate with whatever thing wasn't available or not working on MacPorts.)
I realize this is anecdotal, but my experience has been much more like robbles' than mapgrep's.
One issue is that the MacPorts built versions of things don't always integrate in a nice way with other parts of the system. For example, it used to be the case (and may still be) that LibSDL built under MacPorts would hard-code the absolute build path used to build it into the shared object, so that the frameworks wouldn't work if you moved them (e.g., if you wanted to use the frameworks to distribute an SDL-based game to your friends). This appears to be a choice that the MacPorts devs made explicitly, since standard source builds of SDL don't do that, and the standard SDL binaries don't do that.
I understand there are trade offs here, but the ones the MacPorts devs chose were not the ones I would have wanted. Homebrew, for better or worse, makes it much easier to make local changes to recipes, if you disagree with what the devs choose. However, I find that most of the disagreements I had with the MacPorts folks just haven't shown up with Homebrew.
Yes, that is exactly what I did. It still took forever to find the right Google keywords to find what I needed, and figure out the right command line invocations. And I would need to re-learn it all over if I ever upgraded SDL. By comparison, downloading the pre-packaged binaries from LibSDL.org takes all of 10 seconds...
You should also look at otool as well while you're at it. They're useful commands, just difficult to discover unless you already know what you're looking for.
Having both of those directories in your $PATH would definitely cause problems, you're loading two different collections of installed ports at the same time.
Not clear on your meaning. The fact that the two directories have completely different ports inside of them is the reason you want to add both to the PATH.
Ditto. And Homebrew's policy of avoiding installation of built-in packages by default means that you don't accidentally break your whole system when MacPorts shadows a critical core package.
Not sure about the above poster, but the only times I've heard folks claim that macports broke their system was when they were setting DYLD_LIBRARY_PATH.
edit
should be noted you really ought not do this and if you absolutely have to you should use DYLD_FALLBACK_LIBRARY_PATH
Run otool to check but Apple at least builds their stuff right and it's getting harder for companies to shoot themselves in the foot like that cause Xcode automatically does the right stuff for devs that are rushed not careful.
I switched after having to blow away my Macports directory and start over four times in the first year I used it due to unrecoverable levels of brokenness. ~4 years of HomeBrew later and the worst I've had to do is `chmod`.
Having used both, I agree. Homebrew is just better. I have not had to start from scratch with Homebrew. I had to start from scratch multiple times with MacPorts once I had installed a critical mass of conflicting packages.
If you've been using Homebrew for 4 years, you probably missed MacPorts' adoption of binary packages, automatic detection+repair of broken library linking, sandboxing of builds, etc.
That's resolved all of the fragility (and heinous build times) that previously existed.
Agreed, I don't understand why so many people use homebrew instead of macports. Macports seems to be immune to so many issues that complicate homebrew, and I love that it keeps everything in its own dir, '/opt'. Easy to see what it installed, and easy to uninstall (rm -r /opt). The commands are also easier for me to remember -- no awkward, overstretched analogy.
I often help people start hacking on open source projects and I can't tell you how many times they've made a mess with homebrew, nothing works. Replace homebrew with macports and their problems are solved, rarely to return.
Maybe in some cases homebrew installs something a bit faster, but it's rarely a meaningful amount of time and it doesn't make up for all the time spent fixing homebrew when it messes up.
For a while at least, when Homebrew was getting popular, MacPorts seemed highly moribund and destined to be abandoned. It seems to have recovered some life since then, though.
Homebrew got big in the ruby community first -- I distinctly remember trying to get Rails to work with MacPorts' MySQL install and the whole thing turning into a clusterfuck, and then someone introducing me to homebrew where the whole thing just worked smoothly because every other Rails dev was walking the same path.
Homebrew exploded among ruby devs because adding new recipes to it was a couple of orders of magnitude easier (just write a few lines in a simple ruby DSL and send a pull request on Github) than MacPorts (hack about in bash cruft and then, what, open a ticket somewhere with a patch to ask for someone with an SVN commit bit to commit it, I guess?
Nothing really changed in terms of MacPorts liveliness; development continued as it always had, and Homebrew has gradually learned why MacPorts adopted the solutions it did -- usually the hard way.
Homebrew's popularity was built on incredibly negative (and often dishonest) marketing that painted MacPorts as old and busted.
For example, Homebrew touted the security advantages of not using sudo, as compared to MacPorts, ignoring the fact that:
1) MacPorts dropped privileges when performing port builds to an unprivileged user, providing generally higher security than running with the current user's full permissions.
2) MacPorts has always also supported non-root installations that didn't require sudo.
Dishonest marketing? Really? Homebrew, a free tool was out spreading lies about MacPorts? That's ridiculous. When pretty much every major Ruby shop uses Homebrew, why would I want to kick around with MacPorts? I trust Thoughtbot, Pivotal, etc, far more than the two guys actually using MacPorts. Perhaps Homebrew 'won' in that particular community because it was better, ever considered that? Wide community adoption is a far bigger incentive to use a tool over potentially trivial conceptual disagreements.
> Dishonest marketing? Really? Homebrew, a free tool was out spreading lies about MacPorts? That's ridiculous.
Ridiculous? Homebrew's marketing tagline was "MacPorts driving you to drink? Try homebrew!". The (obviously uninformed) slams of MacPorts and Fink didn't stop there.
It was the first time I'd ever seen negative marketing against a competing OSS project.
> When pretty much every major Ruby shop uses Homebrew, why would I want to kick around with MacPorts?
That wasn't the case then, was it?
> I trust Thoughtbot, Pivotal, etc, far more than the two guys actually using MacPorts.
Yes, this is the kind of specious negative marketing Homebrew specialized in.
> Perhaps Homebrew 'won' in that particular community because it was better, ever considered that?
Sure, I considered that. Then I objectively reviewed Homebrew's claims.
Perhaps you never didn't use macports or fink a few years back?
They were terrible.
Perhaps, now, they're slightly better, but there was a time about oh... 2 years ago, when macports specifically actually made me want to punch my computer.
I can't speak for anyone else, but I can say, that for me, personally, every complaint leveled against them was absolutely spot on.
Macports never worked for me. I'm not sure if I'm not smart enough to make it work, but there was always something where updates would fail, or some package was broken.
Homebrew just worked, and when it breaks, brew doctor is typically spot-on in telling me how to fix it. I've never had the time to figure out why MacPorts gave me trouble.
I'd advise caution doing an `rm -r /opt`, and suggest it might be more precise (and generally better) to `rm -r /opt/local`. Macports will typically install to /opt/local, and in my experience, stuff like vagrant and certain VPN clients will install into subdirectories of /opt.
"Agreed, I don't understand why so many people use homebrew instead of macports."
This might seem very odd to you, and maybe I'm a weird outlier, but after having used macports for years, and installed it many times on many different systems, the requirement for full blown xcode is quite a turn-off and makes any alternative interesting and attractive.
The OSX system I am typing on now should have macports on it, and I sort of need it, but I just never have time to devote an hour to app store -> dev login -> xcode install -> blah blah -> macports.
Just FYI anyone can download XCode from the App Store, it doesn't require a "dev login". (Agree it would be nice if MacPorts only required the XCode Command Line tools like homebrew, it would be one less download.)
It can, in theory. In practice, there are some ports that actually need a full Xcode installation to build, and those would just fail without Xcode.
In an ideal world, there would be a dependency declared on Xcode for all those ports so MacPorts would work with command line tools only and fail gracefully whenever Xcode is required. However, that is currently not the case because nobody implemented an easy and automated way to tell whether a port will build fine without Xcode.
Yes, that's true, and of course that's what I'd like to do (command line tools only), but last time through that was yet another ten minutes to find the correct xcode and match things up and so on ...
I fully accept that I'm just being a complainer here, but in answer to the parents question, those are the mental blocks for me.
I started using Homebrew when I was a new Mac user because MacPorts didn't work out of the box through the corporate firewall. It's kept working for me so I haven't found a reason to try MacPorts again.
The problem is even _root_ can't add the missing /usr/local, because SIP prevents even root from touching /usr. SIP doesn't block /opt; if it did, MacPorts would also have broken, sudo or no sudo.
SIP doesn't block /opt for the exact reason MacPorts relies on /opt: It's not part of the existing operating system. In other words, this situation isn't just good luck for MacPorts, it's a direct result of the design decisions behind MacPorts, and the design decisions behind homebrew.
The specifics of System Integrity Protection are that no user, application, or process will be able to write files or modify files in the root System folder or the /bin, /sbin, and /usr directories, which are hidden by default in OS X’s Finder. The /usr/local folder remains accessible, however; it’s a long-running convention in Unix and variants as a place to stash material and software that individual users rely on.
The bit you quoted doesn't contradict what the OP stated, if /usr/local _does not_ exist, not even root can create it.
If it exists, we can use it as before.
Considering how many times I've installed OS X, I can't recall if /usr/local is there by default (excluding 10.11 for the moment). Will have to do another few installs and check.
Unless you're making iOS software, why not just use Linux?
I always have Ubuntu Server running in the background in a VM, so I get the desktop experience that I prefer (Windows) and a better Unix experience than OS X provides.
If you're OK with Apple taking away root privileges on your box, then I guess a lot of things that are fine with you are not going to be fine with other people.
Enjoy using Apple's computers...and they are Apple's, not yours because if they were, you'd be in control.
I'm curious to know if there's a reason everyone installs Homebrew in /usr/local (other than it being the default installation path). I've always chosen to install it in ~/.homebrew and haven't had any problems. Everything I install with Homebrew seems to handle an alternative prefix without issue.
brew supports pre-compiled binary packages (what they call bottles) only when you use the default install location /usr/local. If you use any other prefix brew will always compile all packages on your system, which might take a long time depending on the performance of your system. So using /usr/local saves time and energy since all brew needs to do is to download and install binary packages.
Actually, many bottles are marked as safe to install anywhere, so even if you don't use /usr/local you will still get a lot of things installed via bottles (but not everything).
Yep, I agree and that's where I install stuff on everything other than OS X. One distinction though, I think, is that most of the stuff I install on OS X I don't want to be available system-wide. I'm (typically) not using Homebrew to install daemons that run all the time or things that serve critical system/network functions, so I've never seen a reason to make them available to the entire system. I agree that goes against the Unix way but I started preferring this way of using Homebrew after I had similar problems upgrading and even updating OS X.
Also, what if for some reason a single machine is shared by two people and they need different versions of some programs installed with Homebrew? Installing everything in /usr/local isn't going to look like a good idea then.
It's not going to look like a good idea even if they need the same version. You're not supposed to run homebrew as root; it runs as your own user instead. If two users try to install things with homebrew in the same directory, you're going to end up with some things owned by one user and some things owned by the other, and things will start failing pretty soon.
A while ago I floated the idea of having a separate, low-privilege "brew" user that installs things, with the brew command automatically switching to that user, but there was no interest.
It is if by manually managed we mean _not managed by the system/OS, and not updated without user interaction_. If you update OSX and you had homebrew binaries in /usr/bin, they'd be gone. Homebrew also won't you that there are updates and never updates packages unless you decide to get a new version.
I installed mine in /usr/local/brew because I already had a ton of stuff in /usr/local managed with GNU stow before brew even existed. I don't have any problems with it.
Someone I know puts it in ~/brew and that works just fine, too. His reasoning is that /usr/local/ is for all users, and though he's really the only user on his laptop, it's just wrong to install a bunch of stuff that's just for him into a global user directory.
Mostly because of my (perhaps irrational) OCD in not wanting to touch global system paths or files, even though under FHS /usr/local is where you're supposed to install manually-managed libraries and binaries. I believe Homebrew likes to have its path owned by you instead of root, so I think it makes more sense to have stuff that's going to be owned by me to be in my home folder rather than /usr/local.
Right. The OS X image now has to contain a /usr/local directory so that it exists unrestricted after you install the OS (otherwise you would be unable to create it yourself, because /usr is restricted). It has to ship with some permissions, so it rightly ships owned by root. The installer will apply these permissions each time it runs.
Aside: I really wish Homebrew didn't encourage having a single user own /usr/local. If they're going to insist on never needing sudo to install things, it should just default to installing in your home directory.
"Apple documentation hints that /usr/local will be returned to root:wheel restricted permissions on every OS X update; Homebrew will be adding a brew doctor check to warn you when this happens in the near future."
To be fair, homebrew shouldn't need to change perms, it should follow common unix standards or at least fit in with OSX rather than require a hack. For years I've felt homebrew should really use either /opt/ or ~/.homebrew/ by default.
It did - before now /usr/local was available unprivileged with ownership.
> rather than require a hack
It only needs that now, since requirements to "fit in with OSX" have changed. That's not Homebrew's fault - they need a fast solution. Kudos in my book for responding before it's a problem.
No, /usr/local is normally 755 root:root (or whatever the effective equivalent is). Homebrew wants it to be 775 root:admin, so that users with the admin bit can write to it. That's not standard.
People keep writing here as if El Cap removes /usr/local entirely. It does not. /usr/local is there by default. You only have to go into recovery mode if you've somehow removed it yourself.
quesera's reply points-out just how bad it is, but even for just 775 root:admin I would have to use sudo (plus edit the sudoers or /usr/bin/login -p first) anyway:
Yeah, the more I read this thread, the more I'm thinking "why is that?" - `apt-get` et al have no problem with you using sudo, why does `brew` scare you into an alternative? 'because Mac'?
But /usr/local has never been writable by non-privileged users.
Homebrew depends on that nonstandard and undesireable permissions model. Apple doesn't bend to accommodate an inappropriate use.
Hombrew doesn't want to use a location that is outside the default PATH, and they don't want to install with proper permissions, so they purposefully misinterpret the "standard".
Hijinks ensue, and people blame Apple, which is silly.
A. There is no real standard here
B. Apple doesn't bend, period. They don't actually care whether the use is appropriate or inappropriate.
C. Blaming this on homebrew is hilarious
Remind me again why i should need to reboot into recovery mode to create a directory?
You need to boot into recovery mode to create /usr/local (or to disable SIP altogether) because /usr is immutable, so that code executing under an admin account can't inject itself into binaries (or into PATHs) under /usr.
I use Homebrew and will probably keep doing that because Homebrew is really convenient and just generally works better than MacPorts for me, but what Homebrew is demanding --- ownership over /usr/local --- is not in fact reasonable. It makes a lot of sense to me that OSX will want to keep perms on /usr/local (which is in everyone's PATH) locked down, and so it seems extra hacky that they're going to add a 'brew doctor' check to see if Software Update has "broken" those privileges.
The commenter upthread that suggested Homebrew should use ~/.homebrew or something by default is right.
And I contend that homebrew did not want to used ~/.homebrew because that would mean modifying files to add it to PATH and be additional testing for people that write recipes.
And that's totally fine, and I let Homebrew own /usr/local too. What I'm not going to do is freak out when a totally sane and useful security feature breaks that install mode, especially when I could, if I cared that much about convenience, just take 30 seconds to disable the security feature.
I just want to make clear I'm not freaking out (and I know you're not either).
I would hope though that out of this someone would think, "Hmm... oh drat... yeah I guess this is just one of those things that nobody predicted at first. Shot I think it was a bad idea to have a user own /usr/local/. But you know, it's a good time to improve things... hmm... I know let's build and stage as nobody and then copy under /usr/local/! We still can have binary packages always work and nobody has to mess with PATH, man pages, or dynamic linker trivia. We might have to write something though that asks for a username and password, but whoa... OSX makes that surprisingly easy... I should do that!"
It's not going to be me cause I don't use homebrew, but that would be way better than these hacks that will ammount to people being encouraged to just disable SIP cause of the headache after ever software update.
A. /usr/local has never ever been writable by a nonprivileged user. Every standard that exists, including the principle of least surprise, is emphatic on that point. The security implications of "any other way" are massive.
B. If Linus or the FHS changed their ways in this, there would be outrage. It's inappropriate and just plain dumb. [edit: I shouldn't have said "dumb", please replace with "in violation of all guarantees of system integrity".]
C. See above.
You apparently have issues with SIP. That's a separate issue from the current thread, but OK. You should reboot to securely change nvram settings that will allow you to manipulate the protected zone of the file system (including /usr) because this concession to inconvenience saves you from privilege escalation attacks.
The real question is why you deleted /usr/local in the first place. It's standard OSX, and always has been. If you deleted it before SIP, but object to its absence now that SIP applies to the directory where it lives...I'm not sure how to help.
A. Single user systems are not the same as server systems, and plenty of single user systems have had the equivalent of /usr/local as writable. If you want to argue this security is worth it, that's a different argument. I suspect most people who just want to get shit done are going to find it a huge pain in the ass.
You can also argue these people are stupid.
As for standards, vendors follow FHS and friends exactly as far as it helps them justify whatever they want to do.
If you really want to argue from the perspective that "this is standard and that's the reason it's done", that seems awfully silly to me given the layout and other permissions of apple systems.
It's pretty non-FHS/etc to have /Applications be writable, for example.
B. let's be clear: there is outrage no matter what is changed and why, and the argument is always "it is inappropriate and dumb". So this statement is fairly independent of this change.
A. OSX is not a single user system. It inherits from various Unixes, and no Unix has ever had nonprivileged writability on any directory under /usr.
FHS does not apply to OSX in any way. Why do people think so?
Also, /Applications is not writable by nonprivileged users either!
Agreed on ~"changes yield outrage", but in this case, it's just a software application vendor violating obvious historical and specific guidelines from software OS vendor, and people blaming OS vendor when things break. This perplexes me.
You might just as productively argue that there is no such thing as a modern single-user system, since Windows isn't that either. His point is pretty clear.
OTOH: The debate here is a little confusing. What SIP is doing w/r/t/ /usr/local isn't unprecedented; securelevels and the immutable flag could have the same effect on OpenBSD all the way back in the 1990s.
> You might just as productively argue that there is no such thing as a modern single-user system, since Windows isn't that either. His point is pretty clear.
Correct, and nonprivileged users can't install software globally on Windows either. Because it's now a modern OS, and that's how things are done. For very good reasons.
His point, as far as I can tell, amounts to: applications should be able to trash permissions on convenient directories and not have the OS complain or repair their damage.
Perhaps I'm being ungenerous here? I apologize if so.
On re read, I realize that tptacek is responding only to the SIP part of the discussion. OK, SIP is surprising to desktop computer users. But it's a good feature that requires a little adjustment to work with. But really not much, and none for the vast majority of users who don't muck with system directories.
He's arguing that users should be able to elevate their privileges by supplying root's credentials and then create a simple directory without having to reboot.
I more agree with you than with him, but his argument seems pretty straightforward.
Well, if you unwind far enough, I think the original discussion was about why Homebrew has trouble using /usr/local. Where "using" means "owning, adding git repos to, changing permissions on, etc".
OSX suggests /usr/local as a good location to install software, but insists that the directory itself be owned by root, and protects the parent directory /usr under SIP.
My assertion is that Homebrew is using /usr/local incorrectly (and in contravention to all expectation), and that blaming Apple for the problems thereby created is inappropriate.
I see what you are saying but please step back and take a wider view for a moment.
Even assuming OSX is a single user system (which is arguable), it is 2015 and should that single user be able to accidentally damage /usr/local? I think not, that by today an OS should help protect you from broken postinstall script or what not.
Moreover please think about this: homebrew is not wanting /usr/local/ root:admin but instead /usr/local/ `/usr/bin/whoami`:admin.
Further even on our laptops where there is just one user, I use a non-admin user. I have no problems using FUS, GUI prompted temporary admin permissions, or /usr/bin/login -p and sudo day-to-day, but homebrew has a problem with sudo.
Apple SIP has no problem with /usr/local/ 0:0 but homebrew does. Because of shortcuts homebrew has forced a number of poor decisions on it's users.
Under B, as an example of standards by another vendor who would be pilloried if they allowed the same lackadaisical approach to preserving system permissions integrity. In response to the comment that Apple was being unreasonable.
This is from RedHat. Not implemented by everyone, but modern OSes (e.g. CoreOS) seem to follow the patterns. CoreOS also do everything in their power to prevent you from mounting /usr as writeable.
Perhaps that doesn't pass your bar for 'real' though since it's not globally implemented ;)
It's been a while since I looked into it, but I opted to stick with MacPorts rather than Homebrew in large part because I got the impression that Homebrew wanted to own /usr/local rather than sharing it with other software. (Maybe that's inaccurate?) I still occasionally compile software by hand rather than through a package manager, and /usr/local has been the default target for a system-wide install on every Unix-style system that I've used in the past 20-ish years.
(Also, my wife and I both have active accounts on my machine, which makes me uneasy having files in a systemwide directory like /usr/local owned by one specific user. That also feels distinctly non-standard to me.)
I have used ~/.brew for years—it means that I have to do a slightly custom install every time I move to a new machine—but have never had a problem with it.
As others have mentioned, /usr/local may be a fairly standard place to put stuff, but homebrew's permissions model changes the way /usr/local works.
Additionally, /usr/local is used by other applications, and there's no way to ensure that it isn't written over by them or by homebrew when installing or upgrading software.
Putting homebrew in /opt/homebrew means that you can have a systemwide directory that is self-contained, can have a working unix hierarchy underneath and should work.
Putting homebrew in ~/.homebrew/ lets you have a user-specific directory that is self-contained with a hierarchy underneath.
In an ideal world, homebrew would use something like /opt/* for systemwide hierarchy and ~/.homebrew for user-specific hierarchy. We don't live in an ideal world, so we just have to make do with fudges to /usr/local permissions for now.
Unfortunately, /usr/local is a total mess now. There is no internal package management in os x so applications could (and do) overwrite files in /usr/local. These conflicts are very annoying sometimes.
iuguy is not talking about use of /usr/local, but rather the fact that homebrew makes it owned by the user who installed homebrew with permissions so that the user can maintain homebrew packages without sudo. Usually /usr/local is owned by root.
I run my Mac with a non-admin account. When I want to install or update Homebrew applications, I su to my admin account and use sudo. Just checked my /usr/local and it's owned by root. I've never had a problem running Homebrew apps from my non-admin account.
That's very clever. There is one problem though, now make clean all install is run as root (well maybe install stages now in homebrew). Imagine an env var not set (the way make vars interact give lots of way to shoot off a big toe here) and the wrong dir gets rm_-rf_-ed or it obliterates something in /bin instead.
If you don't run brew as root you at least protect places like /usr/lib, but you are protected from that and more with macports (where stages and uses nobody).
But for now your approach seems the best pragmatic course to take, thank you for the suggestion.
This sounds like the best suggestion/solution in this thread. But if you run brew from admin, why can't admin take ownership , and run brew without sudo?
So that Homebrew always requires my password to install an app, just like native OS X installs. It also preserves the default permissions. And it's not that hard to sudo.
> Homebrew always requires my password to install an app
But isn't that _less_ safe?
Instead of trusting Homebrew with /usr/local, you have to trust every formula with `sudo`? Or does it only use sudo for part of it, a `mv` or something?
That is why I said I searched and got the standard for the person who said "certainly it isn't the standard." We have the ultimate resource to know the answer to any question that does have an answer.
I gave this down voted "I googled it for you" because people need to just look up what the Standard is and than speak from a position of knowledge and not some vague non-answer.
For the still lazy the Unix File Hierarchy Standard for /usr
/usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere.
Large software packages must not use a direct subdirectory under the /usr hierarchy.
> The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable amongst a group of hosts, but not found in /usr.
> Locally installed software must be placed within /usr/local rather than /usr unless it is being installed to replace or upgrade software in /usr.
It says "The /usr/local hierarchy is for use by the system administrator when installing software locally."
That seems to kind of conflict with what you quoted "Large software packages must not use a direct subdirectory under the /usr hierarchy.", unless they mean a vendor-specific directory rather than "/usr/local".
I dunno, all I can say is that in my experience on various unixes, it is a "common unix standard" to install software that is not controlled by the OS package manager to /usr/local. I don't interpret "common unix standard" as "FHS" though -- if you mean FHS, say FHS. To me "common unix standard" just means "something that is commonly done on unix". And installing locally-installed (not using OS distro) software to /usr/local is pretty commonly done, as many people can attest. That's why homebrew chose it, they didn't just make it up.
Well I feel like an idiot since I absolutely know that but when I was searching I was looking at Unix with no /usr and than went brain dead and thought it was showing /usr as a Linux issue with no standard for /usr. Thank you for correcting me..
As I can't edit the comment, I wasn't referring to the name /usr/local as much as the permissions. It'd make more sense to store systemwide hierarchy in /opt/homebrew/* and user-specific hierarchy in ~/.homebrew/* instead.
Many unix tools are written to build looking for dependencies on default paths including /usr/local.
Putting packages installed in a custom /opt/homebrew location would make it harder to install and use software that often assumes a /usr/local possibility, but not an /opt/homebrew it's never heard of.
It is in fact a pretty common place to put non-distro installed software. And was not (and still is not?) used by OSX out of the box. It made a lot of sense for homebrew to use it.
If I have already installed a bunch of packages, how do I move everything to something like ~/.homebrew? (1) Change the HOMEBREW_PREFIX (2) move the existing aliases from /usr/local/bin (3) add the directory to my path (4) ???
Installed El Capitan yesterday. Process took almost half an hour.
Afterwards, brew just gave me a warning to do "chown" to my usr/local. That's it. Everything works fine.
I think the bigger headache will be the next time you buy a new Macbook with El Capitan or later installed on it, because you'll need to do a song and dance to create the /usr/local directory in the first place.
"T.F." is that /usr is immutable under SIP, so if you somehow manage to nuke /usr/local, you have to disable it to put it back.
This is a good feature, and it's not a new one: the BSDs have done it for over a decade with securelevels and immutable mounts --- where you also had to take the system down to make alterations.
If you think about it for a second, you'll see that anything that would have allowed you, a normal user typing at the keyboard, to create directories under /usr would probably render the security of SIP meaningless. It's the fact that you do have to go into recovery mode to hack up your filesystem that indicates that there's actually a meaningful security check happening here.
Just think of SIP's filesystem protections as literally just being "you have to boot the system into a special mode to change certain parts of the filesystem". I want that feature, and I hack up my filesystem all the time! The overwhelming majority of Mac users don't even know they want the feature and benefit enormously from it regardless. A major win.
99.99% of users will never need to touch those system directories, (They aren't even required for homebrew), and, at the same time, writing to them poses a huge gaping security hole for all OS X users. Locking them down (particularly in a way that can be trivially reversed with a user who has physical possession of the device and the password) is the right thing to do.
For the 0.01% who need to screw around with the operating system directories, they can take the 90 seconds to hop into recovery mode.
This is a situation where 100% of the user community has a reasonable solution, 99.99% aren't inconvenienced in the slightest, and the security situation goes up significantly. It's like hiding ~/Library by default - 99% of users probably don't need to mess around there, and hiding it probably reduced a lot of failure modes. (and, once again, for the 1%, chflags solves the problem in 5 seconds)
Honestly - I've always been terrified whenever I've been asked for my password during installation (I do a gut check every time an ARQ upgrades asks me for my password - "Do I trust them one more time? Well, they are my backup software, so, ...) - What if it's dropping something in my system directory?
I usually follow up with a cd /; sudo find -cmin -5 to see if things look reasonable - but I'm guess not all users do that. Also - with all the extended attr stuff on OS X, it's not even clear that a find -cmin -5 would be sufficient.
So - count me in the group that is really, really pleased that Apple is locking down those system directories by default.
To add a directory to a system-protected directory. It's all about protecting the integrity of the installed system. If you don't like/want it, you can reboot into recovery, disable it, and leave it that way.
> It's all about protecting the integrity of the installed system
Why is this important? It's trivial to reformat my drive and have a fresh install. The stuff I have that's actually sensitive is in my home directory and still unprotected =/
Your system is much more secure with SIP in place because it makes big parts of your system immune to trojans and privilege escalation attacks, the two biggest ways malware creeps into your filesystem. It's not very difficult to disable (even permanently if you are inclined), and offers a big security boost.
And yes, it is Apple "Gatekeeping"... the same gatekeeping that's kept OS X almost completely free of major hacks for over a decade. I don't consider that annoying but to each his own.
Yeah, I couldn't resist. I never understand down-voting something you disagree with so I'm shocked each time it happens to me as I rarely down-vote people myself. I just wish someone would require you enter a reason for down-voting.
I bought a new Air in mid September. I was a little annoyed that documentation seemed out of date, but it took all of five minutes to fix this. No song and dance needed, just a sudo mkdir.
This is one of the reasons I love HN: a grapevine of important things to help developers fix their dev environments, security things we need to be aware of for our servers, new APIs to take advantage of, and other important stuff.
I live in a rural part of France, use OS X, Ruby/Elixir, I'm not working for any SV startup, and indeed what the OP describes in one of my favorite things in HN!
If you only need a Unix-style package manager for things like coreutils, nodejs, tmux, irssi, vim, etc, pkgsrc runs on OS X and installs to /opt/pkg. There is no Google Chrome, or fonts, etc in it though, so if you enjoy installing those from Homebrew then this is not for you.
Yeah I see Permission denied messages everywhere now.
$ brew update
error: unable to unlink old '.gitignore' (Permission denied)
error: unable to create file .travis.yml (Permission denied)
error: unable to unlink old '.yardopts' (Permission denied)
error: unable to unlink old 'README.md' (Permission denied)
Error: Failure while executing: git pull --quiet origin refs/heads/master:refs/remotes/origin/master
Nice. I saw that I can't change also system icons ( like Automator, AppStore, Maps, Notes, etc. ).
Quick check on some forums suggest that I should do that in recovery mode. I guess it will be a mess with almost any other app that needs a bit more permissions and needs the raw UNIX filesystem.
It's not so much anything that needs the raw UNIX filesystem, it's anything that touches files that Apple has deemed not needing to be changeable, which I think app icons safely falls under. Fortunately, you can turn the whole thing off if you want.
The only problem is that after the update, the /usr/local belongs to root again. That's easily fixed with a chown though.
Apps still have access to the raw UNIX filesystem. The only restriction is that they can't write to /usr, /usr/bin, /usr/sbin, /System and into pre-installed Apple apps. Normally, Apps shouldn't need write access to these directories.
Can't see how that applies. Does FHS really say anything about the structure underneath /usr/local ? Besides which, the Mac's filesystem isn't FHS in the first place...
Okay, so can you explain to me how putting .git/, README.md and Cellar/ in /usr/local/ doesn't break that convention but putting them in /usr/local/homebrew/ does?
I'd say the "-R" might actually be harmful; for example the mysql GPL distribution binaries from dev.mysql.com installs in /usr/local/mysql and you definitively don't want to mess with the database file permissions there.
It might not be "best practice" to let non-homebrew stuff put things in /usr/local, but it happens.
This didn't work for me. Upon 3rd bullet point 'Reboot back into OS X' my system kept showing me the apple logo, black background and status bar. When the status bar progressed ~30%, information about my system would popup and then stagnate/momentarily freeze. Then the system would go to black saying 'os had a problem, rebooting in a few seconds' then...it would repeat this process again and again, without any input from me.
It does kind of make sense to me that user-owned things should kind of live in /Users/username.
Unfortunately that pattern in this case would lead to the redundant-seeming /Users/username/usr/local, but at least that's rooted to the user in the event other users don't want a "global" Homebrew install.
Also, I believe (even if it kills some ignorant build scripts) that Homebrew lets you reconfigure where this directory lives, anyway.
It's very common for linux applications to store stuff in $XDG_DATA_HOME (~/.local/share by default) (config files and caches go somewhere else)
In my ~/.local I also have a bin/ directory (created by some Haskell tool), so rather than putting stuff in /Users/username/usr/local, I think it might make more sense to recreate the (needed parts of) a unix fs hierarchy in /Users/username/.local
There's also ~/LibrarySupport on Macosx, if I remember correctly... but I think that contains user editable configuration files as well
Homebrew can be installed anywhere, and after ~2 years of having it outside of /usr/local I have not run into any problems. There is a disclaimer somewhere that states some "packages" may not work well with this setup, but that seems like an upstream bug to me. Things should not require things to live in a specific path or prefix.
Where does your homebrew live? I have a few different users on my mac that I want to all share resources so I've kept it in /usr/local (also the "easiest" because it was the default).
TBH, things like this make me really mis debian. It's really nice when your package manager is a first class citizen (and geared towards software development rather than consumption).
I considered a ~/ directory but wanted it available for the rare occasion when there is someone ssh'd into my machine (though I didn't go so far as to update the system-wide shell configs to use it.
Why does brew need to install everything owned by my user in /usr/local? I would honestly prefer if everything installed by brew was owned by root:wheel and brew would gracefully abort if I forgot to run it with sudo.
Is all that current mess just so that I don't need sudo for brew?
Using /usr/local is just fine. But I feel uneasy when a system directory is owned by just my user. What if there is another user on my system that also wants to perform a brew update?
Using sudo would also be a security advantage since not every program running under your username would be able to modify all binaries in /usr/local.
/usr/local is for machine local packages vs. packages that are meant to be shared across multiple machines. /usr/local is still under the control of the local sysadmin though and not a user.
/usr/local makes sense to me if you've installed brew with sudo. But as other comment says, it's weird to me to have files owned by login user in system directory. It's such a rare scenario now I guess of multiple people on same machine, but isn't user home directory place for files owned by login user?
Typically /usr/local relies on group permissions: the accounts that you trust to build and install stuff are added to, eg. 'admin' group, and /usr/local is group writable for admin with the group s-bit set.
This is what you used to see when you tried to use sudo with brew:
Error: Cowardly refusing to `sudo brew install`
You can use brew with sudo, but only if the brew executable is owned by root. However, this is both not recommended and completely unsupported so do so at your own risk.
I don't know if it's different now, but a quick Google search indicates that brew still behaves like this. I would like to use brew with sudo, but I don't want to use unsupported methods.
When XCode needed some license approval after the recent upgrade to support iOS9, Brew warned me that license can only be approved ( programatically ) by root. I tried to sudo brew and received the warning you showed. That was 1 week ago.
Why would they run as root, just because they were owned by root? Only brew itself would need to run as root (just like most package managers do anyway).
Will homebrew switch the default install location out of /usr/local for new installations then? I doubt most developers will want to dive into recovery mode just to install wget.
you don't need to. You only need to if you have manually deleted /usr/local for some reason. /usr/local is exempt from system integrity protection.
The only problem is that its owner gets reset to root on every OS update, whereas Homebrew wants its owner to be the Homebrew user.
The advantage of /usr/local as the installation root is that /usr/local/bin is in the default PATH of the OS and that setting the PATH in a way that it takes effect no matter what starts an application is a bit... tricky in OSX as there are many ways to launch an application that doesn't go through a shell, so .profile and friends aren't enough.
Are you sure it gets reset to root? I've been on since the first public beta, quickly fixed the homebrew problem, and haven't had any permission issues since. Admittedly I can't say if normal updates will behave different from the beta ones.
I can't be sure obviously, but if it's done once by the installer, there's no reason to assume it won't ever do it again. So in the worst case, you'll have to re-chown every time you install an update.
But I've found that many GUI applications do not take /usr/local/bin into account. It's like only terminal applications, which are rightly affected by terminal environments, care about it.
My MacTex (Latex) installation broke for the same reason. Luckily, all I had to do was set put "/Library/TeX/Root/bin/universal-darwin/" in my $PATH and change the path of "pdtlatex" and "bibtex" to the GUI tools I use. But still, I lost about an hour or so trying to figure out what is happening.
I was just reading the Arstechnica article, which mentions this:
> Instead of allowing developers to put files wherever they want, El Cap offers four canonical safe locations for applications, support files, and drivers:
> /Library
> ~/Library
> /usr/local
> /Applications
> The local or system library directories are the preferred stand-in for /System, and /usr/local is the preferred stand-in for /usr, /bin , and /sbin. However, Apple strongly recommends that developers use /Applications for everything if possible, since that keeps all of an app’s files in a single location for ease of uninstallation.
The first thing I did after the upgrade was disabling SIP. Now my system acts exactly the same way before with all the goodies (except rootless) el capitan has to offer.
Having root not able to write system stuff is good for people that like to blindly execute shell scripts from the internet but I think apple should add something that makes it easier for developers to disable it.
I don't know if there is a new method yet, but before you had to boot into recovery and execute `csrutil disable`. In the early days there was a boot flag you could set but that got removed quickly.
It has to be a method that can not be accessed by malware - the very thing SIP is meant to protect from. I believe that is the reason the option ended up on the recovery partition.
To me, the fact that you are directed to install Homebrew to /usr/local is wrong since there's no guarantee how /usr/local is going to treated by Apple in the future and this is now an example of that. The only directory that Apple, probably, doesn't have control is your home directory.
I've been using homebrew for a long time without issues because, in my opinion, I installed it on ~/homebrew. I wonder why is that so difficult for others to use.
Should I create this folder or install brew now? I've been thinking about installing it on my macbook and doing some development on it but haven't yet. Last night it told me the OSX update was available. Which order should I do this? Or should I just not put brew in /usr/local ??
I definitely recommend installing El Capitan first, followed by Homebrew. That will lead to less possibility of the upgrade screwing with your Homebrew installation.
Using a $150 Chromebook from Walmart with the built-in ssh and my VPSs or xfce via crouton when I feel like it. Works great for web development. No UNIX/linux compatibility problems since it is actually Linux.
I installed El Capitan this morning and was expecting to have to do this - I did not. chown stayed owned to me properly, brew functioned as it was before... everything fine.
Did you do anything in advance of the install?
Eg some people were mentioning earlier to move files out of usr/local before upgrade then copy them back in. Do anything like that?
Did you do anything in advance of the install?
Eg some people were mentioning earlier to move files out of bin/local before upgrade then copy them back in. Do anything like that?
When I first read this I was a little pissed at Apple, but what if brew were to install everything to /brew? It doesn't need to install into /usr/local.
You'd run into exactly the same problem. Apple doesn't provide /brew in the default image, so something would have to create a new inode in /, and that something presumably requires SIP be disabled (assuming / is protected in SIP at least).
Even if it worked, you'd be no better off than just using ~/.homebrew or something like it. Their whole reason for wanting /usr/local is that (a) /usr/local/bin is in the default $PATH, and (b) it can be written to by a normal user. Using /brew gives up (a) anyway, so you may as well just put it in a user's home directory somewhere.
I think you misunderstand the problem. /usr/local exists in OSX, and is one of the recommended install locations for user software. But if you change the permissions on /usr/local, OSX will interpret that as damage and fix it. Homebrew does this, which is in conflict with all uses of /usr/local in Unix history.
> and (b) it can be written to by a normal user.
Critically, no. This is a security risk. A larger one back when multiuser systems were the only Unix systems, but still a risk. Violates all standards, even the ones OSX doesn't attempt to comply with.
Oh, I know it's a huge problem on multi-user systems. I was just pointing out that homebrew treats it that way so they can have sudo-less access. Technically I guess it's not world writable in a default homebrew install though, just owned by a normal user. Which also defeats the purpose of /usr/local really, but I've never found that design decision from Homebrew to be very good.
It is entirely possible I misunderstood SIP. The first thing I did was disable it, and I haven't bothered any more about it. My impression though was that it locked down all "system directories", however Apple chooses to define that. Which would disallow changing the permissions on /usr/local, but also would disallow creating new files or directories under a protected directory. Is that not what it does?
Those fixes applicable to Yosemite will be available as a separate Security Update. Apple usually supports the current OS and the OS X version behind (for example, this for Mavericks was released in August: https://support.apple.com/kb/DL1834?viewlocale=en_US&locale=...). So security is probably not a reason to upgrade.
No, and thank God for that. The only thing Rootless prevents is system modification like iOS jailbreak, i.e. prevents tinkering with the system with things like debugging running processes or modifying system files (for example, utilities that modify the UI theme don't work with Rootless). Most malware out there is not that sophisticated, does not try to modify the OS and can still happily live in any other location of the system, like any other program.
Why do you feel this way? I mean, seriously, at the end of the day we're talking about computers and operating systems. That's it. No reason to be upset. It's just stuff.
If it's in a home directory, then the stuff installed is really only usable by that person. I can see some instances where that would be desirable, but I would prefer that the stuff I install is usable by the whole system, even though I'm the only one on the machine.
Users should have full control over their own machines. System Integrity Protection is obnoxious paternalism at best, and at worst, it's just another step toward iOS-ification. If Apple keeps this up, I expect a revival of Linux desktop distributions.
> Users should have full control over their own machines.
Agreed.
> System Integrity Protection is obnoxious paternalism at best
Couldn't disagree more. The user maintains full control: you can disable it at will, temporarily or permanently. Protections like this are becoming essential as a result of current malware threats. Thanks to System Integrity Protection, we're now much more protected against rootkits and similar attacks. This is a good thing.
I feel like I have to keep defending it because it's perceived as this removal of user freedoms, when it's factually just not the case–you can still do everything you could do before, you just have to restart into recovery mode to do so (to be able to do this without the restart into recovery mode would undermine the security this technology provides). System Integrity Protection is closer in spirit to something like SELinux than anything.
Many of the "I want full and complete control of my system at all times!" people mostly just mean "I want to continue doing it the way I have done it for the last fifteen years and will resist any change to that traditional method on principle".
Immediately upon reading 'csrutil disable' I thought of 'setenforce 0' and the fact that this is usually the first measure taken to deal with SELinux by many users, usually against better judgement. However this, like SELinux, begs the question: is this a good user interface design? Or simply a small stumbling block for malware authors? I suppose the devil is at least partly in the details: can a kernel exploit at runtime easily flip a few bits somewhere and disable SIP?
Well, if we somehow knew for sure that you'll always be able to disable it, sure, this feature is fine.
But Apple can't be trusted -- they sell two very popular lines of computers (iPhones and iPads) that are completely locked down. It's not unreasonable to be worried that they'll try to move towards locking down their other lines also. Not to mention that they're a remarkably developer-hostile company and probably wouldn't care much about the outcry it would raise from the HN crowd.
It seems a little unfair to judge a company against their hypothetical future actions. I agree that if they do remove the option to disable it in future then that's a problem, of course.
> they sell two very popular lines of computers (iPhones and iPads) that are completely locked down.
Note that iOS is less locked down now than it has been in the past. Previously, you had to enroll in the $99/year developer programme to run your own code on your device, but now you just need to register for a free developer account. So they're moving in the right direction, even if this is still too restrictive for some.
I'm not judging them based on hypothetical future actions. I'm not really judging them at all; I'm just uneasy and worried. I'm predicting that the probability of hypothetical future actions for which I'd judge them is too high for my comfort.
I'm also judging the hell out of them for their past and current action of keeping iOS locked down.
The developer program thing, IMO, is a bit of a red herring; everyone who can buy an iPhone can also pay $99, so pragmatically there is nothing you can do now that you couldn't before, unless you're a student and your parents bought you your phone. Sure, it's a step in the right direction or at least not negative, but I don't think it's particularly meaningful.
> unless you're a student and your parents bought you your phone
It's a bit tangential, but speaking for myself this actually matters. I was a student until recently, I bought my own phone, but I couldn't afford the developer programme (especially when it was two separate programmes, one for OS X and one for iOS). Now, I can build apps and only need to buy the developer programme when I want to deploy them, and then I only have to subscribe to the one programme for both OS X and iOS too. So honestly it does matter to some people.
I hear you about your concerns though. I'm hopeful, with the iPad Pro, that they're actually going to open up iOS more in future, not less. It'd be nice to do proper development on an iOS device. A BSD-like jail environment (e.g. a full sandboxed unix environment with shell access, with maybe iCloud Drive mounted in the filesystem) would be lovely. If they want to sell 'pro' iOS devices, they must be thinking about these sorts of problems.
I think SIP gives me more control over my machine, not less. I don't want programs installing junk into my system directories. Don't mess with what my distributor shipped me. I've had this same complaint on Linux and OS X for years: to install system wide, you are typically asked to become root, and then some random installer has free reign to wreak absolute havoc.
It turns out Haskell Platform was installing symlinks in /usr.[0] Why? There was no need for that and no easy way to get it out or even know it was doing that in the first place. So as a solution people had "uninstaller" programs that would look for gunk like this left by other programs? Ridiculous.
I'm glad Apple is locking this down. If you don't like it turn it off.
It's an unusual linux installer these days that requires root. Programs aren't going to write junk into your system directories on linux unless you give them permission to. I'm not sure how adding an additional layer of complexity on top of this will solve anything. Now, rather than asking for root, your random installer would be asking you to open up SIP permissions for it.
They do have full control - see the bit about being able to disable/enable via recovery mode. I for one am very happy to have moved on from an era where desktop security was abysmal and compromises lurked around every corner.
Isn't the goal to prevent malware from modifying core system files? How different is this than a simple mandatory access implementation? As long as it is possible to disable it, should I choose to, I am not outraged.
The common problem of today OSes is that shall malware breach security at least once, it can hide in the system and you will have no idea that you have it. SIP is a good step towards fixing that problem.
A couple of year ago I made a prediction that Apple is going to try to throw out OSX long term. While I thought it would proceed at a greater pace, gradually locking down things because of "security" can easily have double utility. I hope I'm wrong, I like my OSX machines ...
Yeah, iPhone a very very unpopular for this reason, right?
You have the same amount of control, this is an option you can disable if you need to. I think the amount of people who need that is very small, even among technical crowd.
> System Integrity Protection is obnoxious paternalism at best
As is ASLR? Users should always know what addresses their libraries are loaded at in their own machines! While we are at that, let's remove file permissions. That's paternalism - users should know what they are doing.
Nobody else sells nice, super-useable, high-end UNIX personal workstations.
Agree with your point -- Apple is a developer-hostile company and their OS is bad for people whose work involves tinkering with computers. Still the best development platform, though, which is a sorry state of affairs.
You could have a stable free environment that will work for a decade or a proprietary one that's designed to cause older computers to break and require expensive upgrades. For various reasons many people love the latter.
I also don't mind having excuses to upgrade my hardware but I dislike being driven forward by planned obsolescence, which is basically how every piece of Apple hardware I've ever owned has ended its life. Exploding batteries. Horrible one-way OS upgrades. I thought it'd be helpful for art and music software, but I found the opposite. I thought it'd be good for programming but had to invest absurd amounts of time in installing basic open source development software. Never again.
I'm not totally sure how exploding batteries and optional OS upgrades are driving you forward through planned obsolescence. I feel like maybe in your apparent "rage" you lost the logic thread of the point you actually wanted to make and conflated a few things into an incoherent rant.
+1, and yet my post got downvoted to oblivion, lol. I'm sorry I bothered all those developers at starbucks with their macbooks and their hard-to-prononunce coffees.
Macports just has a much more robust philosophy of building its own universe largely separate from OS X, and only rarely impacted by system updates. The downside is that it takes forever to install the first few packages. Homebrew offers a much faster initial install because it leans on OS X libraries for various things. But then it is much more fragile when your system updates. Macports really only depends on a slice of Xcode; from time to time updates will fail to install until you open Xcode and accept whatever new license Apple has bundled with it. But you don't get key components changing out from underneath your packages due to an Apple System Update.
My bias here is that I'm a longtime happy Macports user who has tried to use homebrew repeatedly, encountered lots of failed recipes, and gone back to Macports, shaking my head at all the hype around homebrew and the fact that is has somehow become a defacto developer default despite some really sloppy, poorly thought through practices. (And I know that's harsh but for a long time they trashed Macports right in their tagline — "Is MacPorts driving you to drink?" — which just seemed gratuitous.)