Yep I have done that before to make a 6 bay NAS. Gotta tape over a couple connectors on the sata connector but works great. Datahorders on reddit is a great sub for that sort of stuff. Guides as well as people posting when the deals happen for WD elements (external hdds)
not an argument but just an FYI for people reading about that idea for the first time: some NAS enclosures (QNAP, Synology?) and some power supplies can use the drives directly without disabling that 3.3V pin.
Most enterprise level stuff can (ie, anything that'll go into a rack, the expensive QNAPs/Synos, any supermicro I've encountered, etc.) handle the 3V3 pin issue just fine, in my experience.
It'll depend on where you live and what your goals are. If you have free-time to tinker and enjoy that kind of thing, you can build something very fast and reliable and prevent e-waste by building your own storage server with used parts on the cheap.
If you're in the United States, electricity is cheap enough that you can pick up much older SAS drives for really low $/TB cost and have it be worthwhile.
For example, I bought a used Supermicro CSE-836 [1], which is like a 3U server chassis with 16 hot-swappable drive bays and a backplane of some sort.
The backplanes vary, but mine came with the BPN-SAS2-836EL1. I paid $300 in total for the chassis itself, backplane, dual power supplies, heatsinks, etc, along with a Supermicro X9DRi-LN4F+ [2] and two Xeon E5 2660 V2s as a bundle from someone in the 'ServeTheHome' classifieds section [3]. From there, I picked up a load of HGST 3TB 7200rpm SAS2 drives on eBay for about $10 each from a recycling company. And then 192GB of DDR3 ECC memory from the same place for about $80.
I also grabbed a couple less-than-production-ready 3.84TB U.2 NVMe drives on eBay for a little over $100 each.
I think if I were to do it again, I'd have gotten slightly larger, newer drives. These are all totally fine, but I started seeing ~6TB drives for about 3x the cost per terabyte, which would pay itself off quickly with the energy reduction. The other reason is that I ended up going a little overboard; I have about 56x3TB drives right now, which is a lot more than 16, so I needed to get a couple of JBOD expansions to put them in, each of which were like $250 -- if I had gotten fewer, larger drives, I'd have had another $500 to work with & be saving on energy.
Another thing I'd have done differently is get fewer but larger sticks of memory. I have a really nice amount of RAM right now, but the energy consumption with 24x8GB isn't worth the upfront savings compared to getting 16 or 32GB DIMMs.
All the storage is in OpenZFS on Linux. The 56x3TB drives are configured as 7 RAIDZ2 vdevs, so 2 drives each are for redundancy, and 6 for actual usable storage. This leaves me with a bit over 100TB of usable space. And the 3.84TB U.2 drives are mirrored and act as a "special" device (lol, literally what they are called) [4] to automatically store small blocks and ZFS metadata.
I am sure I could have done a bunch better, but, so far, everything has been lightning fast and reliable.
I am using ZFSBootMenu [5] as my bootloader. It's cool since it is basically a tiny Linux distro that lives in your EFI and comes with a recent version of ZFS, so you can store your entire OS, including your actual kernel and such in ZFS, and you can enable all sorts of ZFS features that GRUB doesn't support, etc.
This is nice because, since the entire OS is living in ZFS, when I take snapshots, it is always of a bootable, working state, and ZFSBootMenu lets me roll-back to a selected snapshot from within the bootloader.
The Supermicro board has a slot for a SATA DOM [6], which is sort of like the form fact of an SD card. I picked up the smallest, cheapest one I could on eBay for like $15 and use that to store my bootloader. I did this so that my tiny 128GB SSDs that I use for my OS could be given to ZFS directly for simplicity instead of having to carve out a small boot partition, etc.
All in all, I'm probably out about $1750 for >100TB usable, redundant, fast storage, and a decent bit of power for virtualization and whatever else. It costs me like $50ish a month in electricity because of all the drives and DIMMs. But I was already paying 65 euros a month for a 4x8TB server from LeaseWeb to use as a seedbox, and ran out of space, so it's been worth it, even with my dumb decision to use 3TB drives.
Edit: Also, figured it'd be worth mentioning, but the way I got the chassis+motherboard+cpu bundle for such a decent price was by posting my own thread. So, if anyone reading this is broke like me and not finding anything suitable, that is an option.
You won't always find exactly what you're looking for if you just browse around. But I've always had good luck explaining my situation, my budget, my goals, and someone tends to have stuff they don't need.
eBay seems to be pretty useless right now for the chassises (chasses? chassi? I give up) due to memecoin Chia miners. Forums are your best bet if you don't want to pay scalper rates.
I'm one of the two ZFSBootMenu authors - it's great seeing people using it in the wild.
I'm not sure if this is something you're doing already, but don't forget that if you zfs send a snapshot to your storage pool, you can boot from that snapshot in a pinch.
We also (this past week) added a modifier to zbm.prefer - !!. If you specify zbm.prefer=zroot!! on the KCL, it'll only import the zroot pool. This might help your boot times by skipping importing your 56 drive storage pool.
We should have a new release out in the next few weeks, otherwise that feature is in the master branch if you build your own EFI executable.
Oh, interesting, cool! I appreciate the advice -- I am definitely not making use of all of the features.
And thanks for the software! I've been using it on all of my non-Apple devices for a while now. And my mom has been using it on their computer for a fair bit as well. It's been a total lifesaver on a couple of occasions and streamlined a lot.
I really like your writeup and I like how you shared some lessons learned and other stuff that didn't quite work out. Buying hardware can be a real adventure and it's a skill that for some of us is underdeveloped.
I'd encourage you to write more of this technical content!
Thank you, really! Maybe I am just in a weird mood, but that made my day, as strange as it might sound.
I don't think people, in general, are anywhere near as vocal and explicit about the things they appreciate compared to the things they actively hate or that frustrate them. And I think this applies even more so in careers and hobbies like tech where people don't seem to think of the things you make as consumable or creative in the same way they do with others (e.g., movies, videos, music, and other art); my personal experience has been that if someone is reaching out, it's—more often than not—to let you know about an issue or complaint.
So, thanks, it felt nice! And thanks for the reminder; I should let others know the same more often too.
> And the 3.84TB U.2 drives are mirrored and act as a "special" device (lol, literally what they are called) [4] to automatically store small blocks and ZFS metadata.
Man, even enterprise servers, sold directly to data centers, max out these devices at 64 gigabytes. Many are offered with a pair of 32GB SLC old-school SSDs (for the extra durability).
Using 3.84TB for this is an enormous waste unless you host half the GitHub yourself and have specified "all files under this or that size go to the special devices".
Better make a torrent or book or comics mirror or something on this pair of huge SSDs. I use a pair of 32GB extra-durable (SLC) USB pen drives for special/metadata on a fairly decent ZFS dataset (~9TB) and they have something like 4MB of taken space...
Have you put a power meter on that setup? I've always wanted to do something like this but electricity here is AU$0.27/kWh, and my house is poorly oriented for solar panels unfortunately.
It's a little under $0.12 per kWh here. I haven't gotten around to putting up a meter on it since fully completing it, but in the past, with a bit over half the drives + all of the ram and everything else, it was peaking a bit over 400w -- I'd guess like +- 700ish right now?
So, give or take, with a stupid build like mine, here, it's like .7 * 720 * $0.118 = ~$60.
Another thing to note is that mine stay spinning instead of idling for much lower energy consumption, cause I've always read that the stop-start cycle is what really tends to kill the drives.
I think it could be a lot more affordable/justifiable with:
- fewer, larger drives (6-8TB each?)
- fewer, larger DIMMs (16-32GB each?)
- perhaps slower, more energy-efficient drives (5400rpm?)
Also, I have seen some of the Atom C2750/C2758 boards for fairly cheap here.
If you don't need an absurd amount of RAM/PCIe/CPU, that could be a good option to save another bit of power.
> and my house is poorly oriented for solar panels unfortunately.
And yeah, same, unfortunately. Ironically, despite being in Florida, I barely get any sun; my entire neighborhood is filled with massive trees, lol.
I'd still like to look into solar at some point when I've got more time & money, though. It's been awful the past couple of hurricane seasons because of the trees -- something always seems to fall and cause a bunch of damage, so I'd like to cut down a few of the scary ones anyway (I am sorry trees!)
