My secret protip: old Fujitsu desktop/nuc PCs. At least in Germany (Europe?) they are cheap on ebay since a lot of businesses use them and upgrade on a regular schedule.
In one case I had a nuc where on Linux after enabling power saving features for the sata controller, idle usage even fell to 5W when the pdf claimed 9.
Having an actual pc instead of a random sbc ensures best connectivity, expandability, and software support forever. With almost all sbcs you're stuck with a random-ass kernel from when the damn thing was released, and you basically have to frankenstein together your own up-to-date distro with the old kernel because the vendor certainly doesn't care about updating the random armbian fork they created for that thing.
Parkytowers is site about repurposing thin clients of various kinds, it's a goldmine for finding out power consumption, Linux compatibility, possible hardware mods, etc: https://www.parkytowers.me.uk/thin/hware/hardware.shtml
Don't share this information too far and wide, you might drive up the price for these in the second hand market, which will hurt us dirt-cheap-pc-gluts.
> I cannot say no to cheap compute for some reason.
I sympathise - for me I think it comes from growing up with early generation PCs that were expensive, hard to get, and not very performant; now you can buy something that's a supercomputer by comparison for almost nothing. Who can resist that?! I'll think of something to do with them all eventually...
I bought one of those thin Lenovo clients for about $200 and use it as a home server with Debian, it works great for pretty much everything and is a lot more bang for your buck than raspberry pi or a brand new mini-pc.
The only downside is that it doesn't have space for multiple 2,5/3,5 disks, but that is just personal preference anyway.
in my experience those come from mining and other FIFO operations that are shutting down. source: used to de-com those, wipe em, and get them ready for bulk sale to a different group.
Have had great success with this myself. Oddly resellers on Amazon seem to be the best source I have found, just search for something like "HP ProDesk" or one of the generic corporate-focused lines from other manufacturers and find one that fits your budget. Maybe filter to 100-300 dollars to get rid of the new stuff. There's also a surprisingly vast selection of recycled commodity servers and similar on there, too.
That's... Odd. I didn't click my own link yesterday after posting, but that was straight from Google and it did load just fine. Now I get the login too. But for completeness sake, another example: https://www.fujitsu.com/global/Images/wp_ESPRIMO_P758_E94.pd...
That happens. I've once foolishly linked brochure page for a trash-found HSM on social media, supposedly that logged too many referrers server side, and the URL was mis-configured into something else less than a day later. It wasn't even a 404.
ODroid H-series SBC's are standard Intel CPUs with (at least for the H2+) Linux supported hardware for pretty much everything (haven't tried running X on them though :-P )
they are my favorite 'home server' currently...cheap, standard, and expandable - oh! And SILENT! :-)
In general some of the Intel Atom CPUs intended for embedded applications support the so-called In-Band ECC, where a part of the memory is reserved for storing ECC codes.
Two of the three ODROID H4 variants use the "Intel Processor N97" CPU, which is intended for embedded applications and it appears to support In-Band ECC, even if this is not clearly advertised on Intel Ark (i.e. at other CPUs of the Alder Lake N family, like i3-N305, at ECC Support it says "No", but at N97 it says neither "Yes" nor "No", but it is mentioned that it is intended for embedded applications, not for consumer applications, and the embedded models normally support In-Band ECC).
The ODROID H4 BIOS allows to enable In-Band ECC on ODROID H4 or ODROID H4+ (the latter is slightly more expensive at $139, but it has more I/O, including two 2.5 Gb/s Ethernet ports and four SATA ports; to the bare board you must add between $10 and $20 for the case, depending on its size, and a few other $ for SATA cables, RTC battery and optionally a cooling fan; you must also buy one 16 GB or 32 GB DDR5-4800 SODIMM, so after adding shipping and taxes a NAS would cost a little more than $200, besides the SSDs or HDDs).
From Unraid forum thread[1] there's links to another forum[2] and a wiki page[3]. Not yet sure myself what all of this means, but it looks interesting.
Correct. I have a 5 node Fujitsu esprimo D756/D757 cluster that has i5-6500 CPU's and 96GB RAM and 5x NVMe + 5x HDD that usually sits around 80W total. Removing HDD and reducing RAM would drop the power usage but in my case it's not important to go after the last Watt.
I bought them them for 50 euro a piece without RAM and disks.
I was under the impression, that for most (popular) chip families, like RockChip, Allwinner, Amlogic, some assorted Broadcoms, .. the Mainline linux kernel support has mostly been sorted for, and it's only the stragglers like Hisilicon, Huawei, Most Broadcom, Qualmcomm where mainline support is not on their priority list?
If the application fits, choosing one of the BSDs with the more popular chip families can work well. In other words, if the BSD crowd likes an SoC, you'll get a good, stable system running with good kernel support. In my case it's the RockChip family, specifically from PINE64.
Examples:
- PINE64 Rock64 running FreeBSD 14.1 replaced an aging RPi3. I use this for SDR applications. It's a small, low power device with PoE that I can deploy close to my outdoor antennas (e.g. 1090mhz for dump1090-fa ADS-B). It's been really solid with its eMMC, and FreeBSD has good USB support for RTL-SDR devices.
- PINE64 RockPro64 running NetBSD 10. I have a PCIe card with a 500gb SSD M.2 slot. NetBSD has ZFS support and it has been stable. This lets me take snapshots on the SSD zpool. I generate time-lapse videos using the faster cores.
You don't get 100% HW support (e.g. no camera support for RockPro64) but I don't need it. The compromise is worth it in my case because I get a stable and consistent system that I'm familiar with: BSD.
Maybe, but regular distros on x86/x64 thin clients are even more sorted out. GPIOs are better handled through an Arduino clone over USB than with scripts running on inherently laggy desktop OS.
After configuring the vendor uBoot to chainload into a newer uBoot-compile with JustEnoughUEFI compiled in, you can just launch the standard Debian Arm64/UEFI install iso on many/most(?) popular SOCs.
W.r.t. GPIOs, I agree, that delegating that to an e.g. Arduino connected via USB/UART or one of the available internal(often RTC), or external(HDMI/VGA) I2C connections as an I2C slave is the preferred solution.
The cheapest computers with Alder Lake N CPUs, like ODROID H4 and many others, have only a single SODIMM socket and they are limited to 48 GB (which works, despite Ark advertising a limit of only 16 GB).
However there are many NUC-like small computers made by various Chinese companies, with AMD Ryzen CPUs and with 2 SODIMM sockets, in which you can use 64 GB of DRAM.
Those using older Ryzen models, like the 5000 series, may be found at prices between $220 and $350. Those using newer Ryzen models, up to Zen 4 based 7000 or 8000 series, are more expensive, i.e. between $400 and $600.
While there are dozens of very cheap computer models made by Chinese firms, the similar models made by ASUS or the like are significantly more expensive. After the Intel NUC line has been bought by ASUS, they have raised its prices a lot.
Even so, if a non-Chinese computer is desired and 64 GB is the only requirement, then Intel NUCs from older generations like NUC 13 or NUC 12, with Core i3 CPUs, can still be found at prices between $350 and $400 (the traditional prices of barebone Intel NUCs were $350 for Core i3, $500 for Core i5 and $650 for Core i7, but they have been raised a lot for the latest models).
EDIT:
Looking now at Newegg, I see various variants of ASUS NUC 14 Pro with the Intel Core 3 100U CPU, which are under $400 (barebone).
It should be noted that this CPU is a Raptor Lake Refresh and not a Meteor Lake CPU, like in the more expensive models of NUC 14 Pro.
This is a small computer designed to work reliably for many years on a 24/7 schedule, so if reliability would be important, especially when using it as a server, I would choose this. It supports up to 96 GB of DRAM (with two 48 GB SODIMMs). If performance per dollar would be more important, then there are cheaper and faster computers with Ryzen CPUs, made by Chinese companies like Minisforum or Beelink.
I have since years, an usb disk connected to my Fritzbox and it works amazingly well. I have a real NAS, but i ended up never using it. Fritzbox with the USB disk is enough to use as scanner dropbox, saving pictures, documents, ROMs.. Sometimes the simplicity beats the whole complication of having extra devices
You can buy a DAS (Direct Attached Storage) enclosure[1], some even support RAID. If your Nuc is multipurpose, you could then run a virtualized TrueNAS guest (BSD or linux) in QEMU and give it control of the DAS block device for ZFS pools. Being able to run a virtual NAS that actually gets security updates on demand is pretty neat - TrueNAS has an excellent API you can use to start/stop services (SMB, SSH, iSCSI, etc) as well as shutdown the vm cleanly.
1. Newer DAS devices connect using USB-C, but USB type-A/e-SATA ones can be found.
Edit: figuring out how to run TrueNAS as a guest OS was a nightmare, the first 5+ page of results will be about TrueNAS as a host.
Isn't running a NAS on top of USB storage very strongly discouraged? TrueNAS cautions against it.
I also want to set up a NAS on a mini-PC with some sort of attached external storage, but I haven't been able to get past this blocker. USB is the only external interface these mini PCs typically support.
There are issues with USB from a compatibility standpoint.
I think its mainly a factor of the ubiquity of it, there are SO many poor controller chips out there, even when you buy seemingly reputable hubs/drive cases. Its hard to find a good one sometimes.
I did, however, stumble upon a gem early on, it was a 3.5" usb drive case from BestBuy which has since been discontinued(because it was good). Never in 15 years has any of the half dozen ones i got dropped from thier system.
This is more than i could say about alot of pricey stuff on amazon sadly.
Its typically manifested as a random loss in connectivity to the system.
Similarly,
heres a very low power writeup I did for using 2.5" drives with a dedicated power hub/splitter.
http://www.jofla.net/?p=00000106#00000106
This will still have issues if the mains lines sag (a pole goes down somewhere), but you can fix it with a reboot remotely. Other than that it works great.
This was/is definitely a labor of love, primarily as I've come from a time when all you could get for a server were huge boxes idling at 50 watts, so i felt guilty of all the power I used to consume.
> I also want to set up a NAS on a mini-PC with some sort of attached external storage, but I haven't been able to get past this blocker
You could also buy a PCIe to multiport SATA (RAID) card, but you'll also need a bigger[1] NAS-like case[2] to house the hard drives and multiple power cables for the drives
1. Depending on how mini your PC is and how many drives you want to attach to it
2. I have noticed any case that's sufficiently NAS-like (hotswap trays) is sold at a premium.
> Isn't running a NAS on top of USB storage very strongly discouraged? TrueNAS cautions against it.
You're right, and I don't recommend USB as the only backup solution. I an old eSATA enclosure - not sure if that's an improvement over plain USB - as a secondary backup and keep the device on long enough to complete)
Used PCIe HBA cards pulled from retired servers can be found on eBay for ~$50. They have external facing ports and/or internal facing ports. External is the way to go if you're using a small form factor PC like a business class Lenovo. These are almost all low profile cards, so they will fit in any SFF PC with a PCIe slot. There are special cables which will connect one port on the card to four SATA- or SAS-based disks.
The PC's PSU will need SATA power on its cables or else you'll need to scavenge a separate PSU and use the paper clip trick (or better yet, a purpose built connector) to get it to power things on without a motherboard connected.
Once you have all of that, then it's just a matter of housing the disks. People have done this with everything from threaded rod and plastidip to 3D printed disk racks to used enterprise JBOD enclosures (Just a Bunch Of Disks, no joke).
Total cost for this setup, excluding the disks, can easily be done for less than $200 if you're patient and look for local deals, like a Craiglist post for a bunch of old server hardware that says "free, just come haul it away".
Check or r/DataHoarder on reddit or ServeTheHome's blog
It looks like some NUCs have M.2 adapter slots that allow PCIe. Seems possible to do with a NUC, you probably can't put 20 drives with it, but 12-16 extra drives sounds feasible.
they make external HDD chassis that connect via USB. I don't have any experience with them so I can't comment on their reliability, but search for "ORICO 5 Bay USB to SATA Hard Drive Enclosure".
FWIW, I wouldn't recommend Orico. I don't live in the US so my options are somewhat limited but I found a local retailer that carries Orico. I've had five of them in the past five years and four died within 12-18 months.
If it was just one, I'd put it down to random bad luck. But with that many failures I assume they are doing something stupid/cheap.
Usually they would simply fail to power on but sometimes individual slots seemed to die (which RAID just loooooves).
And having an entire enclosure fail and waiting days/weeks for a replacement sucks as you lose access to all your data.
I eventually bought a Jonsbo N3 off of Aliexpress and PCI SATA card (to support more than the 2-4 drives most motherboards support) and that has been working well for months.
I've never tried Orico, that was just the first brand that came up when I searched. I suspect these things are fundamentally unreliable, especially because they are powered by external AC adaptors, meaning there is no real ground between the two switching power sources (one in your PC, the other in the HDD caddy). It's either due to that, or due to the very sensitive signaling along the line, that eventually you get USB disconnects (if you try to run it as an appliance) that wreaks havoc on filesystems, particularly RAID.
The Jonsbo N3 is not comparable. I own one as well (how quiet is yours? I upgraded the rear fan but my CPU fan is noisy), but it's a complete PC case, not an external HDD array.
Highpoint has some decent-ish toaster-style drive docks. I have an a couple of the older model with dual drives dual usb-a ports - 5422A - but the Highpoint RocketStor 3112D seems available for $70 with a single 10Gbit usb-c port and dual drives.
There is one deeply troubling flaw to them though, they don't turn back on of the power goes out, until you physically hit the button again. I think this is alas all to common for many of these enclosures!
Years ago there were auction houses specialising in selling off business recycled PCs and bankrupt stock - it was great fun to go and mooch around and see if there was a real bargain not spotted, but they seemed to vanish under the onslaught of eBay and frankly for second hand tag I struggle to trust ebay
I’ve had good luck recently (sample size: 2) buying small form factor PCs off of ebay. Way more powerful than even a new raspberry pi, and a 4 core/16G RAM/256 ssd machine can be had for less than $60 if you are patient.
I would question all these Raspberry PI-ish NAS attempts, especially when it includes some power adapters and milling out cases. It all feels so fiddly and sluggish while still being "not that cheap". Storing my important personal data on an USB-drive somewhat feels risky. It probably wouldn't burn a house down, but still...
The real benefit is the small form factor and the "low" power consumption. Paying 43 bucks for the whole thing - now asking myself if it is worth saving a few bucks and living with 100Mbit network speed, instead of spending 150 bucks and having 2.5Gig.
There are so many (also "used") alternatives out there:
No I don't. The FriendlyElec NanoPI R6C can be brought to 1W Idle including NVMe SSD and 2.5GBe (and REAL transfer rates of >200MB/s). It's more expensive, but totally worth it in my opinion. See https://www.hardwareluxx.de/community/threads/ultra-low-powe...
Since it has neither ECC nor support for common Open Source NAS Operating Systems, I still would not buy it as my daily driver. I just don't think that a difference of 5W Idle Power is worth the effort of milling out stuff, using USB-Storage and the additional maintenance effort keeping this system up to date.
Everyone is an expert at storage as long as everything is working great. It's when stuff fails that you feel like an idiot and wished you had one extra hdd in your RAID array or a secondary NAS you were backing up to or one extra site you offloaded your data to.
I don't do cheap any more. But I can see the appeal.
These are all strategies and the price point of the unit doesn’t affect it.
Need extra drives, buy extra drives. Need extra NAS for backups?, buy an extra NAS. Need an offsite copy?, buy space and get an offsite NAS and drives for an offsite copy.
Price point of the unit doesn’t change anything here.
Synology sure provides an expensive but complete package for home office and enthusiasts.
Just buy it and be done with it. It's certainly more expensive than DIY it yourself using off the shelf components and things bought out of online classifieds. But for most people that have no interest in tinkering or don't know what to do, just paying the price of a complete solution might be worth it.
I'm not so sure anymore. My synology is now on its third backup application, so doing recovery means you have to hunt down the old client first.
The last one seems to be in Javascript. It's slower than the previous one, eats more RAM and has a few strange bugs. Backing up my smartphone photos is not always reliable.
I used to be happy about them, but since my update to 7, I don't feel I trust my backups anymore. Maybe I have to byte the bullet and do some rsync based scripting instead.
Totally. If you enjoy the config and have the time, by all means.
If you just want it to work, by a Synology. Mine has been running strong for several years now and has docker images for my unify controller, pi hole and Plex. Took minimal time to setup and none since that day. Love it
Edit:
And my encrypted cloud backup in Backblaze B2 was equally as easy to setup and costs a whopping $2 a month for every family pic, video and doc.
I have triple backup, with mirrored RAID for one of those. No effort, maximum peace of mind.
Redundancy and backup are not the same thing. RAID gets you redundancy but if you get pwnd, you get redundantly pwnd!
So you backup to elsewhere. Now you have two copies of your data. Cool. Now you probably can recover from a silly mistake from up to a week ago or whatever your retention period is. However, if you don't monitor your backups, you'll never notice certain snags such as ransomware. OK, that might be low risk for your home backups.
It's quite hard to get the balance right but I think that you might not be quite as protected as you think you are. Why not buy a cheap hard disk and clone all your data to it every three or six months and stash it somewhere?
I have a similar argument with a colleague of mine, to the point that I will probably buy a LTO multi head unit myself and some tapes.
RAID is not a backup, its a data integrity thing. It ensures that what you save now stays saved correctly into the future. It protects now. Backups protect the past.
Think long and hard about what might go wrong and take suitable steps. For you I think a simple, regular off line back up will work out quite well with minimal cost, for disaster recovery.
I didn’t actually specify it out, but my third backup is an offline SSD that I plug in every once in a while and store at my office. I only mentioned the RAID for local redundancy reasons.
You are right about the data being corrupted, either maliciously or bitrot. The NAS is not accessible outside my home network, so I think I am ok there.
Bitrot would require snapshots and full backups stored over time, which I could do fairly easily but I am currently not.
My Synology DS918+ ran for years without an issue.
Then suddenly I woke up and I had the blinking blue light of death, it starts but won't boot. It could be anything from the BIOS battery going dead to parts not being soldered properly.
The only way I could get data off their proprietary SHR system (which is great, except when the HW breaks) would've been to buy a new Synology system, put the drives there and pray it works out of the box.
At that point I decided to move out of a NAS appliance and build one myself, based on Unraid. Now I've got the same 1 drive redundancy BUT every file exists on exactly one drive, that drive has a bog-standard XFS filesystem. So if the HW breaks I can just take the drive(s) out and use them in a cheap USB dock.
The Synology's issue turned out to be a bad PSU - the light turned on but for some reason it didn't provide enough power for the system to POST. I grabbed a 3rd party one off Amazon and now it's running offsite backups for the Unraid system :)
> The only way I could get data off their proprietary SHR system (which is great, except when the HW breaks) would've been to buy a new Synology system, put the drives there and pray it works out of the box.
++1 for synology. I've been running their 5-bay model for a few years without any issues. It just works. I have it paired with a synology router which also just works. They both do their jobs transparently enough that I can basically forget they exist.
I don’t like that symbology provides ridiculously small amount of ram - I had one 7 years ago and it had 512 mb of ram, and was chugging. Looking at their website, they still sell a version with 512 mb. That’s a total joke in this day and age.
That being said, there are plenty of companies providing NAS drives at lower price points, with various levels of quality. Generally, they are not worse than a random AMR SBC with a closed source kernel, just like the author here assebled.
I appreciate not everyone wants a mini pc they have to manage, I had a nice setup but now it has died and I can’t find time to deal with it
I had a (dis)pleasure of running multiple Synology units in a business setting. They do die out on you like everything else if not more frequent, the QC is generally non-existent.
Synology's biggest reliability issue was when they used the Intel Atom C2000 CPUs. Designs with those CPUs have a 100% failure rate on the longer term (not just Synology, everything with it, Cisco was hit hard too). There's a workaround by soldering on some resistors to pull up some marginal line that will fix dead units.
Look, I'm sorry, but you literally asked for this:
Hello, I'm from the Department of Redundancy Department with a citation for a "Do It Yourself it yourself". Please pay the fine in internet chuckles, thank you.
I think this depends on what you're storing though.
Business documents, accounting records, family photos - sure you probably want to keep them safe.
But if my NAS is just 20TB of pirated movies and TV shows (not that I'm saying it is...) then I'm much more comfortable buying the cheapest drives I can find and seeing how it goes.
For me it's the opposite in a way: I need a proper remote backup of things like business documents and photos because they have to survive an issue with the NAS or my house not just a drive so local copies can go on "whatever" and something like cloud backup makes more sense to meet the reliability mark. Generally it's not tons and tons of terabytes which is great because the backup needs to actually be usable within a reasonable amount of time when I need to pull it.
On the other hand terabytes and terabytes of pirated content is a lot of work but not necessarily worth paying to try and to backup over the internet. I can redownload it if I need but I'd rather not do that because some crap drive or NAS I saved 20 bucks on died and now I need to spend a week rebuilding my entire collection. It doesn't need to be Fort Knox but I'll spring for a proper NAS, drives, and pool redundancy for this content.
Same, that was also my lesson that RAID is an availability mechanism, not a data safety/backup one. (Of course, Ransomware would also hammer that point home for many later on)
I sync a few single external drives every week or two over good old USB. In house sneaker net. Tools like Freefilesync make this easy and fast (and give me a manual check so accidental deletes are visible too).
Very cheap, has served me for more than a decade now. Highly recommended. I dealt with dataloss through drive failing, user error and unintentional software bugs/errors. No problem.
When people set up these NAS's, how are they accessing the files? NFS? SFTP?
And how are you accessing it when away from home? A VPN that you're permanently connected to? Is there a good way to do NAT hole-punching?
Syncthing kind of does what I want, in that it lets all my computers sync the same files no matter what network they're on, but it insists on always copying all the files ("syncing") whereas I just want them stored on the NAS but accessible everywhere.
Nextcloud kind of does what I want but when I tried it before it struck me as flaky and unreliable, and seemed to do a load of stuff I don't want or need.
- On mine I use NFS and SMB which covers most possible clients.
- I use an ssh bastion that I expose via Tailscale to connect to mine remotely. So a VPN but it's wireguard based so it's not too intrusive. I have a gig up, though, YMMV.
- My NAS has 28TB of space. I'm still working on backup strategy. So far it just has my Dropbox and some ephemera I don't care about losing on it.
- Regarding other services: I use Dropbox pretty extensively but these days 2TB just isn't very much. Plus it gets cranky because I have more than 500,000 files in it.
This is my personal setup but I think it's a bit different for everyone.
glinet routers are amazing, but their file sharing lacks authentication sadly (well, mine does, at least). So anyone getting access to you network also gets access to your files.
It's mostly fine, until one of you devices gets compromised. Then it's not fine at all anymore.
Wow! What kind of data are you generating that 2TB ‘just isn’t very much’? (Video editing?) All my personal files take up around 10GB in my Google Drive.
One example: If you take picture with a decent camera in raw format, your storage gets filled ridiculously fast. A short travel with a mere 200 pictures can easily be like 25M*200=5G. Another example: if you're doing any kind of AI training (especially picture based), the training materials can easily amount to many terrabytes.
Google takeout of my personal pictures from Google photos takes 600gb+ alone. And I'm not avid picture taker (that's the archive since 2000s, I did upload a lot of my old dslr photos to google photos when it was unlimited). I guess if people make more personal videos, they will use more space easily
Synology does all that.
I run two one at home one at the office, my only complaint is that it’s a bit “idiot proof”… both other times the web based GUIi is great. Also has free software that punches through NAT and dynamic IPs works great (quickconnect.to)
I use sftp, media server, primarily
I second that wholeheartedly, and I also run two 19" Synology NAS units, one at home and one at the office. All smooth sailing so far.
A colleague uses a QNAP instead, which he claims is better price/storage ratio at the expense of lesser software usability, and I'm okay paying a bit more of my own money (at home) as well as taxpayers' money (at work) on better usability, because it will likely pay off by saving time in the long run, as I currently don't have a dedicated sysadmin in my team.
The only question mark to date was when installing with non-Synology (enterprise SSD) drives I got a warning that mine were not "vendor sourced" devices, and decided not to take any risk and replace all drives with "original" Synology ones just because I can. This may be just disinformation from Synology to make their own customers nervous, and it reminds me of the "only HP toner in HP laser printers" discussion, but it would have been a distraction to investigate further, and my time is more valuable than simply replacing all drives.
Syncthing for a small collection of files I want available from all my machines - commonly used documents, photos, stuff I want quickly backed up or synced automatically.
Samba for my long term mostly-read rarely-write storage with larger files, ISOs, etc.
Same here. I have wireguard vpn for the few times i need it to tunnel my traffic through home or need to access larger files not sync’ed with syncthing.
My nas is a Synology. Vpn is also used so that i can continue sending timemachine backups back home when i’m traveling.
I usually just use SMB shares within my LAN. It serves my modest needs. I have used WebDAV or FTP in the past. Depends on the specific use. Away from home, VPN is essential. Too risky to just forward ports these days.
An easy solution for the VPN part would be Zerotier / Tailscale. IIRC Zerotier uses chacha20 for encryption which is faster than AES, especially for a power-strapped SBC.
For added security I limit my home ssh access to a handful of trusted IPs including my cloud VM. Then I set up an ssh tunnel from my hotel through the cloud VM to home. The cloud VM never sees my password / key
Its worth keeping this (from their readme) in mind though:
> However, at present SSHFS does not have any active, regular contributors, and there are a number of known issues (see the bugtracker).
Not that it is unusable or anything, it is still in widespread use, but I'd guess many assume it to be part of openssh and maintained with it, when it isn't.
An interesting alternative might be https://rclone.org/, which can speak SFTP and can mount all (of the many) protocols it speaks.
I used samba, it's supported everywhere. I also served files with HTTP server which might be convenient way for some use-cases. I also generated simple HTML-s with <video> which allowed me to easily view movies on my TV without all that nonsense.
My router has public IP so I didn't have any problems reaching it from the outside, so any VPN could work. Another approach is to rent some cheap VPS and use it as a bastion VPN server, connecting both home network and roadwarrior laptop.
No idea about any "integrated" solutions, I prefer simple solutions, so I just used ordinary RHEL with ordinary apache, etc.
Also I use SonicWall VPN to connect to my house to be in the network so it covers most of it. I also use Synology QuickConnect if I need to use the browser without VPN which also covers most urgent needs. Haven't failed me over a decade and my NAS also syncs with Synology C2 cloud which is also another peace of mind. I know it might sound unsafe a little having files stored on the cloud but it is what it is.
I won't play with half-baked library dependent homebrew solutions which cost way more time and cause headache more than commercial solutions. I won't open ports and forget them later either.
Seafile + Samba + OpenVPN is my stack. I use Seafile for a dropbox style file sync on my devices, and Samba for direct access. OpenVPN for remote access on all devices. Works quite well.
I’d replace OpenVPN with WireGuard at this point - WireGuard is a lot faster and the client software is pretty good. All of my Apple devices are set up to use VPN 100% of the time automatically if I’m not on home WiFi.
When you install WireGuard client, there's "On Demand" option there that you can enable. That option has two additional settings - it can turn WireGuard only for a particular list of SSIDs, or it can _not_ turn it on for a particular list of SSIDs. So you just add the SSID of your home WiFi to the list for which WireGuard will not be turned on. On macOS client there is an identical option. This works really well.
Tailscale works perfectly for remote access, I do "backups" with rsync over VPN nightly to an offsite location.
Syncthing over Tailscale is running smoothly too, it doesn't matter where my machines move, they find each other using the same internal address every time.
I have a (completely overkill) Ubiquiti Dream Wall that lets me VPN in using WireGuard. I do have a Raspberry Pi that runs (among other stuff) a script to ping a service on hosted server that keeps a dns entry updated in case my IP address changes, although that's rare.
I built the service to keep the dns entry updated myself, so I'm sure it's not as secure as it could be, but it only accepts pings via https and it only works if the body of the POST contains a guid that is mapped to the dns entry I want it to update.
IIRC they have improved the free plan over time, and even mailed users suggesting the relaxed limits might enable moving from paid to free tier [1].
I barely use my tailnet now, might have more of a case for it later, but they are near the top of my "wishing you success but please don't get acquired by a company that will ruin it" list.
Depending on the make and model - I've got a Synology NAS box and can't recommend them enough.
RAID support, NFS/SFTP/Samba support, a nice Web UI to set up access and configure sharing, and even the ability to enable sharing outside your own NAT.
Depends on what you need. I have a NAS with syncthing, and it's a combination.
- I use a lot of different folders within syncthing, and different machines have different combinations to save space where they aren't needed; the NAS has all of them.
- on the LAN, sshfs is a resilient-but-slower alternative to NFS. If I reboot my NAS, sshfs doesn't care & reconnects without complaint...last time I tried to use it, NFS locked up the entire client.
- zerotier + sshfs is workable-but-slow in remote scenarios
Note I'm mostly trying to write code remotely. If you're trying to watch videos....uh, good luck.
> The remedy is to turn UAS off via adding usb-storage.quirks=152d:0578:u to the kernel cmdline.
This is the point where I'd have thrown it in the trash and given up. I simply don't know how people have the patience to debug past stuff like this: I get that the point of the project is to be cheap and simple, but this is expensive in time and decidedly not simple.
Yeah, reading through the linked https://hub.libre.computer/t/source-code-git-repository-for-... really sours my opinion of Libre Computer - shipping with UEFI so you can just use generic images is a huge advantage, but creating your default images (and firmware! which is worse, IMO) with a proprietary process is such a big red flag that it makes me question the whole thing. If the firmware is FOSS and you can build it yourself using only FOSS inputs (which isn't obvious to me from that discussion), then you could do that and any old image (again, UEFI to support generic images is a huge win) and it would be fine, but the fact that that's not the default really makes me question the values/culture of the company.
Because they barely do anything. It's like like there's 4TB of RAM in there churning away at multiple databases. It's debatable if you even need it in enterprise servers.
You absolutely, positively, 100% need it on anything that carries data you care about. I personally consider it a hard requirement for a NAS. I don't want to lose data just because a cosmic ray flipped a bit somewhere.
There's a lot of places the file can get corrupted on its way to your drive. The memory of the NAS itself is only one of those. If you want any certainty you need to verify it after writing, so ECC RAM is not enough. And once you do set up that verification, you don't need the ECC RAM anymore.
I am not sure this is correct - there could be an error not only in the data, but also in instructions - but flip could cause data to be written to an incorrect location of the hard disk.
Possible, but I bet the risk of writing/losing the wrong sector based on that kind of RAM bit flip is a lot less than the other sources of writing/losing the wrong sector.
That differs based on the program you're using to put files onto the NAS.
But I'll note the even with ECC you need to double check things in case there was corruption on the drive wires or in many other places. With the right filesystem you can find some of those locally during a scrub, but double checking end-to-end isn't much harder.
Storing error correcting codes/checksums/signatures and sending them along the data seems like a more cost effective solution. Without those you need to ensure that every single link in the chain supports ECC (and all the hardware works perfectly)
ECC may still be needed for the actual processing, but I don't see a point on having it on a NAS (especially considering you need to send the data over a network to read or write it)
Absolutely. Imagine you are saving a text file to NAS with a super-secret password to your Bitcoin wallet, for example "password". While it was in memory before it reached disk, one bit was flipped and the file contents became "pastword" which OS happily saved on your RAID. And now you've lost your Bitcoins forever.
Consider all the RAM along a network transmission. Maybe you’re using authenticated encryption, maybe your transfer has an internal or out of band checksum. Maybe not.
That's pretty cool. Fits their use case for sure. I would probably opt to spend a little more for a gigabit port. From what I've seen watching Jeff Geerling, you can setup a pretty reasonable performing NAS on something on these small SBCs.
Any of the latest generation Arm SBCs is actually pretty adequate for NAS purposes, especially if that's all you want to run on it.
If you get a Pi 4 or Pi 5, or one of the Rockchip boards with RK3566 or RK3588 (the latter is much more pricey, but can get gigabit-plus speeds), you can either attach a USB hard drive or SSD, or with most of them now you could add on an M.2 drive or an adapter for SATA hard drives/SSDs, and even do RAID over 1 Gbps or sometimes 2.5 Gbps with no issue.
Some people choose to run OpenMediaVault (which is fine), though I have my NASes set up using Ansible + ZFS running on bare Debian, as it's simpler for me to manage that way: https://github.com/geerlingguy/arm-nas
I would go with Radxa or maybe Libre Computer if you're not going the Raspberry Pi route, they both have images for their latest boards that are decent, though I almost always have issues with HDMI output, so be prepared to set things up over SSH or serial console.
As other comments in this thread, I want to echo the value for money that is to be had in refurbished SFF office PCs that come available on the second-hand market.
I picked up an HP ultradesk something or other for dirt cheap a while back. When I got it it turned out to be surplus stock, so not even second hand - was brand new, for maybe 20% the retail price. Dead quiet, and super power efficient. It's not the most powerful CPU, but it's 10th or 11th generation which is perfect for hardware encoding for my media server use case.
It does not have all the hardware for RAID and multiple hard drives and all that, but one NVME boot disk, and one 16TB spinning rust disk is more than enough for my needs. It's media, so I'm not worried about losing any of it.
These boxes are cheap enough that you can get multiple ones responsible for multiple different things in a single "deployment". At one point I had a box for NAS, a box for media server, a box for my CCTV IP cameras and a box running homeassistant. All humming along nicely. Thankfully I was never masochistic enough to try some kubernetes thing orchestrating all the machines together.
This is all obviously for the homelab/personal use case. Would not recommend this for anything more serious. But these machines just work, and they are bog standard X86 PCs, which removes a lot of the hardware config and incompatibility bullshit associated with more niche platforms.
> To access SMART functionality, smartmontools must be able to send native ATA or NVMe commands directly to the drive. For USB devices, at least the following conditions must be met:
> The USB bridge provides an ATA or NVMe pass-through command.
> This command is supported by smartmontools.
> The operating system provides a SCSI pass-through I/O-control which works through its USB-layer.
> SCSI support is implemented in the operating system interface of smartmontools.
So in some cases, it may work depending on the USB chipset capabilities. I haven't gotten it to work on my old supermicro.. yet :) Cheers.
Personally I like the Dell/HP/Lenovo Micro PCs. For ~200€ you can get one with an i5-10500T, 16GB DDR4, and 256GB NVMe SSD and it can be upgraded to 64GB RAM with
lot of storage (1x NVMe + 1x 2,5")
So the price of used Sandy Bridge or newer laptop (optionally cracked screen) with 1Gbit ethernet, USB3, couple SATA, couple PCIE lanes (ExpressCard and mpcie slots) and build-in UPS.
But, not to waste space on this mindless joke, here's my (or, more precisely, my wife's) success story.
So, I've had it with a laptop and built myself a PC. Money wasn't really a problem, I just wanted to make sure I will have enough of everything, and spares, if necessary. So, I've got a be quiet case with eight caddies and a place for a SATA SSD. It's expensive... but it doubles as my main workstation, so I don't have any regrets about spending more on it! It has a ton of room for installing fans. It has like ten of them at this point, plus liquid cooling. The wifi modem that was built into the mobo that I bought doesn't have a good Linux driver... but the case has a ton of space, and so I could stick an external PCIe wifi modem. And I still have plenty of room left.
Anyways. My wife was given some space for her research in the institute she works for. And they get this space through some unknown company with vanishing IT support, where, in the end, all the company does is putting a fancy HTML front-end on Azure cloud services, sometimes mismanaging the underlying infrastructure. While the usage was uncomfortable but palatable, she continued using it. Then the bill came, and oh dear! And then she needed to use a piece of software that really, absolutely, unquestionably needs to be able to create symlinks. And the unknown company with vanishing IT has put together their research environment in such a way that NAS is connected via SMB, and... no symlinks.
So... I bought a bunch of 4T hard-drives, and now she has all the space she needs for her research. She doesn't even pay for electricity :(
Cheap, but not necessarily good, expandable, or resilient.
Although, I can tell you what not to do: a 45 drive SAS/2 or /3 4U JBOD case takes way too much power to use all the time and uses screaming 1U fans and PSUs by default.
I do have 45 drives in various XFS on md raid10 arrays. Don't even mention ZoL ZFS because that was a fucking disaster of undocumented, under-tested, and SoL "support" for something that should only be used on a proper Sun box like a Thumper. XFS and md are fairly bulletproof.
Perhaps one can Ceph their way into significant storage with a crap ton of tiny DIY boxes, but it's going to be a pain to deploy and manage, take lots of space, and probably damn expensive to physically install, network, and provide UPS for.
It's triple the price so I would hope it's lightyears ahead.
But if you're open to paying that much, well, I was considering that specific board along with some others, then I found an entire 12th gen Intel mini-PC for only 50% more and immediately changed my mind.
> 100Mb may seem like a joke nowadays, but the main purpose of such a toy NAS for me is to keep a copy of a directory with ~200K small files. Having 1Gb would only marginally improve the syncing speed even if the SBC supported USB 3.0.
He's wrong here. The most important thing with small files is latency, and a 1000M network will have significantly less latency than a 100M network.
Anyone running TimeMachine over network knows what I mean - local attached storage is blazing fast (particularly SSDs), wired network storage is noticeably worse performing, and wifi is dog f...ing slow.
What convenient solutions exist for graceful shutdown of a NAS, without data loss or other drama, in case of power outage? It seems a more pressing concern than flipped bits or network failures.
My Helios64 has a built in backup battery. I'm not sure they are available anymore (and I wouldn't recommend anyway for other reasons) but I imagine other NASs have similar setups.
My NAS is just my old gaming PC - I swapped out the GPU with a more basic one, and I add another hard drive or two every time storage gets low. It works great and costs me very little in both money and time.
I'm currently at 46TB or storage, and I recently threw in a 2.5Gbps NIC when I upgraded the rest of my home network.
(Mine certainly uses more electricity than the one in the article, but I pay $0.07/kwh, and run a few docker images that take advantage of the added performance, so I'm happy with it.)
That's basically my setup. I used to have a dedicated gaming PC and a desktop server but haven't used them in a few years. They both have something wrong with them so I'm just going to frankenstein them together into something pretty good. A 10 year old core i7, 16gb of ram, and an old Nvidia quadro is more than enough to run a bunch of apps and Plex transcoding on top of basic file serving.
I'm happy with XigmaNAS (BSD) on a used mini-pc and a multiple USB3.1 HDD enclosure. Speed is excellent as is stability. Having some memory and CPU cycles to spare, I also am playing with Home Assistant Supervised run as a Virtualbox VM inside of it.
Regarding that LaFrite board, I mailed a while ago LoverPi, which appears to be the only one selling it, to ask them if they accept PayPal, but got no reply. Does anyone know of a distributor in the EU or a different worldwide seller?
Personally I don't want to put too much stuff on my router/firewall. The reason I run OpenBSD on it is because I want it to be as secure as possible, it has the final word in my network and it's the outer most device of my network.
For storage I've been using Synology for a long time, first ds411+slim and now a ds620slim. I love the slim form factor, only 2.5" drives. It just works™
The entire concept of network attached storage is kind of cargo-cult in the vast majority of personal use cases. Just put the drives in your computer. Fewer abstraction layers, fewer problems, cheaper, faster, less fragile, easier to debug and fix if problems do happen. It's just not as hip or cool sounding as "NAS".
A NAS is not magical. It is just a rather limited computer. Anything a NAS can do so can a normal desktop, and it'll do it better. Sharing files over a network is one of these things. Managing the files is way, way, better since you just use your normal desktop file manager rather than some NAS web stuff that'll break due to CA TLS issues in a few years.
Desktop computers are becoming increasingly uncommon. I'm a pretty technologically inclined person and I havent owned a desktop in years. For most people, laptops have been more than capable enough for their needs for a long time. And for general purpose computing, smartphones are pretty heavily used too.
Yes, people do think that. And then they end up with an octopus of fragile external drives prone to problems and breakage. It's a sad state of affairs for the non-tech inclined. Like everyone is buying mopeds and pulling around trailers rather than just buying a small car.
Eh, I'm much more likely to accidentally reformat all my drives on my desktop than a NAS. Sure I can just restore from backup but it's just a security blanket to not have to worry about it
My “NAS” is my late model Intel iMac. I have, like, 5 USB drives hanging off it.
I SyncThing my wife’s laptop to it. I serve a bunch of videos off of it to our AppleTV. All our photos are there.
I have a Time Machine backup of the main system drive (my wife uses TM as well). The whole thing is backed up to BackBlaze, which works great, but I have not had to recover anything yet.
I would like to run ZFS mostly to detect bit rot, but each time I’ve tried it, it pretty much immediately crushed my machine to the point of unresponsiveness, so I’ve given up on that. That was doing trivial stuff, have ZFS manage a single volume. Just terrible.
So now it’s just an ad hoc collection of mismatched USB drives with cables not well organized. My next TODO is to replace the two spinning drives with SSD. Not so much for raw performance, but simply that when those drives spin up, it hangs up the computer. So I’d like SSD for “instant” startup.
Not enough to just jettison the drives, but if opportunity knocks, I’ll swap them out.
The thing I don't get is, why do I need to drop $1000 on a Synology machine just to do this? Let's say I want to create a setup with 8x18TB drives or something, do I really need to spend $1000 just to make this accessible to a couple clients at once (say a smart TV + another machine in the house)?
Right now I have Plex running on a raspberry pi hooked up to an 8tb external HDD. Works fine, but I want to scale up to the 100-200TB range of storage, and it feels like the market is pushing me towards spending an inordinate amount of money on this. Don't understand why it's so expensive.
This is my rationale for my specific circumstances:
With my NAS, I pay for the box, install the drives, and it Just Works with basically no maintenance other than me clicking the upgrade button every few months when it emails me that an OS update is ready.
I could build a similar system myself, but the hardware isn't that much cheaper. Cases, PSUs, hot swappable drive mounts, and all that add up quickly. And when I'm done, I have to install the OS, read the docs to see how the recommended configuration has changed since last time I did this, and periodically log in to look at it because it doesn't have nearly as much monitoring set up out-of-the-box.
Given the choice between the small amount of money I'd save vs the amount of time I'd have to invest in it, I'd rather pay the money and outsource the hassle to the NAS maker.
As to why I don't just hang a bunch of drives off the computer I'm already using:
- Backups. If my Mac dies, I can restore it from the Time Machine on my NAS.
- Noise. The NAS has busier fans than my Mac. It's in a different room from where I spend most of my time.
- I run Docker stuff on it. I don't want those things running 24/7 on my desktop.
- Availability. I want to reboot my desktop occasionally without worrying if it'd interrupt my wife watching something on Plex.
2 bay Synology NAS' are less than $200, 4 bay are less than $400. Yes, if you need 144TB of storage the NAS unit is going to be more expensive, but the drives themselves are the majority of the cost.
Or just put drives in all your computers and back-up between instead of buying a limited (potentially proprietary) NAS computer just for the purpose and having it being one central point of failure.
I just don't understand the reluctance of people to put storage in their actual computer(s).
Is there any NAS software that just lets you add disks whenever you want, while using them for redundancy if they aren't full? I wish something was as easy as adding another disk and having the redundancy go up, then removing a disk and having the redundancy go down.
If you just want a mirror, that's easy (I like ZFS but any software RAID should let you add/remove mirrored disks easily). If you mean mirror until it's full then automatically convert to striped, I don't think anyone does that and I don't think anyone would want that, because people who care enough about protecting their data to use a mirror don't want it to automatically stop being a mirror.
Interesting cheap low power system. My brain connects "NAS" with "Data I value" and there isn't anything in the author's system that is focused on enhancing the data integrity. Not saying that is bad, just saying it's the thing that often makes commercial NAS system both more expensive and more power consumptive. I've got an 8 drive RAID-6 setup as a ZFS pool[1]. More power, last I checked about 50W when idle, close to 150W when "working", but I think at this point I've had failures in nearly every piece that ended up replacing something but never lost any data, never corrupted any files. Replaced the power supply, the mother board, and two drives. I haven't had to replace any memory but I do have an extra stick "spare" because that is the kind of thing that ages out and is hard to replace (it's DDR3 memory but I've had systems with DDR2 so really hard to get DIMMS for).
That said, I do see a lot of value in low power systems like that of the author and run a couple. The way I do the energy calculation though is that I boot them off internal storage (MMC/SD) and then mount a root filesystem from the NAS. That way they don't have any storage power cost directly, they are easy to replace, and the power consumed by my NAS is amortized over a number of systems. giving it some less obvious economics.
I found an ex-office HP computer with and i5-4670 on the side of the road and have been thinking about setting it up as a home server. Does anyone have a recommendation for how to set it up as a NAS, VPN, Home Assistant and Plex server?
Home Assistant + plugins/extensions ~might be the easiest. Unsure about the VPN part. If you want to tinker and do some sysadmin work, then go install Proxmox and have separate containers for each as you wish.
Tailscale VPN addon for HomeAssistant is relatively simple to set up. You can use it to access your home network remotely ("site to site networking") or to proxy your network traffic through your home network when you're away ("exit node")
Yeah Tailscale is great. I assumed VPN as in tunneling the whole network external traffic through the machine. I haven't searched for something like that. Might even need a full blown router service/VM.
If you need to add storage to just one computer (at a time), consider just getting a hard drive enclosure. It's much simpler, cheaper, more secure, and faster than a NAS.
You can turn it into a NAS at any time by adding a mini pc or similar.
My "NAS" and homeserver is an old Lenovo ThinkCentre mini PC with a large SSD inside. My "RAID" is an rclone cloud backup.
Might be a bit scuffed but it works really well, at least for me.
I'd say the cheapest NAS would be an external HDD plugged in your WiFi router - most of them have at least one USB port nowadays, with some offering advanced features like photo gallery etc.
Although not quite as cheap, I bought a mini pc (Intel N-95, 8GB, 256GB) for not a whole lot more. It has room for a 4TB SSD in a built-in enclosure, which I mirror to an external 4TB HDD nightly. Important stuff is cloud-synced and manually backed up monthly to a external HDD that lives at work. It also runs Jellyfin, minimserver, syncthing etc.
One of the nice things is that it has a full sync of my cloud storage, so I don't have think about backing up individual devices much any more: I create a file on my laptop, it syncs to cloud storage, then to the minipc. From that point on it's part of the regular nightly/monthly backup routine.
If I hit the 4TB limit it might be a pain, as I'm not sure it'll support an 8TB SSD.
Backup solutions usually need to cover more scenarios than "PC explodes". In fact solutions for that are usually called "disaster recovery" instead.
A real backup solution ought to cover the case where you deleted the wrong file, and you find out the next day. Or it got corrupted somehow (PCs and disks can explode slowly) and you find out the next time you open it, a week later. If the cloud service happily replicated that change, it can't be used to restore anything.
> A real backup solution ought to cover the case where you deleted the wrong file, and you find out the next day. Or it got corrupted somehow (PCs and disks can explode slowly) and you find out the next time you open it, a week later. If the cloud service happily replicated that change, it can't be used to restore anything.
a) my cloud storage has file versioning
b) as I mentioned, I have a ~24hr old snapshot as well as a ~30d old snapshot
The worst practical case is that I lose a month's data, and I'm fine with that.
You're probably fine then! I was replying to the post above, which might not reflect your opinion. Mainly because I've seen it multiple times throughout this comments page...
You definitely can make a good backup solution that includes cloud storage, I didn't mean to imply otherwise.
Sure, there are different acceptable levels of restoration and backup but a 'backup' by definition is just a spare copy of data made in case the original is lost or damaged. I was only pointing out that a cloud copy is a backup, albeit a fairly weak one and as you say not useful by itself in every eventuality. It's just absolutist to deride cloud storage as "not a backup" as it can absolutely save data that would otherwise be lost.
The "personal files" I really want to keep safe amount to around 500MB of truly important data (pdfs, scanned documents, text files etc) and ~200GB of images and videos.
Both of which are 3-2-1 backed up.
Adding an ECC motherboard to my NAS would cost more than a quarter century of cloud storage for that amount of data.
The rest of my terabytes of data would be inconvenient to lose, but not fatal. In the worst case I'd need to track down a few rare DVDs and re-rip them.
Keep in mind that backups are a solution to a different problem than ECC.
If a file is silently corrupted you could be backing up the corrupt file for years and by the time you discover the problem, it has spread to every available backup.
Well part of it is you likely won't see the problem until a long time after it happens. But on servers with ECC and reporting I saw several different patterns:
a) 99%+ (or something) of the servers had zero reported errors for their lifetime. Memory usually works, no problems experienced.
b) Some of the servers reported one error, one time, and then all was well for the rest of their life. If this happens without ECC, and you're lucky, it's in some memory that doesn't really matter and it's no big deal. Or maybe it crashes your kernel because it's in the right spot (flip a bit in a pointer/return address and it's easy to crash). Or maybe it managed to flip a bit in a pending write and your file is stored wrong and the zfs checksum is calculated on the wrong data. If you're really unlucky, you could probably write bad metadata and have a real hard time mounting the filesystem later?
c) some servers reported the same address with a correctable error once a day; probably one bit stuck, any time data transits that address, that bit is still stuck, and that will likely cause trouble. If it's used for kernel memory, you'll probably end up crashing sooner or later.
d) some servers had a lot more ram errors; sometimes a slow ramp to a hundred a day, once or twice a rapid ramp to so many that the system spent most of its time handling machine check exceptions, but did manage to stay online but developed a queue it could never process. Once you're at these levels, you'll probably get crashes, but you might write some bad data first.
Ram testing helps on systems without ECC, but you won't really know if/when the ram has gone from working 100% to working almost 100%. I have a desktop that was running fine for a while, tests fine, but crashes in ways that have to be bad ram, and removing one stick seems to have fixed it.
Non shielded RAM is subject to bit flipping. Non-ECC always carries this risk in general computing, but the problem is compounded when you run a filesystem like ZFS which uses memory as a significant storage element for write cache.
If it would hugely impact your life if a bit were flipped from a 0 to 1 in your stored data - say you make videos or store your bitcoin wallet key on your NAS - you are running a risk not using ECC.
You may not have had issues or ever have issues with non-ECC. Your car may never be stolen if you leave the keys in either, but it's not a good risk proposition for most people.
As for my anecdote, I had a computer with 3 HDDs in a raid5, it had some of my very early programming projects and other various things which I wish I still had. But, I don't have any longer because something, I'm assuming memory, was silently failing and over 40% of the files were turned into jibberish and random binary bytes.
I now use ECC EVERYWHERE now. My laptop, my desktop, my little home server. All ECC. Because, ECC is cheap and provides a lot of protection for very little effort on my part.
If you just want to buy something that'll "just work", The Lenovo P16[1], is ECC capable from the factory. Basically anything AMD "should" support ECC, it may need to be turned on in the bios. The problem with "should" is the trail-and-error you'll have to do to find a working combination, though, I personally I've never had many issues getting ECC working.
Note that AMD APUs prior to the 5000 series only supported ECC on the "PRO" models. For example, the Ryzen 3 PRO 3200G supports ECC, but the Ryzen 3 3200G doesn't.
You might have been leaving fire pits to burn out for 20 years when you go to bed, but it only takes 1 errant spark for the "overkill" of wetting the ashes isn't overkill.
There are various trade offs you can make depending on your filesystem, OS tooling and hardware which can mitigate risks in different ways. However non-ECC invites a lot of risk. How often are you checksumming your backups to validate integrity? It seeks unlikely you've had 0 memory corruption over 20 years, more likely you didn't notice it or your run a filesystem with tooling that handles it.
You pretty much need a second, similar system, hopefully not physically nearby. Tape doesn't scale down to home use, and optical is too small.
My home NAS has several roles, so I don't have to backup the full capacity. The family shared drive definitely needs to be backed up. The windows desktop backups probably don't, although if I had a better plan for offsite backups, I would probably include desktop backups in that. TV recordings and ripped optical discs don't need to be backed up for me, I could re-rip them and they're typically commercially available if I had a total loss; not worth the expense to host a copy of that offsite, too; IMHO.
You might do something like mirrored disks on the NAS and single copy on the backup as a cost saver, but that comes with risks too.
Buy the hardware to make a lightweight backup server. Make backups work with it. Take it to your friend's place along with a bottle of scotch, plug it in, and then: Use it.
Disaster recovery is easy: Just drive over there.
Redundancy is easy: Build two, and leave them in different places.
None of this needs to happen on rented hardware in The Clown.
The low-tech and not super resilient method is to buy a second 28TB NAS, put it in a different location and sync them periodically when you know your primary is in good shape.
Back in the days of DVDs, I used to backup my 20GB drive onto DVDs. I wonder if you could do something similar today but instead of a bunch of 4GB optical disks, you would use 4 x 8TB drives?
I been using Interserver[1] + borg[2] for the last 3 years. With the 10TB plan comes out to $25/mo, but if you prepay a year there's discounts.
For the OPs use case, they have a 40TB plan for $84/mo. Still pricey, but, cheap compared to most other cloud storages. If you have data you care about, off-site backups are required.
AWS, GCP, and Azure all offer cold storage for about $1 per TB per month. If you want any cheaper you need to build a second NAS.
You could also take the awkward route and add one or two large drives to your desktop, mirror there, and back that up to backblaze (not B2).
The other suggestions you got for hot storage strike me as the wrong way to handle this, if you're considering $80 per year per TB for backups then just make another NAS.
For the OP - be careful with AWS, the closest pricing to one dollar per terabyte is S3 Glacier Deep Archive and you'd be surprised how expensive a full restore can be in the event that you need to do so in terms of restore pricing, egress cost, etc.
Another NAS isn't really a good solution (unless you can place it in a different house) - the goal of a cloud back up is that it's offsite.
I have a NAS that has 18 TB effective storage, 36 TB mirrored. It all gets backed up to a B2 back blaze which is about six dollars per terabyte - but I'm currently only using about 8 TB at the moment so it's only about 50 bucks a month.
So this might be on the higher end of the price range if you're using up all 28 TB uncompressed since that's about $168 per month though...
if your talking cloud backup Wasabi (which uses S3) is the cheapest i could find it’s pay as you go and they don’t charge for upload/download. The pay as you go is $6.99 per TB which would be pretty pricey at 28 TB, but it’s super cheap for my 4tb NAS.
been using zfs on my home nas without ecc for well over a decade and never had any problems. i've seen people claiming this since before i started using zfs and it seems so unnecessary for some random home project.
i've heard people say this, like i said, since before i started using zfs and i've never had an issue with a corrupted file. there's a few things that could be happening: i'm the luckiest person who has ever lived, these bit flip events don't happen nearly as often as people like to pretend they do, or when they do happen they aren't likely to be a big deal.
If all you have on your Nas is pirated movies, then yes
> when they do happen they aren't likely to be a big deal.
But with more sensitive data it might matter to you. Ram can go bad like hdds can, and without ecc you have no chance of telling. Zfs won't help you here if the bit flip happens in the page cache. The file will corrupt in ram and Zfs will happily calculate a checksum for that corrupted data and store that alongside the file.
I'd happened to archive the files to CD-R's incidentally. I was able to compare those archived files to the ones that remained on my file server. There were bit flips randomly in some of the files.
After that happened I started hashing all of my files and comparing hashes when I migrate files during server upgrades. Prior to using ZFS I also periodically verified file hashes with a cheapo Perl script.
ZFS does nothing to protect you against RAM corrupting your data before ZFS sees it. All you'll end up with is a valid checksum of the now bad data.
You can Google more, but, I'll just leave this from the first page of the openZFS manual:
Misinformation has been circulated that ZFS data integrity features are somehow worse than those of other filesystems when ECC RAM is not used. This is not the case: all software needs ECC RAM for reliable operation and ZFS is no different from any other filesystem in that regard.[1]
One snapshot would help because, if EVERYTHING collapses, and you need data recovery, the snapshot provides a basepoint for the recovery. This should allow better recovery of metadata. Not that this should EVER happen -- it is just a good idea. I use Jim Salter's syncoid/sanoid to make snapshots, age them out, and send data to another pool.
I agree that ECC is a damn good idea - I use it on my home server. But, my lappy (i5 thinkpad) doesn't have it.
Strictly speaking I don't think ZFS itself does, but it is very common for distros to ship a cronjob that runs `zpool scrub` on a schedule (often but not always default enabled).
If you're unlucky enough to experience memory errors in one of the intermediate buffers files go through while being copied from one computer to another an incorrect copy of the file might get written to disk.
When running software RAID, memory errors could also cause data to be replicated erroneously and raise an error the next time it's read. That said if the memory is flaky enough that these errors are common it's highly likely that the operating system will crash very frequently and the user will know something is seriously wrong.
If you want to make sure that files have been copied correctly you can flush all kernel buffers and run diff -r between the source and destination directory to make sure that everything is the same.
It's probably way more likely to experience data loss due to human error or external factors such as a power surge than bad ram. I personally thoroughly test the memory before a computer gets put into service and assume it's okay until something fails or it gets replaced. The only machine I've ever seen that would corrupt random data on a disk was heavily and carelessly overclocked (teenage me cared about getting moar fps in games, and not having a reliable workstation lol)
I wonder whether something like Syncthing would notice a hash difference with data corruption caused by such a memory error? And whether it’d correct it or propagate the issue…
Data that's about to be written to disk often resides in ram for some period of time - bit flips in non-ECC ram can silently corrupt the data before writing it out. ZFS doesn't prevent this though it might detect it with checksumming.
Hate to see this downvoted because I have personally lost files to ZFS on failing non-ECC memory. It was all my most-used stuff too because those were the files I was actually accessing, then it would compare checksum in memory to checksum on disk and decide disk was the one that was wrong. I noticed via audible blips appearing in my FLACs and verified with `flac -t <bad_flac>`.
cost and power are legitimate motives.
I'd probably have started with a rpi-02 though: storage and ethernet over USB is not going to win any races, though it'll compete with what he ended up with...
I mean wtf wouldn't you just buy a G1 Elite Slice, or any of the various NUC's you can buy for 50$ and get you a full Intel computer with a 6700 or 8700 cpu 4-8gb of ram and a full drive slot, and normally extra space for a m2 and a gbit nic lol
If you care about power consumption like I do, you can Google "$model energy consumption white paper" which contains very accurate data about idle usage, for example https://sp.ts.fujitsu.com/dmsp/Publications/public/wp-energy...
In one case I had a nuc where on Linux after enabling power saving features for the sata controller, idle usage even fell to 5W when the pdf claimed 9.
Having an actual pc instead of a random sbc ensures best connectivity, expandability, and software support forever. With almost all sbcs you're stuck with a random-ass kernel from when the damn thing was released, and you basically have to frankenstein together your own up-to-date distro with the old kernel because the vendor certainly doesn't care about updating the random armbian fork they created for that thing.