SMR drives have more of a place in the enterprise market than the consumer market, just like tape drives. Enterprise/datacenter customers tend to have a much better understanding of their workloads and are more able to determine when they have a use case for niche technologies. They also have a greater ability to deploy tiered storage with SSDs in front of hard drives to cover up the performance limitations.
Adding to this confusing debacle is how HDDs are now being marketed like running shoes: one type for running, one for squash, one for tennis, one for basketball...
There are only maybe... 3 or 4 elements of HDD designs worth talking about.
* CMR/PMR drives -- Classic hard drive you've been buying for the last 30 years. Not much to say about it.
* SMR Drives -- "Shingled" has very poor write performance, but adds 20% more capacity at lower costs. For "write once, read many" workloads, like archival or backups. This is because "Shingled" drives write data on top of the old data, so that your data is physically overlapping at the microscopic level. Any write must read the old data, rewrite the old data, and then finally write the new data you're planning to store.
* 5400 RPM vs 7200 RPM -- Higher RPM drives are obsolete, because SSDs are faster. 7200 RPM is kind of standard, but 5400 exists for those who want less noise, less electricity usage, and are willing to put up with the lower performance.
That's pretty much it. "NAS" drives tend to have anti-vibration sensors and other such things, but those sensors don't really cost much and I don't really see why it deserves its own category.
-------------
There only needs to be maybe, 3 kinds of hard drives made today. 7200 PMR drives, 5400 PMR drives, and finally 5400 SMR drives.
I just wanted to add that SMR drives' performance is a bit more nuanced than you described. In most SMR drives, there is a PMR portion allocated as a cache. I've read the cache is usually in the range of 20-50G, depending on the size of the drive, but I'd love to know how accurate that is. So write performance is generally pretty good until you saturate that cache. Then, as you said, it becomes pretty atrocious.
How do "surveillance" drives fit into that? They're supposed to be special for being reliable with a lot of writing, several orders or magnitude more the drive's capacity. Apparently normal hard drives aren't reliable enough for recording CCTV footage.
All hard drives have effectively infinite rewrite capability, unlike SSDs. I know I've seen surveillance drives before (WD Purple), but I never figured out what made them special compared to others.
Looking at the WD Purple drive marketing material... they describe "support for 64 cameras". Which means the CCTV community is willing to pay for write bandwidth. It'd probably be a standard 7200 PMR drive, but with assurances and maybe some tests on the write performance.
I've heard WD allude to their surveillance drives having some kind of firmware optimizations to handle multiple streams of writes with good QoS. It seems plausible they they would have a bit of a different write caching strategy based on minimizing seeks and trusting that the data coming into the drive is interleaved from multiple sequential write streams even though at any given moment the command queue will look like a bunch of random writes.
The key aspect of WORM here is that the data you write is not modified in-place or overwritten in part; it stays put until deleted. That's true of most backup strategies, whether or not they generate a lot of read traffic to compute incremental backups. The "read many" part of WORM is usually never about how many actual reads are performed, but about the capability to handle them better than multiple writes.
You are right but the guy you replied highlight a strong point, we want to write once very quickly just to archive something and chance of reading those is low. But we would need to write it. (He did it every night. I just archive it and recycle the storage in my NAS, but not as frequent as every night.). If I want archive reading in the very long term, should use tape or as you say WORM. But not for hard disk usage.
Low cost write quickly. That is the 5400 coming in but not this.
But the key is to tell us so we can choose. Not as scandal like to get such knowledge and still hiding.
10 000 RPM SAS drives absolutely still exists and are relevant. I got my last batch 2 months ago or so.
Those SAS drives have nice capacity (2,4TB) and you don't need to worry about DWPD.
What kind of workload do you have where there are enough writes that a 1+ DWPD SSD isn't viable but the performance of a 10k RPM hard drive is adequate?
It would have to be a highly sequential workload for hard drive performance to not be a problem, but that means it's also a workload that's well-suited for QLC SSDs that are cheaper per GB up front and draw less than half the power and are faster in every way. If your worries about write endurance are at all realistic, you must be averaging upwards of 50MB/s of writes over the entire lifespan of the drive, meaning your hard drives are spending at least 20% of their service life actively writing data.
Can you confidently confirm or cite a source that 7200 SMR doesn't exist? It doesn't sound like I can avoid buying from a sleazy company, but I at least want to avoid this SMR nightmare for a new 8-bay NAS build I was planning right up until all this broke.
IIRC, PMR(perpendicular magnetic recording) technology is also used on SMR drives, so It's not correct term to differentiate SMR. And the technology have been used around 15 years.
CMR(conventional magnetic recording) is retronym to differentiate SMR.
supposedly, part of the difference between the NAS drives and desktop drives (back when they first started segmenting consumer desktop and consumer NAS drives) is that the NAS drives have a much higher inactivity timeout before spinning down. It's supposed to help with longevity since spinning up/down wears out the drive faster and with performance coming back from an idle. I have no idea how much that actually matters or whether that's still true anymore.
I don't know if there's any truth to this, but I've read NAS drives also declare themselves "bad" much more readily through SMART, so you replace them sooner.
NAS drives will more readily return errors for individual IO commands. That's not the same thing as the drive saying it has failed generally, though primitive RAID controllers often treat any IO failure as a whole-drive failure.
It probably is a setting. And manufacturers simply make us pay $20-50 more for flipping a bit. I'd be stunned if they have entirely different firmware versions for different products.
The annoying part about all this is that I want to buy some SMR disks (HM ideally, but HA will do) to experiment with storage tech. I know that there are lots of devices out there that are SMR internally, but because of vendors hiding the information, I can't actually buy what I want.
(On that note: if anybody has a disk that supports ZAC or ZBC and wants to get rid of it, I'll happily buy it off of them)
The parents point was that, by hiding disk type, they are intentionally device managed, not host aware (HA) nor host managed (HM), so do not fit the bill.
If customers cared enough, wouldn’t someone just market their drives as PMR and this would solve itself? It would make sense to have a lower cost alternative that would fulfil most consumer needs though.
I can't speak for the drives mentioned in this article, but the WD Red drives were de facto marketed as PMR since the Red line was marketed by WD as the best drives for NAS. It'd be a little like if you went down to the your local Ferrari dealership and bought a Ferrari only to find that they'd started selling a model that couldn't reach 60mph.
Do you have a reference? Or any evidence of customers knowing or caring about the internals of the drive before this controversy arose?
Claiming that anything is the "best" at something is pure puffery, as we say in the legal trade. Ferrari publishes quantifiable specs on how their vehicles perform; WD does not do so in detail.
The worst thing that WD done is adopting SMR on NAS brand without announce. SMR for parity RAID is completely disaster.
Adopting SMR on Desktop brand is acceptable compared to NAS brand, but it should be announced.
I just got fucked by this change in models - I had an array full of 6TB WD Reds, had one fail, ordered a replacement. Rebuilding the RAID-Z3 took over 2 weeks. Previous replacements had taken under 2 days - usually under 1.
Passing SMR drives off without saying anything, especially when that exact model had been a PMR drive prior, is incredibly anti-customer.
What's going on, is that WD has for years, sold a PMR 3TB drive called "3TB WD Red".
Suddenly, without warning, the "3TB WD Red" is an SMR drive. Technically they changed the model number, but the marketing number (3 TB Red) remained the same.
Do you believe it is reasonable for customers to see this and expect that a drive called "WD 6TB Red" when you purchased previously and when you go to purchase it now, that are described identically, with no way to purchase the previous model, and know that the fundamentals of the storage medium have changed and completely flip it's performance characteristics?
Especially when the drive is specifically marketed as being RAID and NAS friendly - one of the primary differences between the Red and Green drives is disabling some of the sleep power saving mechanisms that are known to cause issues with RAID arrays. And they make it so that the drive is largely unusable in any parity based array? When minor revisions, updated firmware, etc., will result in minor changes to model numbers all the time?
I think the real answer here is to demand that disk manufacturers publish performance specifications. This leaves them free to change implementation details (which consumers usually don't know or care about) as long as the published specifications are accurate.
Then again, drive manufacturers are also known for changing what the definition of a unit of measurement means. Some of us remember when they arbitrarily changed the definition of a gigabyte nearly 20 years ago from 1<<30 bytes to 10^9 bytes - a nearly 7% difference! (See: https://www.zdnet.com/article/attention-hard-drive-manufactu...)
I don't really care if a drive is 150MB/s or 200MB/s. I understand that hard drive manufacturers change their technology on a regular basis (PMR, HAMR, etc. etc.). Adding platters, removing platters, increasing platter density, etc. etc. As hard drives change, so do their performance.
What makes this a sore spot, is that SMR drives have something like 1/8th the sustained write performance of PMR drives.
Think about how you went from HDD world into the SSD world a few years back. Except do that in reverse. SMR drives are great for archives, but they should never be sold as a mainstream solution, because their performance is so much worse than PMR.
> I don't really care if a drive is 150MB/s or 200MB/s.
You do care about performance - you say so in the next paragraph. In this case, it is rewrite performance. This is quantifiable and can be disclosed on spec sheets and labeling. I'm not saying the recording method can't be disclosed too, but I think to most consumers, the metrics are more important than the technology.
SMR is like, 30MB/s write speed best-case scenario. PMR is 200MB/s write speed, easy.
We're talking about 700% difference in performance. In the case of RAID5, you have many, many physical writes per OS-level write. If you have 5-hard drives storing 100MBs, you'll need to actually write 120MBs to disk.
Not possible. Modern drives require precision external equipment to write the servo markers, you can't "low level format" a modern drive and haven't been able to for about two decades. Changing from SMR to PMR would require rewriting the servo tracks for the different density.
It's competition. Every cost cutting tech gets adoption except for boost the main muggle specs like capacity.
On the SSD side we have DRAMless and TLC.
The annoying thing is that unless the government forces disclosure, the honest manufacturers are the ones hurt, having to start advertising "no SMR!" to differentiate, like the organic/health food products in the grocery store that have to list all the poisons they don't have.
I'm happy to buy a DRAMless + TLC SSD for my boot disk on my web browsing PC (not development), it is a reasonable compromise for a lower price. Same for SMR drives, I won't have problems with them as backup disks.
I think the problem here is SMR drives are sold without disclosure, and it has already created some serious problems in practice.
> What’s worse, they’re shipping DM-SMR drives as “RAID” and “NAS” drives This is causing MAJOR problems – such as the latest iteration of WD REDs (WDx0EFAX replacing WDx0EFRX) being unable to be used for rebuilding RAID[56] or ZFS RAIDZ sets: They rebuiild for a while (1-2 hours), then throw errors and get kicked out of the set.
> They rebuiild (sic) for a while (1-2 hours), then throw errors and get kicked out of the set.
I don't doubt it, but do you have the source for that? I read stories about people using the Seagate Archive disks in RAIDZ and it worked fine for them.
Mine completed the rebuild, but it took over 2 weeks vs. the usual 1-2 days my prior WD Red 6TBs had completed in.
This has accelerated my move to enterprise 16TB drives, because I really don't want to expose myself to data loss on rebuilds taking hugely extended times.
> Mine completed the rebuild, but it took over 2 weeks vs. the usual 1-2 days
This also means you'll need a lot more HD's per RAID set to guard against multiple failures during rebuild. Which means that even the supposed data density/space advantage of SMR is moot - you'd be better off with PMR.
This, plus the previous post about the WD Red drives, makes it all three manufacturers - there are no honest options left. It's like saying "honest telecoms".
Also, "boost the main muggle"? I usually think I know my Potter references, but this one escapes me.
Yes, the NAND flash "decline" (for lack of better words) is sad to see, and it's as you said --- the only marketing spec people seem to care about with storage is capacity.
TLC and QLC are misnomers, a TLC cell holds 3 bits but has eight levels and QLC likewise 4 bits with 16 levels. You get a multiplicative increase in capacity for an exponential decrease in reliability. The manufacturers have been very obfuscatory with the marketing, since they know the inconvenient truth.
Does anyone still make a reasonably large (>100GB) SSD with good old 100K SLC? I have a 64MB (true 2^26 bytes capacity!) flash drive with 100K SLC; 0 bad blocks when it was new, and still 0 bad blocks a decade and a half later despite plenty of abuse.
This feels like it could be managed in firmware. It's been well established that many modern SSDs will use a large volume of the NAND in a SLC style mode for performance reasons, and only switch to MLC/TLC/QLC mode as the available space to do so runs down. I would be curious what effect that has on reliability-- if an individual QLC flash cell goes bad, is it still typically usable if you're using psuedo-SLC coding?
To go a bit further, I'm amazed you don't see a drive manufacturer offering an enterprise-client "firmware builder" tool, or at least as a build-to-order service if you're a big enough client. I could see customers that wanted to respec their 1TB drives as, say, 750GB to ensure an abundance of over-capacity for reallocation later, or the YOLO "I'm gonna run the cheapest drives in RAID0" crowd slashing overprovisioning and getting 1050Gb out of the same unit. If you dialed down the capacity low enough, the drive could presumably stay in "use SLC mode only" mode for its entire duration.
The main issue I have with increasingly complex firmware is that it's also a source of bugs; they might not show up in normal usage, but any unusual edge-cases that result in data loss are catastrophic.
I'm amazed you don't see a drive manufacturer offering an enterprise-client "firmware builder" tool, or at least as a build-to-order service if you're a big enough client.
QLC with an SLC cache is the SSD equivalent of SMR. They are fine for read-mostly workloads but performance goes completely down the drain when you do random writes.
I've tried QLC drives for cheap cluster storage. It turns out one QLC drive was slower than 4 old HDDs. I had to go back to HDDs until I switched to TLC flash because they were faster. Tail latencies hitting one second on QLC. It's incredible how bad they are. We shouldn't even be calling them SSDs at that point, they don't even remotely live up to the performance expectations you might have from an SSD.
>They are fine for read-mostly workloads but performance goes completely down the drain when you do random writes.
This doesn't come out in tests or consumer experience of the drives that have a significant cache, like the aforementioned "Rocket Q", which has a DRAM cache and a large SLC/TLC cache behind it.
Cluster storage is not a mainstream consumer use.
> We shouldn't even be calling them SSDs at that point
Is the intel 660p range not an SSD? It significantly outperforms SATA SSDs.
> It turns out one QLC drive was slower than 4 old HDDs.
Was that a consumer QLC drive with SLC cache or an enterprise QLC drive with no cache and firmware that actually cares about latency? Something like the Micron 5210 ION enterprise QLC drive should still have at least 20x the sustained random write performance of a single hard drive.
I do not know whether it's actually possible but if you partitioned an 1TB Intel 660p to only 140GB and left the rest open would it make that into a 140GB SLC SSD? If so, then the 1 USD/GB for an SLC drive is an absolute bargain...
> You need 6 times more QLC chips to get an equivalent write performance to TLC on average.
Depending on how you define average, yes. But a sufficiently bursty workload is always writing to SLC. How the drive performs in practice depends on where you sit on that spectrum.
Speed-wise no, but it (almost) keeps up with the lower end of TLC NVMe drives.
> QLC + huge cache is not good deal cost wise in comparison to pure TLC.
Cost wise? Yes it seems to, Sabrent's "Rocket Q" is about the cheapest 2TB on the market, and performs pretty well, Reads more or less competitively with the non-QLC drives from the same vendor and writes at around 2/3 the speed.
Intel's 660p range performs a little less well, but typically still multiple times as fast as SATA drives, and is generally cheaper too.
Is there any way small-scale ~4 drive NAS or file server systems can accommodate SMR drives in software? It looks like they're going to need to do so...
It would really be best if the drive properly identified itself, so the host os/filesystem could adapt to the drive parameters.
That said, you would want to use a log structured filesystem, with a compaction feature. You want to avoid read-modify-write on the shingled areas, it's time intesive and competes with other I/O.
You don't need a single Log structure -- the shingle stripes are on the order of 128 MB of or so. Any CoW filesystem can relatively easily adapt to this kind of system; in-place filesystems are basically boned on SMR. The CoW filesystem aren't going to be fast on SMR, mind you, just not quite as awful.
Toast0's great-grandparent comment (and my grandparent comment) are talking about Host-managed SMR, not "transparent" drive-managed SMR like these fraudulent WD Reds.
Yes, nothing except extremely low load or extremely linear (i.e., security video / tape-style backup) workflows works well with drive-managed SMR.
storage drive recycle after certain period and hence it is good for the first write but after that as those (and then as which support it) would need to assume low load environment. May be ok, just if we can tell we can test.
"Can relatively easily adapt" is not the same as "has already been adapted". It still performs awfully without the work put in to make the hardware and software cooperate, but doing that work is not nearly as big of a burden.
I believe these SMR drives don't allow direct access to "write a block here". They try to write your writes to a staging area first, and them move them later.
The SATA/SCSI spec, together with operating systems, block device interfaces, and filesystems, would all need to be updated to have more parameters to correctly handle SMR drives properly. Since hard drives are going out of fashion and architectural changes like that will take 5+ years to be widely deployed, it's never going to happen.
Host-managed SMR came first. When it wasn't being adopted because it sucks, drive-managed SMR like the crap WD is selling as Reds came about.
SCSI and SATA and operating systems already have support for host-managed SMR — the name used in this context is "ZONE" and those changes happened around 5+ years ago.
> They try to write your writes to a staging area first, and them move them later.
If a zone is empty, and you try to write sequentially to it, I would sure hope the firmware isn't so hopelessly dumb as to redirect that data to a staging area.
This does require that they don't disable TRIM support, I guess.
I'd really like to see some detailed analysis of how these drives perform.
You'd need a filesystem that was pretty careful to write a very large chunk extremely sequeuntially. That's easiest with Host-aware SMR (to have some idea the size of the zones) or a linear LogFS.
But the point I'm making is that "host-aware" should be good enough for this type of use. Even though it's still drive-managed, the internal logic in the drive should pass through certain write patterns without interference. And you can make the host aware through feeding in a few parameters; it doesn't necessarily have to interrogate the drive.
In other words, I'm disagreeing with the idea that this big laundry list of things would need to be updated. The filesystem should be sufficient if these drives act in a way that's at all reasonable.
Ah, maybe. It’s not a big step from aware to managed, though. The zoned api is actually quite simple: what are the zones?, open zone, finish zone. The filesystem that’s tolerant of these characteristics is the hard part.
> Is there any way small-scale ~4 drive NAS or file server systems can accommodate SMR drives in software? It looks like they're going to need to do so...
The thing is you can't really buy SMR disks that are OS controlled unless you are some big enterprise client.
SMR disks discussed here are DM-SMR disks, which basically are same disks, but with a firmware based hack that makes them appear as regular SATA disks.
A small-scale NAS implies home usage, which again implies plenty of idle time. With plenty of idle time, the SMR drive should perform decent enough, as the drive has enough time to shuffle stuff between the persistent cache region and the SMR region.
They have problems being chucked out of pools when resilvering, ie. when added to the pool, so the general usage pattern isn't so much the issue as is the way the filesystem or RAID controller reacts to them. In particular I think people have had problems with ZFS because the resilvering process involves a lot of random writes, and once the PMR cache on the drive is full, it will stop responding as it flushes the cache for long enough for ZFS to think it's dead.
So we could make the build process pause until the drive resumed responding, or have the ZFS/whatever filesystem drivers recognize the difference between a dead drive and an SMR drive that's working on its cache?
I'd want the drive statistics or benchmarks to include a complete rebuild process, though.
Yeah, it does seem like maybe there could be improvements on the filesystem/RAID software side to better support these drives. If only they were labelled properly as SMR so that the filesystem devs could buy one to test with.
I'm not sure about how to moderate the reported DMSMR behaviors reported in this thread.
But I wonder if folks are better off buying host managed or host aware drives instead, and pairing it with either dm-zoned and their preferred file system; or a file system that supports ZBC and ZAC natively (Btrfs does, I'm not sure about others).
Note that dm-zoned has significant changes starting with kernel 5.5.
Are the disk vendors publishing performance metrics that are being violated by the use of SMR technology? Or is this just one of those cases of performance generally not being as good as it used to be, and customers becoming reliant on past behavior despite the lack of an explicit performance promise from the vendors?
> Are the disk vendors publishing performance metrics that are being violated by the use of SMR technology?
Not really, no. SMR drives overlays data on top of each other, like "shingles" on a typical American roof.
As such, whenever an SMR drive writes data, it must read the data "underneath" the current spot, rewrite the "underneath" data, and then lay the new data on top of it.
SMR drives are great for archives (write once, read many). But they're pretty much useless to the typical consumer. SMR drives should be avoided if you're building your own computer.
I think the real answer here is to demand that disk manufacturers publish performance specifications. This leaves them free to change implementation details (which consumers usually don't know or care about) as long as the published specifications are accurate.
For literally years, people bought WD Red drives, DESPITE being more expensive, because they had higher performance.
WD Blue drives had SMR / Shingles in them. WD Green drives were shoddy, low quality, and had issues with RAID Rebuilds. WD Red commanded a high price, but came with higher quality.
What's happened here, is that WD has begun to sell low-performance drives under their high-priced "Red" brand.
I think it's a question of write vs. rewrite. Most archival backups are write-once in the sense that they're append-only, especially now that storage is so cheap nowadays (see: Amazon Glacier).
SMR performs generally fine except in the physical rewrite case. A 6TB disk is unlikely to see a lot of physical rewrites except in this pathological case of a RAID rebuild that people are complaining about.
How do you backup your motionless piles of sand, then? /joke
Perhaps all the data you care is on the cloud, but HDDs are still important for backups. Although it's the most acceptable scenario to use an inferior SMR disk. The real issue of SMR is RAID/servers, which, of course, is what the cloud is made of...
If the cloud ever used RAID, the switch to erasure coding happened a long time ago. With Zoned Storage for SSD and HA-SMR for HDD we are Further seeing adoption of these novel technologies in the cloud.
I do my backups in the cloud, which uses hard drives for slow storage but I couldn't care less about it. The terrible reliability and performances are not my problem anymore.
What if you lose your job and can't afford to keep paying the monthly fees, or there is corruption in your backup? You won't know until you perform a disaster recovery.
If I lose my job I get notified in advance and then I get unemployment benefits for years. It's more than enough time to find a new job before I don't have enough savings to justify cloud storage costs.
What if you connect your hard-drive and the data is corrupted or the drive isn't spinning? That did happen to me. I rather have people replacing the hard drives for me. Uploading corrupted backups to the cloud is the same than saving corrupted backups locally.
Their customers are increasingly likely to be the kind of people who care about this kind of thing.