The 8Tb WD SMR drives I have use a 40gb chunk of the surface for non-SMR reads and writes as a buffer. Altering anything in an SMR track requires copying the data from that track and tracks near it into the buffer, then laying them back down into the SMR region sequentially. Like erasing a flash block before writing. Unfortunately, this means that the drive's performance takes a dump from 200mb/s down to 300kb/s if you do enough transfers to fill the 40gb buffer, or attempt multiple transfers at once.
I stopped buying WD after they slapped the Red label on their SMR drives. I’m sure there are good uses for them, but being a member of a RAID ain’t one.
I run a bunch of Seagate 8TB SMR bought a few months after release, and the ZFS raidz2 pool has grown over the years.
They are wonderful for write once/read many storage of large files. Due the the sheer number of drives the write speed isn’t a major concern as it can saturate the 20gbps Ethernet bond to the machine.
Reads are the multi-GB/sec range though, so long as they are sequential.
They would not be my first choice, but for specific uses like backups and media storage they are a good value.
For low usage like on a NAS ,they are fine for me because they usually last me about 5 years. Maybe more if I don’t replace them for larger versions. They’re about $50 cheaper than other better performing models. They’ve also outlasted every seagate drive I’ve had in the same size and price category. Of course I would never use them for anything other than a NAS.
For low speed writes -- e.g. trying to download things over the internet say and then save them off, but not for trying to backup multi terabyte data to it.
If they increased the size to 4TB say, to make it easier to import data to it, it might be worth it. But if it takes several days to write 32GB of data to it, it's not really gonna be worth it to most people.
I've wondered why WD/Seagate don't invest in a consumer solution where drives are literally just plug and play together and give you raid without the hassle of a raid controller or software. Eg. An 8TB drive connected to an 8TB drive is a mirrored raid. SMR drives would serve as "cold storage". Once written to, would never be rewritten again.
That'd be a leap from USB/SATA/sas native chips to a full blown nix os with a microcontroller and secondary bus on it to handle drive to drive.
if you want that I think they do have in box solutions with two or more drives and a while back sold their own NAS drive system. Not sure what they have today but there are plenty of free or free enough solutions out there these days.
FYI all modern hard drives have multi-core ARM or Risc-V microprocessors built-in. Usually one core manages the motor control and signal decode from the RW head and another core runs an embedded OS and manages the data interface to the host computer. Folks have managed to get malware running on the hard drive's embedded CPU(s)[0] as well as full blown Linux distributions [1]. So your average hard drive definitely already has enough grunt to get the job done. Just needs a firmware written to expose the functionality.
I think what's important is that most SMR drives can only write a few MB per second anyway once their buffer gets saturated. That's well within wifi/100M ethernet speeds.
I feel like 10gb superspeed hardware is more than a fast enough connection. The hardware is relatively cheap to build, it's just the software.
I'm not saying you need a super high speed processor or memory either, since the SMR drive itself can serve can serve as (slow) memory for the CPU. The trick is trying to figure out mostly how to replicate blocks across the raid, and then mapping the files to raid blocks.
They still have valid use cases like backups or unedited video footage. It's just kinda lame that the manufacturers don't market them as "slow backup devices" clearly listing all limitations, and that you have to find it out first time you use it.
When I have used them for backups, the 40gb buffer quickly fills and then I'm stuck with speeds slower than my internet connection until the buffer empties, which it will only do if I kill the existing transfer. SSDs can dump 40gb of data very quickly. Annoyingly, since the r/w head has to do it's back-and-forth dance between the buffer area and the SMR areas, this condition even impacts read speeds. Consequently I wouldn't even use them for backups. Nor would I buy them again. I've set them up as Storj drives, which they seem to handle reasonably, and I am thankful to be done with them otherwise.
If you're considering one, I would pay special attention to the buffer size, and ensure that all the transfers you want to do to or from the drive are significantly smaller than the buffer to ensure reasonable performance. That excludes most video too. Storj files are typically just a few megabytes, and typically arrive at a frequency of just one or two per second, which the drive can handle.
I had that problem as well (8TB Seagate). It would write some data, then get completely stuck to the point where Windows would report an I/O error. So I wrote a small tool that writes data in smaller chunks, monitors the write speed and allows throttling it if needed.
Weirdly enough, just using the tool instead of copying files with Explorer somehow stopped the weird hanging from happening, even without having to enable the actual throttling. Probably some bug somewhere along the driver/firmware stack triggered by the write block sizes.
Overall, I wish the drive vendors would expose some API to directly manage the SMR/CMR areas via software, just like the FLASH memory chips do. That would make the job of appending new backups + overwriting the old ones actually manageable with predictable and consistent timing.
It also seems like a potential opportunity for hybrid flash hard drives, where the flash could take over the role of the CMR buffer region and reduce the amount of back-and-forth required of the drive's rw head, which should considerably increase performance.
It's probably sensitive to exactly how the file is opened. SMR drives need the file to be append only. If Explorer isn't communicating to the OS that it's going to do that properly for whatever reason, the driver would kick the writes back to the random access area which would slow it down.
SMR drives aren't designed for having data shuttled between areas like that. They're meant to be used such that you write in long streams directly to the shingled areas. The slowdowns are clearly due to the abstraction mismatch getting in the way.
> Overall, I wish the drive vendors would expose some API to directly manage the SMR/CMR areas via software, just like the FLASH memory chips do.
They do, on drive models that are sold to the customers large enough to have the resources to re-write their storage stack to handle zoned block devices. The drives sold at retail will continue to pretend to be ordinary random-access block devices for the sake of backwards compatibility.
I was about to comment that this might not be a huge issue in network storage. Throwing a pair of 1TB SSDs as write cache in a Synology is pretty painless. Then I remembered that SMR drives don't like RAID, soo yeah.
Maybe you can make it work in clustered file systems like Ceph if you make sure you have a big enough SSD-based write cache.
Given that 40GB is like $6 of ssd, it wouldn't be hard to provide an actual useful SSD buffer.
I got a Lenovo laptop for the parents a long time ago that had a hybrid hard drive; I think maybe 8 or 16GB builtin SSD. Man that thing was just absolute trash, so slow; ghastly long boot times or time to launch a browser.
Ideally a SMR drive has a SMR capable file-system, where it can write data into linear logs & them try to batch rewrite the metadata. Even the pathologically terrible random writes can be somewhat coped with via accommodating fs.
The SSDs that go that fast also have multiple dies per channel. And write speed is the relevant metric here, but SLC caching isn't useful unless you have flash capacity to spare.
I wonder why drives still use mechanical actuators to fling a single read write head back and forth across the surface, rather than a bar fixed in place that spans entire radius of the writable surface, with an addressable strip of r/w heads that runs the length of it.
I assume it's because the actual read-write head itself must among the most expensive parts to manufacture, but is there no photo mask approach to this that can manufacture such a head array (or staggered series of them if there is some interference problem that prevents putting them side by side in a single strip) the same way but processor dies are made?
Because a typical drive has MILLIONS of tracks, and an RW head is quite wide, compared to its very narrow active area. It would not work.
Let's try some math:
A typical drive has 1.3 Tb/in^2 density. Assuming bits are approximately square (have a 1:1 aspect ratio), and the drive is locally linear, we can calculate that (ignoring ECC and metadata, which make the task even harder) that each square inch is 1.2e6 x 1.2e6 bits, meaning that you'd need a strip with a density of 1.2e6 read/write areas per inch. Assuming an outer platter diameter of 3 inches and inner diameter of 1 inch, you'd thus need this strip to be one inch long. So you need to engineer 1.2e6 RW heads instead of one and somehow fit 1.2e6 of them within one inch. I cannot even imagine how you'd do this, as each RW head needs to be a coil, and some way to get the data out (and possibly even a local amp)
That article provides the physical characteristics of a bunch of drives across several decades. The newest there is a 5TB drive that has tracks 85nm wide and bits 17nm long, so each bit is a rectangle with a 5:1 aspect ratio. The oldest is a 44.7MB drive with tracks 40um wide and bits 2.6um long, for an even more extreme ~15:1 ratio, but all the drives follow this pattern: the bits are much wider than they are long. The ratio for floppy disks is even more extreme; its calculation is left as an exercise for the reader.
Of course this doesn't change your argument that millions of heads would be necessary, at sub-micron spacing, to say nothing of how alignment could be maintained with continual thermal expansion and contraction.
What if the bar were able to travel linerarly via a voice coil, and had hundreds of heads, but the bar was able to shift in a linear fashion only about a micron back and forth in increments of thousandths of a micron.
Wouldn't that get you to your millions of heads of effective track coverage, without all the complexity of having to move two or three different hinges on several different actuators sweeping the entire arc? Naively speaking it seems like you could mount such a bar even closer to the surface of the track since there is no chance of a head crash, as it could be supported at both ends of its mounting.
Surely the entire armature that moves an articulated set of actuator assemblies plus the head itself would be heavier and more cumbersome and failure-proneto move than a linearly-fixed rail.
Obviously someone has thought of this before me so there's a good reason for it, but track density alone can't be it.
What problem does your proposal potentially solve?
A strip of heads that move in unison won't be able to operate in parallel, for the same reason that on real hard drives heads serving different platters cannot operate in parallel without being mounted on independent actuators. So it seems like your suggestion only reduces the distance that any given head needs to move, at the cost of greatly increasing the mass of heads+wiring that needs to be positioned quickly and accurately. You've seriously inflated cost for no gain in bandwidth and likely no gain in latency (and even if everything worked out in favor, it couldn't do anything to improve rotational latency).
I don't know I keep getting answers to different proposals than the one that I have made, all with a significant detail changed so that the answer makes sense and the proposal sounds absurd.
In this case, I don't know how the impression could be gained that I was suggesting mounting 100 or so individual assemblies of the very same read write head currently used on mechanical drives, each with its own separate wiring harness.
Obviously, yes that would substantially increase the cost and the mass and everything else, you're right. Thanks for setting me straight about the impracticality of very different solution than the one I described, which suggested the heads could be manufactured as a single integrated unit, the way every other matrix of active elements (OLED displays, memory cells, someone else mentioned mems, etc.) is made, which might require two additional wires for selection signalling, if that, depending on what could be multiplexed. In exchange for this weight and complexity you get rid of one, possibly two extra servo joints, one possibly two extra armatures, plus all the wiring and additional control logic required for positioning this very delicate articulated assembly.
Now, if this array cannot be manufactured because no current solid state process exists that can reproduce the characteristics of a wound coil, duplicated a hundred times in a linear array of cells on silicon or some other substrate, that's an obstacle I can reason about and accept as a plausible dead end.
If you could make a row of mems stewart platforms like a linear CCD for the read heads, somewhat like the TI Micro Mirror displays then you could make a fast parallel readout head. You aren't going to get to 1.2M, but you could get past 1000 (per platter). 1000x current drive readout speeds is 200GB/s. I have no idea if modern drives strip data across platters, so the final bandwidth of the device is left to an expert.
Modern drives only access one platter at a time, except for the handful of dual-actuator drives that split the stack of platters in two and can simultaneously access one platter from the top half of the stack and one platter from the bottom half of the stack.
The limiting factor is the fact that positioning the head on platter one to access track N does not guarantee that the heads on the other 8 platters are positioned over track N on their respective platters.
The limiting factor is the fact that positioning the head on platter one to access track N does not guarantee that the heads on the other 8 platters are positioned over track N on their respective platters.
Nor indeed does each platter even have the same number of tracks at the same spacing in modern drives. Ever since embedded servo became the norm for hard drives (late 80s), there hasn't been a need to align each platter with the others.
What would be the advantage of having different sector and track layouts for different platters on the same spindle, with the same servo addressing all of them? Are platters binned the same way that processors are?
Not a question of advantages, a question of tolerances. At the track size used now, thermal expansion and process tolerances are enough that even with a position specified to the micron from the center, you'll have different track numbers between platters solely by variance. This is why every track has MANY markers on it identifying its track number, and their values are used as feedback for the servo mechanism that moves the head. When seeking to track X, it will pick up markers while it moves and after it stops moving, use them as feedback, and continue to move in the proper direction. Nobody knows how many microns from center track 1234567 is, nor cares. It is more or less a PID controller, with desired value being desired track and current value being current track seen under the read head.
I would never have guessed how much thermal expansion had to be compensated for in modern hard drives, I would have assumed they were built such that no degree of precision that required accomodating changes in physical parameters was attempted, and/or that materials science had found an ideal set of alloys for the case, the platters, the magnets, arms etc, that all behave in a predictable way under a given thermal map, so that addressing the surface was nearly always a deterministic process.
Have there been attempts to use active cooling/heating (i.e. thermal sensors combined with piezoelectric effect elements embedded in the housing) to maintain a precise, consistent reference internal operating temperature, as a means of eliminating the need to accomodate thermal-induced drift in tolerances?
why? making a feedback-driven servo is a well understood task. it is not hard. and even if you knew the precise micrometer position to move the head to, magically compensated for ALL variables, you'd still need a PID controller to MOVE you there, and feedback to help you measure where you are. You'll have added a significant material science problem, but solved nothing
Sorry if I'm coming across as presumptuous or something, it's just surprising to find that modern hard drives are still using the same basic mechanism that I saw inside the washing machine-size drives my dad worked on 40 something years ago, that would almost walk themselves out of their mounts thrashing their giant head servos around.
What you and others are describing is strikes me the way that opening up the package of a modern memory chip and finding a fabric of millions of tiny magnetic cores would, rather than a silicon array of capacitor/transistor cells - that for some reason no fundamental rethinking of the problem has prevailed and thus the only developments in RAM for almost half a century is miniaturization of what was being done in the 1960's.
The first PC HDDs used stepper motors and absolute positioning with cylinder geometry, just like floppy drives. Then they moved to voice coils and remote encoders (e.g. early Quantum drives), dedicated servo surface (drives with this report an odd number of heads), and finally embedded servo with non-cylinder-based layouts where each surface can have different track pitches and sectors per track.
I wouldn't call that "the same basic mechanism" as there have been lots of refinement over the years, but it turns out that having the heads themselves find what they need to read was the best solution.
AFAIK all optical discs also use a similar scheme.
Incidentally this is why a HDD with a bad head or media area will make clicking sounds, sometimes very loudly, as the head actuator slams against the stops since the controller can't see the signals it's looking for and can only sweep the surface in search of them.
Thank you, that makes sense. The feedback control system that positions a head can only guarantee that the head for a specific platter is in the right location.
I suspect the head flying just above the surface has something to do with it.
A bar would need to be held at a fixed height above the surface. The flying head dynamically adjusts to both distortions across the surface, and temporal variations as the platter changes temperature. This height adjustment is entirely passive, dependant on aerodynamics rather than any electronics.
I remember someone explaining this to me years ago when IOmega released those removable Bernoulli drive cartridges. The (for the time) high densities were possible because even though the recording medium was a flexible bit of plastic, the head was designed to draw surface of the media up to itself using the namesake effect, rather than relying entirely om precision machine tolerances. And this effect was dynamically self limiting, such that there could not be a head crash as long is the unit wasnt disturbed during operation.
What I did not know is that this is still and effect that is relied upon in modern, ultra high precision, fixed medium drives.
> bar fixed in place that spans entire radius of the writable surface, with an addressable strip of r/w heads that runs the length of it.
That was called a fixed-head disk. They got rid of seek latency (still had rotational latency) and so there was a market niche for them as (e.g.) swap devices, but they had lower data density than moving-head disks did, so they went out of style. It looks like they are so obscure today that there's no wikipedia article, but here's an article from some other wiki:
They are briefly mentioned in the general Wikipedia article on hard disk drives. There was also "drum memory" where there was a head per track on a cylindrical recording surface. Those seem to have dried up even earlier: https://en.wikipedia.org/wiki/Drum_memory
Here's an info page about the DEC RS03 and RS04 from the 1970s. The RS04 weighed 120 pounds and had 1MB of capacity.
There's probably an insane data throughput domain that requires autocorrelation filters that might benefit from an optic fibre bundle in which each fibre is slightly shorter than the previous one.
The article itself gives you a good reason. Differential thermal properties which can be compensated for in the servo system but not in your fixed amerature.
The article does mention that this is a complication particular to SMR drives, but I was thinking about this not necessarily as a means of reaching higher density storage, but more reliable and perhaps higher performance solutions that would fall somewhere between the compromises of an SSD and a mechanical drive.
It also seems possible that instead of a single super high density strip of heads which may be impossible to manufacture, that instead a series of much lower density array strips could be mounted in n ranks, each rank offset from the predecessor by n/(distance between heads in each strip).
I assume this is not an insurmountable engineering, but that someone has already had this idea or something similar to it, and after running the numbers found it doesn't deliver enough of an advantage over the conventional design to be worth exploring further.
> I assume it's because the actual read-write head itself must among the most expensive parts to manufacture,
Ehh a modern HDD is a marvel of precision manufacturing and engineering.
> rather than a bar fixed in place that spans entire radius of the writable surface, with an addressable strip of r/w heads that runs the length of it.
Your bigger challenge is getting a set of read/write heads that are across the platter able to maintain a consistent height/etc.
The closest I'm aware of, are the Connor 'Chinook' drives [0] that had two actuators, one on each side of the drive.
Broadly speaking, I'd ask if multiple heads too close throw off decades of head design experience...
I might understand that businesses have use cases where price difference adds up, and they have workloads, where SMRs can satisfy the needs.
However, what's the reason that they are selling SMRs to the consumers? Price difference in stores is not that big between SMR and CMRs, and there are a lot of downsides.
