There are well established methods for time-to-failure and time-to-event data not used here. The author makes no effort to control for the multiple, obvious biases created by the analytical approach employed. A few simple graphs would give a much more telling view of these data.
Would you mind listing some of those time-to-failure and time-to-event methods and how the author might control for them, and which graphs in particular the author should have included?
first, i should've mentioned from the outset that these are really interesting and useful data, and that i'm glad you took the time to generate them. i really wish consumers could systematically report these data in a way that was reliable/trustworthy...!
cox proportional hazards models and KM survival curves are the big kahuna with a data set like this. basically, my impression is that you'd want to pretend that you're doing a cohort study and essentially analyze it like a big clinical trial.
and re: graphing, since you have sub-samples of the different drive models and relatively different but still small numbers of models for each brand, the big take-away graph comparing manufacturers leaves out a lot of useful nuances that could be pulled out of the table, e.g. that most of the hitachi drives are newer and you have fewer of them. it also doesn't portray how consistent failure rates are across models produced by the same brand might be, and it looks like there's a significant range of failure rates across different seagate models. so even just seeing IQRs of pooled failure rates for each manufacturer in box plots might be eye opening..
immortal time bias may be something to consider here as well...when you have some sample groups that mostly include newer individual drives that you have not yet had for a year on average, subtle differences in how the failure event is described can make a big difference in the conclusions you draw...especially in terms of uncertainty. if i have 10,000 hitachi drives and 5 of them fail in the first 6 months, the robustness of conclusions i can draw from those data are different in some important ways from similar insights drawn from a sample of 1,000 hitachi drives i've used for 5 years.
it's also not clear to me how you've dealt with replacement drives. based on what i gather from the post (and i could be totally wrong), if you have a bunch of one model of a drive failing and then get replacements for them... some might argue that refurbed drives should be analyzed almost like a separate drive model, since they often have physical differences compared to those bought via retail channels.
i'd be quite curious to dig into these data a bit further if you're willing to post the original data set...
thanks again for posting information that's quite useful and interesting
Yev from Backblaze here -> We've talked about posting the raw data, but haven't quite decided on that yet. I'll make sure to forward this to Brian so he can take a look and see whether or not any of the above would be feasible for the next one!
