Well, a 10k core cluster is a far cry from a 500TB array. The latter fits comfortably into 2-3 racks (if your DC allows the power density). The former sounds like 15+ racks - entirely different ballgame.
I'm absolutely not denying that you should have a dedicated admin to look after such a deployment. But by today's standards it's just not a lot of hardware anymore. And definitely not the size where you must spend a fortune on "platinum" support contracts that can easily cost as much (per year) as a fully fleshed out spare shelf...
FWIW, I'm personally running a few moderately sized storage clusters (the largest is ~200T, 3x3 MD1000) and we're seeing disk failures at a rate of 2-3 per year.
I'm absolutely not denying that you should have a dedicated admin to look after such a deployment. But by today's standards it's just not a lot of hardware anymore. And definitely not the size where you must spend a fortune on "platinum" support contracts that can easily cost as much (per year) as a fully fleshed out spare shelf...
FWIW, I'm personally running a few moderately sized storage clusters (the largest is ~200T, 3x3 MD1000) and we're seeing disk failures at a rate of 2-3 per year.