The complexity is also in having to deal with limited resources, and factors outside of your control. Have you seen many bootstrapped HFT implementations? The budgets are just... different.
When you spend 10x or 100x hardware, get massively redundant backbone connectivity (connecting to a tier 1 core router directly, and not through an oversubscribed shared switch port that a typical bootstrapped startup would get), you end up with a much better controlled and somewhat predictable environment.
With a typical "web" (non-HFT) distributed system, you've got millions of "subscribers" aka users - and I don't think people outside of IT fully realize just how bad a typical user's [Windows] machine is messed up - and that was before they installed a couple of toolbars, adware, firewall, and AV products... And all of these subs are tying up your publishers in their own unique screwed up way, keeping sessions open, dropping packets, blocking ports...
I've designed global CDNs, and operated one for a decade - and the kind of randomness our distributed systems have to deal with are not always technical, even though we don't typically count microseconds.
The complexity is also in having to deal with limited resources, and factors outside of your control. Have you seen many bootstrapped HFT implementations? The budgets are just... different.
When you spend 10x or 100x hardware, get massively redundant backbone connectivity (connecting to a tier 1 core router directly, and not through an oversubscribed shared switch port that a typical bootstrapped startup would get), you end up with a much better controlled and somewhat predictable environment.
With a typical "web" (non-HFT) distributed system, you've got millions of "subscribers" aka users - and I don't think people outside of IT fully realize just how bad a typical user's [Windows] machine is messed up - and that was before they installed a couple of toolbars, adware, firewall, and AV products... And all of these subs are tying up your publishers in their own unique screwed up way, keeping sessions open, dropping packets, blocking ports...
I've designed global CDNs, and operated one for a decade - and the kind of randomness our distributed systems have to deal with are not always technical, even though we don't typically count microseconds.