>"As we move to leverage cloud computing, this is getting more challenging. First of all, I love cloud computing and think it is an essential step forward for our industry. Still, the way we create robust solutions is under pressure as the individual components experience emerging challenges called
GRAY FAILURES.
In a gray failure, a server or part of the network doesn’t fail fast but instead starts running slow. Running slow is WAY worse than failing-fast. The slow component, sometimes running at less than 1% of its normal speed, may be healthy enough to say “I’m still here!” but slow enough to clog up all the work. This makes fail-fast schemes vulnerable."
[...]
>"In electronics, there is a phenomenon called
METASTABILITY.
It means the system may be in an unstable state for an unbounded time. To have a valid signal as an output to the circuit, it must reside within a certain voltage or current level. If the output lands in the middle gray area, wonky things happen to the next circuit. It, too, can do weird things."
[...]
>"Consider a team a team of jugglers handling dozens of balls on stage. If some of them are sent into slow motion in a non-correlated way, it can be a problem. Each juggler has his own timeframe and sees balls arrive when they arrive. For a short while this may make sense. It becomes virtually impossible to juggle time-based interactions across the team as each juggler’s time speeds up and slows down unbeknownst to them."
GRAY FAILURES.
In a gray failure, a server or part of the network doesn’t fail fast but instead starts running slow. Running slow is WAY worse than failing-fast. The slow component, sometimes running at less than 1% of its normal speed, may be healthy enough to say “I’m still here!” but slow enough to clog up all the work. This makes fail-fast schemes vulnerable."
[...]
>"In electronics, there is a phenomenon called
METASTABILITY.
It means the system may be in an unstable state for an unbounded time. To have a valid signal as an output to the circuit, it must reside within a certain voltage or current level. If the output lands in the middle gray area, wonky things happen to the next circuit. It, too, can do weird things."
[...]
>"Consider a team a team of jugglers handling dozens of balls on stage. If some of them are sent into slow motion in a non-correlated way, it can be a problem. Each juggler has his own timeframe and sees balls arrive when they arrive. For a short while this may make sense. It becomes virtually impossible to juggle time-based interactions across the team as each juggler’s time speeds up and slows down unbeknownst to them."