It amazes me that something like this can be done by a single person.
In aviation any time input is given to the machine, it's entered by one human (typically pilot flying) and then verified by the other human (typically pilot monitoring) before being committed to or executed. For example... when a new altitude is assigned by ATC, say FL300, the pilot flying will spin it in the selector window and keep his hand or finger there until the second pilot agrees with and confirms the selection by reading FL300 out of the selector window.
I know there are meat bags in these giant tubes so that changes attitudes towards safety etc. However, it seems to me that when organizations start putting the power to halt nearly the entire business in the hands of one person, there should be some slightly different attitudes. A breaking change in a million servers could easily cost hundreds of thousands or maybe even millions in lost revenue or employee productivity.
I'm just an outsider though. Perhaps this level of attention is practiced at some shops. It's just interesting to me how in some fields we settle on pretty uniform standard practices whereas others are seen as non-human-life threatening so it's just shoot first, ask questions later.
Best practice for using the "weaponized" version of the tool when you had powers to actually hit all of them at once was to paste the command into IRC and get some of your fellow peeps to eyeball it and make sure it was sane.
<me> team: hey, sanity check this please: hsh -A "dumb_thing && other_thing --foo --bar"
<teammate> shipit
[ I type the command ]
<me> ok, running as job 1234
The last part was a courtesy done so that they could watch the progress of it too without having to dig to find my request. It also meant they could kill it easily if something went wrong and they couldn't raise me for some reason.
Tools like this are best used outside the solo realm.
I think an automated tool would be preferable since there is no 100% foolproof guarantee that what you type in irc is the same as what you type in the terminal.
> It amazes me that something like this can be done by a single person.
In many dysfunctional orgs, having someone to blame is desirable. They will use all kinds of words for it like "accountability".
But at the end of the day, heros who take stupid risks that succeed get rewarded, cautious people that ask questions and try to understand before acting are smugly dismissed, and would-be heroes that burn the house down because of recklessness get blamed and make everyone else look good. It's all too common.
In shops where stakes are high, it’s not uncommon to do just like you said—have mechanisms that force someone else to verify what you’re about to do, before you do it. If someone else can’t verify, the tool will block you. It’s similar in spirit to requiring code reviews on all shipped code.
In aviation any time input is given to the machine, it's entered by one human (typically pilot flying) and then verified by the other human (typically pilot monitoring) before being committed to or executed. For example... when a new altitude is assigned by ATC, say FL300, the pilot flying will spin it in the selector window and keep his hand or finger there until the second pilot agrees with and confirms the selection by reading FL300 out of the selector window.
I know there are meat bags in these giant tubes so that changes attitudes towards safety etc. However, it seems to me that when organizations start putting the power to halt nearly the entire business in the hands of one person, there should be some slightly different attitudes. A breaking change in a million servers could easily cost hundreds of thousands or maybe even millions in lost revenue or employee productivity.
I'm just an outsider though. Perhaps this level of attention is practiced at some shops. It's just interesting to me how in some fields we settle on pretty uniform standard practices whereas others are seen as non-human-life threatening so it's just shoot first, ask questions later.