Most of what the author is complaining about boils down to business needs being more important than the needs of the engineering team. To be blunt, they're paying you to do a job, not to make the organization better. That's what they pay the leadership for. You want to be part of the leadership, work your way through the ranks or start your own business.
My life got much, much easier once I learned to stop straining so hard to fix things that are bigger than me. If you don't like your managers, or the culture, or the business, find another place to work, it's that simple. If you can't do that, you're simply going to have to learn how to compromise.
This is exactly a perfect example of WTF worthy behavior being treated as totally normal; that it's encoded that there's "leadership" and "not leadership" as binary job distinctions with distinct responsibilities.
I've been in both types of cultures and, by far the healthier culture was the one in which "making the organization better" was treated as everyone's responsibility. I've seen very junior engineers drive quite significant culture changes, simply by leading by example. They did it, not by whining or complaining, but by taking baby steps of trying out small experiments with their immediate team and then broadcasting their success to incrementally greater circles within the company until it became the new normal.
If you seriously believe that all (or most) companies disallow anyone not in leadership to improve things, I'd consider getting out of your current situation and seeing things again with clear eyes.
> If you seriously believe that all (or most) companies disallow anyone not in leadership to improve things, I'd consider getting out of your current situation and seeing things again with clear eyes.
I've been around a bunch, including the US Air Force, and I can confidently tell you that heavily hierarchical organizational culture is the norm, individuals contributors who want to have effects beyond their job scope do so at risk of offending other stakeholders. They may tolerate you and even give some limited help, but they're not going to shout your name from the rooftops just because you want to be a boy scout.
Not saying you can't buck the system and get away with it, one of my favorite books was about one of my heroes, John Boyd, but if you want to be a hero, you need to go into it with a clear-eyed assessment of what you're up against and so you can tailor your objectives appropriately.
The DoD (!) is hardly representative of the cultures being discussed in the linked article, which is about tech startups (with a bunch of examples from medicine too). Yes, there are worse examples, but those may be beyond help.
As an enterprise grows, you either retain centralized control and have hierarchy, or distribute control by treating parts of the org like independent, federated entities (think Berkshire Hathaway).
The military's size, scope, and mission are unique. You need top-down control, because otherwise most folks won't decide to run up a hill into a machine gun on their own. That mission focus (do stuff with force), cascades into the supporting services.
Tech start ups are a rather small part of the overall IT industry. I dont really have the numbers to back it up, so i am not going to speculate as far as the actual numbers, but any discussiona bout workplace IT that ignores everything that isnt a tech start up is virtually useless.
And the IT industry is a rather small part of the overall job market. So therefore any discussion about the workplace that ignores everything that isn't IT is virtually useless?
I fail to see how governmental organizations should fall outside the scope of the discussion.
And the Air Force is not the DoD. While, technically, the individual branches of the US military fall under the DoD, they all have very different organizational cultures, and the DoD has staff and culture of its own.
> I fail to see how governmental organizations should fall outside the scope of the discussion.
I won't tell you what falls within the scope of this discussion or not, but I can tell you why they don't add a lot of value to it either.
It's already quite hard to somewhat objectively discuss whether a business company is dysfunctional, in part, or not at all, and what are the reasons for that.
It's almost entirely useless to attempt to have that discussion about government. Because politics. Because people root for their home team. Because people are inclined to dismiss the other team's ideas even when they're real good. If I told you the US Government is dysfunctional, someone would counter that I'm not American and I should see my own government. If we move past that and we got into the details of why things are not working properly, invariably Americans will start quoting bits of the constitution, bill of rights and foundling fathers at each other (and from there on it just becomes religion to me).
Discussing whether the function of Military organisation is working properly is even more difficult. In addition to the above problems, there's also hazing and indoctrination, which are incredibly strong psychological forces (without those, like said, people won't run into their deaths without question). And it pretty much divides the crowd in two slices, those "outside" that have no idea how it really works, and those that have been "inside" who are unable to disconnect the indoctrination bits from forming an objective judgement about how well the organisation functions and how that comes to be.
Certainly, discussing businesses has similar problems, but this just amps up the personal emotion factor to eleven. Also one of the reasons why the businesses were kept anonymous in the featured article.
Not sure why you are getting so many replies disagreeing with you. Narrowly-defined roles are indeed the norm, even in the tech world. When I was an "individual contributor" I'd get frustrated with Worst Practices all the time, but was told that I was hired to write code not to change our infrastructure or suggest different product features or improve our testing practices--MANAGEMENT makes those decisions. But even when you get into management, there are always managers above you to limit what you can achieve.
There may be awesome companies out there where anyone is empowered to just go fix some practice that's not productive, no matter where they are in the org chart, but those companies are few and far between. I'd love to see one.
There may be awesome companies out there where anyone is empowered to just go fix some practice that's not productive, no matter where they are in the org chart, but those companies are few and far between. I'd love to see one.
One thing I would consider doing, if I ever start my own company or rise high enough up in someone else's company to implement this, is to give every single employee some sort of discretionary budget to spend on things that make the workplace better. The more experienced and trusted you are, the bigger of a budget you get. I think this would go a long way in fighting the learned helplessness dynamic.
This is a very interesting idea. Have you, or anyone else on HN, ever seen this in action? I would LOVE to hear stories about this, either successes or failures.
It's called 20% time and it works OK. I always used it when I was at Google, although most people didn't.
20% is not specifically for "improving the organisation" of course. It's most famous on the outside for the new products it led to, like Gmail. But actually most 20% projects were small internal things intended to smooth the rough edges off a particular tool or process, or to improve the company in some way. If the particular bee in your bonnet was a type of bug that cropped up frequently in other people's software, making a linter to spot it and driving adoption through the organisation would be a good 20% project, for instance.
Part of it is vinceguidry makes something of a false distinction. Either fight the system and solve X problem behavior or just accept it and don't think about. A third option is simply investigating classes of behavior of this sort and learning something from them.
That you chose to mention the UAF as an example suggests that your experiences are really on the far side of what is being suggested here. If there's any place where these kinds of things don't work, it's in government or the military.
I recall reading somewhere that while military organizations are very much synonymous with hierarchy, they have historically experienced a lot of pressure to flatten and decentralize, at least as far as field command goes. Napoleon's success was attributed to smaller units that responded to feedback more quickly. Unfortunately I can't find that article.
However, this is not to say that in regular administration there isn't still a lot of de facto trust in the hierarchy. I think unbounded hierarchy is generally one of those memes that people grow up taking for granted as something that just works with any layer count. In some ways, it's a self-reinforcing thing - if you climbed ranks for 10 years, you don't want to see the system change and your "sunk cost" go away.
The US Marines are often cited in management literature as an example of an organization where the leadership communicates high-level intent and purpose to achieve strategic alignment, and then totally decentralizes tactical decisionmaking to the people doing the actual work in the field. The stereotype that they are just all about top-down decisionmaking and following orders, is false. They recognize that individuals need to be able to make decisions on the fly in order to react to volatility, uncertainty, complexity and ambiguity.
But it will always remain an inherently misleading comparison, because they get to train people to an extent and in ways that would be unthinkable for a business in modern civilized society to get away with. The training and drilling is such an engrained component in that whole system. Without having that first it seems to me a bit like prematurely spot-optimizing the wrong loop.
That book made it into four different reading lists from various branches of the military (USMC, USAF, Army, and a chair of the House Armed Svcs Committee). That's encouraging in that at least they're aware that his ideas/approaches were productive.
Yep. There's a whole philosophy based on the idea that improvement of the organization is everyone's job. It's called Lean, or more simply, continuous improvement or Kaizen.
I hesitated to reply to this because what I'm about to say could be taken as pedantic. However, it is my experience that many people experience a good way of working, apply a label to it and then use that label thinking that it will communicate all of the good things that they intended. This is rarely the case. At some point you see many articles entitled "Why X doesn't work" where they describe a broken version of X and why it doesn't work. The broken version ends up getting more popular than the unbroken version for some reason and there is a general consensus that "X doesn't work and anyone that uses X is an idiot", even though X works perfectly fine if you use it correctly.
With that in mind, "lean" as a principle is organized around the idea that it is better have people request functionality (pull) than for you to imagine what people want and then try to sell it (push). Many "lean" organizations happen to be very good at continuous improvement, but you can be "lean" without any capability for improvement.
"Kaizen" is the Japanese word for continuous improvement. It was introduced to western eyes primarily from the "Toyota Way", which is a lean manufacturing system that concentrates on continuous improvement. It is important to understand, though, that while workers have a responsibility for identifying problems, it is the managers who are responsible for working with the workers to implement improvements in the Toyota way. So there are clear demarcations of responsibility, unlike many of the IT processes that borrow the word "kaizen".
Even "continuous improvement" can be a loaded term with respect to responsibilities. The first time I ran across the term was dealing with CMM (the Capability Maturity Model). Once you are up to level 4 or 5 you are using metrics to improve your process in order to obtain continuous improvement. In every organization I've seen that attempted to achieve CMM level 4 or above, the primary driver was management (and often workers were not even consulted about improvements or why they were necessary -- they were only told to do things a certain way). I'm not saying it was successful, but it was very common ;-)
I guess my point is that if you are lucky enough to be working on a team that does continuous improvement well, be aware that what you call "Lean" or "Kaizen" is unlikely to be what other people call it. You can not communicate what it is you want to communicate using those words alone. If you have been reading literature using these words and believe that if you simple follow a "Lean" system and embrace "Kaizen" that you will transform your organization into one that does continuous improvement well, I'm sorry to burst your bubble. If only it were that easy. Unfortunately, all the difficult problems are people problems and those problems almost always require unique solutions. It can be quite tricky no matter what you are doing.
I think the lesson to learn from this is to be wary of anybody who says they can look at a system for organising people that works well and write down a recipe for how to reproduce the essential elements of that system. Even if they were partially responsible for creating it. Most likely, nobody understands why it works in enough detail to write down precisely what it would take to reproduce it.
System of the organization comprises the relationship between systems within it, how those systems change and vary, the methods of knowledge learning and transfer, and the psychology and behavior of the people within it. And importantly, it's how all those pieces interact with each other that makes the most difference.
It's a system. There are some common elements. Above that base level, there is much truth in what you say, but the most important thing you can learn is a mental model for organizations in the most general way possible.
One of the hardest parts of this is organizations are not separate from there problem domain. A process that works for manufacturing cars (high value) may not work so well for manufacturing radios (low value) let alone banking software. Pay, education, and company size are all huge factors that are often ignored.
Part of the extreme difficulty in defining a mental model for an organization is that organizations are necessarily complex systems. You're exactly right.
Regarding people problems, my one note is that we are all more alike than we seem, and you eventually start to see the same classes of problems everywhere. This is why an understanding of psychology and behavior is so critical.
Except this big catch is that employees are compensated for the improvements they bring about. That basically never happens in a typical software company. Maybe if you keep bringing in important changes someone will notice and maybe you'll get stock options or you'll be promote to management
Yep - this is something that I'm thinking about actively. How do you start the conversation in an interview (as either interviewer or interviewee) that gives you insight into how people treat problems? Is their instinct more individual or systemic? Do they naturally look for improvement, or point fingers?
I've had some success, but they're always very deep interviews, which can either be drastically good or quite startling to the other party depending on what direction it goes. So your note to do it 'tactfuflly' is very prudent.
I have often found that two things work well for this:
* have your antennae out for anything that seems weird (weak signals, indeed) in your conversations about the work
* check references (as an interviewee). That's a fancy way of saying, look to see if anyone you know knows someone who works there, who can give you the straight scoop.
This is half true. People will go out of their way to make the organization better if it make their jobs easier (ex: switching to more modern easy-to-work with technology). But if for instance the project is going in a wrong direction, most of the time you have no good reason to fight to change things (and yes, it's fight b/c often it's about changing opinions and demonstrating repeatedly that things needs to be done different). So sure, do your duty and voice your concerns but at the end of the day just go with the way the boss decides. This is called delegation of responsibility and something a lot of people don't grasp. You need to cover you ass to stay sane.
Leadership leads. This is a tautology. It is unhealthy to pretend otherwise.
Lets take for Leadership decides to incur technical debt to allow a faster entry into market, Development Team Lead decides this is unacceptable, and does not take on the Technical Debt, but instead does things 'The Right Way'. This leads to competitors entering the market first, snagging customers and mindshare, which eventually leads to the company going under.
Was the Development Team 'Right'? Even if lets say he was 'Right', is it his decision to make, should he intentionally sabotage or ignore direction from his superiors ?
Most true WTF processes tend to come from decisions made that are out of the control of the parties that they directly impact.
> Was the Development Team 'Right'? Even if lets say he was 'Right', is it his decision to make, should he intentionally sabotage or ignore direction from his superiors ?
That's a bit of a loaded question.
You assume there is a right and a wrong that can be determined before knowing the outcome (a.k.a. deontology).
It's easy to see this is not at all clear-cut if you consider the other possible outcomes. What if it turns out that DevTeam's decision was what saved the company (and maybe even their competitors are now buckling under Tech.Debt).
Would that have made them right? Maybe yes. But in other's eyes maybe not because they still disobeyed their "superiors". But maybe the company would have fallen if they had listened.
Can you categorically say, beforehand, that one decision is right and the other is wrong?
You could, but others might disagree (and rightfully so, IMHO).
Say you claim it's always right to do what management tells you, and it's wrong to decide to go against that. And you can argue this because a large business needs that kind of structural dependability, otherwise it would fall apart in chaos and specialisation is a good thing, so management specialises in determining long-term vision that a Dev.Team leader is not as knowledgeable about. Sounds like a pretty tight justification, no?
Well, except when you're Dev-team and your livelihood (maybe family) depends on this job and following management's will crush the company, you decide to disobey management, and it turns out that indeed the tech.debt would have crushed the company instead of the entry-to-market delay. Then Dev-team gets to claim they were right. Even if management says "you saved the company this time but in general you ought to always listen to us even if you think it's a bad idea", except that (obviously) protecting their livelihood ranks a lot higher on Dev-team's ladder of oughts than following orders to facilitate a smooth tightly-run company.
GP is not pretending anything. Further, taking a position on technical tradeoffs (your example) is an orthogonal concern to defining leadership structure, which is also distinct from the possibility of making bottom-up changes. No one is proposing anarchy here, just people fixing what they can.
OK, I think there are some unspsoken givens that I need to spell out to make my point clearer.
Typically people resolve the issues that directly cause them pain pretty quickly if it is in their control. Further "control" requires understanding how the levers a party influences impact the obstacles that cause the pain.
WTF level issues typically occur when the above conditions aren't met, usually that means the 'pain' doesn't impact the party that has the control to end the pain. next most common is the party that has the control doesn't understand how its levers affect its obstacles.
just fixing what they can is not going to resolve WTF level issues...
One should always keep in mind that a WTF-level issue is very possibly a critical oversight on the issue-holder's side. Lots of things in the world appear very fucked up, but are in actuality slight improvements over even more fucked up states of affairs. One should endeavor to find out the history of the situation before jumping to judgment.
If you want for everybody to fix things, everybody should know things, share the vision.
If management is planning to do things quickly and badly, the dev lead should know that and plan accordingly, trying to do it in the best possible way.
If he's kept in the dark, of course he might blunder. But that's not his fault, assuming a company that encourages initiative and fix it yourself attitudes.
Best leadership leads by empowering people, not by enslaving them.
"Lets take for Leadership decides to incur technical debt to allow a faster entry into market, Development Team Lead decides this is unacceptable, and does not take on the Technical Debt, but instead does things 'The Right Way'. This leads to competitors entering the market first, snagging customers and mindshare, which eventually leads to the company going under."
To be honest, how often does this actually happen? I'd doubt it happens often enough to worry about it. Most startups aren't doing anything that special, and so this whole, "We have to move at twice the speed of light or we're all dead!" attitude doesn't belong anywhere.
> Most startups aren't doing anything that special, and so this whole, "We have to move at twice the speed of light or we're all dead!" attitude doesn't belong anywhere.
burn rate/runway is a critical survival factor to startups
You're conflating two things. One is the principle that engineering needs are only really important as far as they serve some business need. As a developer, I don't love this, because I'd like to be able to work on whatever I found useful, but it's true.
The second thing is people ignoring opportunities to make engineering changes that would be beneficial for both engineering and business reasons. They do this because "it's not how it's done" or "it's impossible" or "we don't have time for that". And this is a problem that affects both engineers and leadership. Some changes can't be done because leadership doesn't allocate the resources, some can't be done because people on the ground don't give a shit.
Since there are problems that can't be solved without leadership buy-in, you're definitely right that you can't fix these problems without having a role where you have a real ability to change the culture, and that you should be realistic about that.
There is some nuance here, though. A lot of leadership that might seem to default to "we don't have time for that" can be more reasonable when the problem, and (critically) the monetary ROI of fixing the problem, is explained to them in terms they can use to justify to their bosses. Effective communication of technical best-practices to non-technical people is something that everyone can practice. Sometimes you'll have irrational bosses, and that's just life, but more often than not they're just too busy to understand technical things, and it's still possible to spoon-feed them.
I appreciate your honesty, but I definitely would not want to work in a company where that was the prevailing attitude, nor would I hire someone who took that as their MO. It's a pragmatic policy for each individual, but it also leads to endless passing of the buck.
"What can I do, I'm just a dev, the team lead should figure it out" turns into:
"what can I do, I'm just team lead, the department head should figure it out" turns into:
"what can I do, I'm just the department head, the CTO/CEO should figure it out" which finally ends with:
"I'm just the CEO, these decisions should be made by the department heads and team leads because they have a better understanding of the problem."
Responsibility gets passed from the bottom to the top back down to the bottom, where the process repeats itself. Taken to its logical conclusion, everyone has a perfect rationalization for why all decisions are either too general or too specific for them to make. The net result is that decisions still get made, but no one questions them or defends them or even knows where they came from, because no one perceives them as a choice they had any influence over.
There are many reactions you can have to a lack of interest in your plan for organizational change. Quitting and moving on is definitely a contender. I am moving on, but I am taking the opportunity to accomplish several personal goals before doing so. My company would hate to see me disappear abruptly, and I've already got the go-ahead for a sizable consulting arrangement so they aren't left in a lurch.
Absolutely, there's always a balance between "voice" and "exit" options when it comes to organizations in need of reform. If you've been there awhile and feel like your voice just isn't being heard, exit is usually right move. And if you can exit gracefully, that's even better!
> To be blunt, they're paying you to do a job, not to make the organization better.
Ultimately what "they're paying you" for is not so specific. Generally, leadership doesn't have the skills to know there is a problem. Even in the cases where the leadership is technically exceptional, there is no way they can consider the myriad technical decisions and how they affect the future of the business.
Maybe leadership still doesn't care. If technical issues stunt the growth of the company or cause it to go under, the response could be "Meh, we had a good run." In that case, I, as an individual contributor, want to know that's the attitude up front. This lets me know to move on before everything hits the fan instead of be one of thousands laid off at the same time.
> Ultimately what "they're paying you" for is not so specific.
I think you'd find in most organizations, roles are well-defined, but not articulated to those assuming those roles, because it helps morale to let employees define themselves, and also keeps them from getting too complacent. A manager's job is not to lead but to manage, i.e. get the greatest possible output from the employee.
If you want to know what your defined role is, a simple way to do so is to simply stop doing your job and seeing what people complain about first.
You're right in that leadership can not know the intricacies of your domain, but you're wrong in assuming there's something special about that state of affairs as it concerns technology. Leadership does not know the intricacies of, say, human resources, that's why they hire a specialist. Even in areas where the leadership does know how to do the grunt worker's job, their attention isn't focused on that area, so they're not going to know about problems until someone, typically a manager, brings it up.
A company's leadership is typically engaged outward, towards the broader market. Our CEO handles big deals with other service providers and retail outlets. If you think about medieval Europe, the petri dish that modern organizational methods evolved in, it makes sense. Someone needs to keep tabs on what the neighboring states are doing, that someone needs all the resources of the nation at his disposal at his command so he can deal with regional situations.
Individual contributors, when they try to "rock the boat", are perceived as unnecessarily taking time away from the much more important job of keeping tabs on the broader world / market. It's not directly making the company more competitive, so it's just noise, is the attitude.
The attitude / culture I described above is the norm, everything else is an exception. You can take this paradigm, and use it carve out a little fiefdom in any traditional hierarchical organization. Essentially, you figure out your defined role, do only the minimum required to not get fired, and devote the rest of your time to company politics. As you come to understand the organization and its needs, you'll be able to position yourself as someone who can meet those needs. If a company needs it, then that means it can't get it with the current resources it has, otherwise it would have it already. So you will need a budget and staff.
> If you want to know what your defined role is, a simple way to do so is to simply stop doing your job and seeing what people complain about first.
Whenever I've done something like that in the past, the complaint is always some variant of "I noticed you goofing off".
Historically this was a source of incredible frustration because it basically acknowledged I was undertasked; the complaint would never be some variant of "Why isn't X done yet?"
I suppose that means I had no defined role in the company?
> the complaint is always some variant of "I noticed you goofing off".
OK, there's a political status-quo you have to learn how to internalize. If you work at the front desk, you can fuck around on the computer, but you can't take a book out and read. One looks like work, the other looks like fucking around. You can't make your organization look bad. The perceptions are more important than the substance. If you can't get the perceptions right then you were never going to make it in corporate America, and should probably stick to contracting.
So the way to actually do this is, to look like you're working, but actually be producing nothing. When they complain, you can say you were busy on X, where X is some obviously unimportant meaningless detail. That forces your boss to clarify what he expects you to be working on.
That your boss never asks you about specific tasks means that your defined role is to be a repository of knowledge and not a cog in a machine. This is a good thing, knowledge workers can bullshit their way to perks that the rank and file could only dream of. My current job is just such a thing.
The key to this is understanding that nobody actually knows what you do. Management can only guilt you into being productive, they have no way of actually knowing if you are being productive or not. Perception is reality and you control the face you put out to the organization.
Pretty much. And there definitely are jobs like that out there. I once had a job where literally no one would have complained if I didn't do any work except for sitting at my desk, replying to e-mails, and attending meetings (which I don't consider "real" work). Because I was part of a management training program, I was not expected to produce any output, I was just "there to learn." In reality, that meant I was a professional internet surfer and no one cared. I was miserable and left that job as soon as I could afford to.
No, I think Dan has a pretty good handle on what the respective needs of companies and their engineering teams are. I think a more accurate gloss is that people's personal self-interest and habits are more important (to them) than the needs of the company. When doctors don't wash their hands, it's not because "the company" needs to kill more patients. It's because they're in a hurry.
I would strongly disagree with you on both points -- that you're getting paid to do a job, and that you have to be paid to be a leader.
For one, you don't become a leader by being paid to be a leader. That's how you end up as an incompetent manager. You become a leader by taking responsibility and doing what you can to ensure those responsibilities are taken care of. One of those responsibilities is to make your organization better.
And you shouldn't think you're getting paid to do a job. I'm going to do a job whether I get paid by my employer or not. What I'm getting paid for is to take my employers priorities and goals into consideration, and the strength of that consideration is proportional to the pay. If you don't pay me very much, my priorities will be considered first, and one of my priorities from a workplace is to be comfortable and happy within a good organization -- I will work to help and support my friends and coworkers. So that's what I'll focus on. If you want me to 'just do a job' or 'maximize company profits', you need to pay me extra, because I won't care about those things for cheap.
You must be miserable at work if you think you're getting paid to just do labor. That's a waste of an education, assuming you have one.
> that you're getting paid to do a job, and that you have to be paid to be a leader.
I did not mean these things the way you seem to think I meant them.
> What I'm getting paid for is to take my employers priorities and goals into consideration, and the strength of that consideration is proportional to the pay. If you don't pay me very much, my priorities will be considered first, and one of my priorities from a workplace is to be comfortable and happy within a good organization -- I will work to help and support my friends and coworkers.
I use a similar rubric to decide how to prioritize my time. My defined role comes first over everything else. Because I am accomplishing my role extremely efficiently my company sees me as a very good employee.
What I do with the rest of my time I consider to be my sole discretion. If I have an idea for something I'd like to build for the company, I'll go over it with my manager to gauge interest. Sometimes he's interested, sometimes he's not. I do not sweat lack of interest, I am an idea machine, I can come up with new ones.
I look at this surplus time as the primary benefit I receive from getting better over time at my job. I spend maybe a couple hours per week on defined roles, the surplus time I mostly re-invest back into my own capabilities. This helps both the company and me, but mostly me. The company simply isn't set up to be able to utilize my talents effectively.
This is my thinking also... it came to me over a long time, and typically has a lot to do with why I switch jobs. At a certain point i give up struggling against the 'WTF' and just start going along with the flow, then one day i look at the stuff i am doing, realize working at the place is making me a worse professional, and move along...
However I do want to note that in general these things do have real concrete negative impacts on the bottomline of the organization.
> However I do want to note that in general these things do have real concrete negative impacts on the bottomline of the organization.
Oh absolutely. But your duty to that organization is to raise these issues to the person best-equipped to see the bigger picture, and to do your best to convince him it's a real problem. Once you've done that, your work is done, you have just done more to help your organization than 100 blog posts would have accomplished, and more than 90% of the other employees would have ever done.
You can comfortably use this approach once a month and be vastly more effective than everybody else at your company at bringing about change. Simply raise an issue, have a conversation about it, then drop it if you get no support.
Exactly. It also works as a meta principle. If you raise issues without them going anywhere, that in itself is an issue worth raising. Sometimes it means going to the manager's manager, but it does work. And if it doesn't work that's an excellent exit cue.
To expand a little, I don't think the point of having a positive effect on the WTF parts of an organization has anything to do with my effect relative to other employees.
If I see a WTF security or ops practice, I don't sleep better at night by telling myself, "Welp, I noticed it and said something to someone, and that's a lot more than most people do."
Maybe I think of my job too broadly, but my job as a developer is, at least in part, to protect the business, to find the WTFs and get them sorted out.
The idea that IT is just there to solve the "business problems" and they should shut up about anything that isn't directly related to making money is absurd to begin with. But even if I'm being completely naive about that, it still implies a really shallow understanding of "business problem."
Security is a business problem. Stupid ops habits that result in downtime are a business problem. Groupthink that results in acceptance of worst practices is a business problem. All the whatthefuckery the article talks about boils down to business problems. Many of these problems are such that the people primarily concerned with running the business are not in a position to recognize as problems.
It absolutely is my job and every developer's job to find these things and take care of them.
If I ever found my self in a situation where I saw some WTFs going on and I went to my boss or a coworker and got no explanation other than 'This is how we've always done it.' I would be concerned. I would go back to my desk and finish what I was supposed to be working on. Then I would go home that evening and write out the clearest and most concise explanation of why this is a serious business problem with citations and take it back to the coworker or boss.
In my experience, the WTFs don't usually come from this is how we've always done it. They had some reason back at some point in the past where someone really needed it to be that way for some reason, and usually temporarily.
Here's an example from several jobs ago, one of my first in the industry, actually:
Why does this set of boxes still have password auth enabled? Why isn't it locked down to ssh key only? Why is port 22 accessible outside of the VPN?
Oh, it's because our CEO likes to get his hands dirty with code every once in a while, but he didn't want to mess with pub/private keys, so we just left it open because these boxes weren't that important. They were 'just' dev machines pointed at test DBs with no real data in them.
But it got written into a setup script or Wiki somewhere, and when those dev boxes got repurposed for production, people followed the scripts or rules or whatever that were specific to those machines. So now you have prod machines running with password access with SSH exposed to the public. Ones that are pointed at live DB servers with creds for them.
That's a serious WTF.
I'm not even close to a security expert of any kind. I wouldn't claim to be in a million years. I work with databases and Python, almost exclusively. Though I dabble in DevOps when I need/want to. Even I know that the above is a serious WTF.
What did it take to get that fixed after my coworkers said, "Yeah, that's how it is."?
Going straight to my boss, who also said, "Yeah, that's how it is."
Then spending maybe an hour of my time writing up how much of a business problem this is and going back to my boss, who said, "Yeah, I know it's a problem, but I don't even know why it's like that. It just is."
So then I ask him who he can think of who might know why that is. Oh, quelle suprise, his boss might know.
And indeed, his boss did know. But he had long since stopped trying to get the point across to the CEO who liked to dabble, so he just went with it. Here comes my very brief paper explaining why this is--you guessed it--a business problem. The CEO responded within a couple of hours requesting that someone come setup SSH keys on his computer and shut down the passwd auth and close port 22 on all prod machines immediately.
That's a lot more work than mentioning it to someone. But I think that it's my job to do that, even though I've never worked in DevOps or Security teams.
My work was absolutely not done when I asked a coworker about it once, and then asked my boss.
-----------------------------------------------
Edited to add:
Astute readers will note that there was a lot more WTF-ery going on than just auth and port exposure on prod. Like, for example, that the non-technical CEO who liked to dabble was doing so on a production box because he was not made aware that the boxes he was logging into to play with had been repurposed.
There was a lot of cascade in tracking down that one particular WTF, and many other WTFs were solved because of it.
At the time for that company, my job description was C# middleware for a web app. I maintain that it was absolutely part of my job to pursue that WTF to its endpoint, and that doing anything less would have been completely irresponsible.
Whether or not it's strictly within my job description, I think dealing with things like security vulnerabilities and poor operational practices definitely falls within my ethical duties as someone who claims to be a professional.
> However I do want to note that in general these things do have real concrete negative impacts on the bottomline of the organization.
Not true. Sometimes, things that have concrete negative impacts can BENEFIT organizations.
U.S. prisons are just one of many such examples of this. A concrete positive impact would be lower incarceration and re-offending rates. American prisons have the exact opposite effect. This leads to lots of "customers" in the form of prisoners and victims seeking a more abstract "sense of justice".
The American prison system is probably not the only industry with that kind of business model. There are, no doubt, many contexts where a "concrete negative impact" can be very good for the bottomline. It's also an important lesson about making money in general. It's not about creating value. It's about making others pay for your actions regardless of whether or not said actions are "positive".
> Sometimes, things that have concrete negative impacts can BENEFIT organizations.
That's a very good point too. If you wander, unaware, into a quest to change something with negative externalities on workers or the public at large but with a positive effect on the business, you'll find yourself suddenly in a minefield of opposition with a target painted on your back and no idea who you've just made enemies of. Tread very carefully.
If you are able to positively identify negative externalities that produce benefits for your company--benefits your company depends on--you shouldn't tread carefully. You should quit your job.
Classically if doing contract software work, bug-ridden software meant a secure maintenance contract for years to come.
Though when I got out of that sort of thing, I recall there being (relatively) new rules requiring a different entity to do the maintenance from the ones that created the software... but my understanding of FAR is basically non-existent, so I don't know.
There's a variety of things you can still do as a middle-tier or above engineer to try to change the situation. (Very junior engineers are advised to A: watch how the situation develops with those poor practices and learn and if ambitious B: find a mid-tier or above engineer to ally with for these matters. You lack the social capital on your own to do much.) In my experience, at least with my development style, writing unit tests is either a net time wash or for larger systems, a net time gain [1], so use them. Even if nobody else runs them, you still get a better system out of them, and you get to run them when somebody complains. You can use analysis tools on your own code. You can set up an instance of a build server even if nobody else has one, and have it monitor your unit tests and other tools and fix things as others hack on your code. Etc. etc.
There's two basic outcomes that can happen here. Either you become a leader, and gradually and often quite begrudgingly at first, people start following when they see how well it works. (Unit tests can be a real eye opener sometimes, especially when you start having reasonable cause to show how the problem is unlikely in your code, being either in the other code or the specifications.) Or you get quashed from above. Contrary to the cynical answer, the latter is not inevitable, but it is certainly a possible outcome. At that point, yeah, it's just time to say you got some valuable experience and move on.
What you MUST NOT do is simply whine... from the point of view of those above, anyhow. Even if it's perfectly sensible complaints from your point of view that are all but objectively correct, it's unlikely to be heard as anything but whining. You need to lead by example. You also very much need to do so with some idea of cost/benefits analysis; you can't go from no discipline at all to a perfectly disciplined project in one step, so consider your steps carefully. Keep them small; stay away from "big rewrites". (Probably the biggest failure case I've seen is someone who thinks that code X is in the wrong paradigm and sets out to entirely rewrite it to make it "better". YMMV but in my personal experience this is usually someone who thinks the code needs to be OO, or a different kind of OO. This is guaranteed failure. Even at surprisingly small scales! You destroy everybody else's knowledge of the code.)
And I'll say it again to underline it... the cynical answer that this is impossible is wrong. It certainly won't be easy, but I can guarantee great growth as a developer if you follow this path, both technically and in dealing with people. Even if you have to change jobs.
As for the bottom line point, arguably you have a duty as a professional developer to be doing this stuff I describe, precisely because it does mean resources are being continuously and avoidably drained on issues that shouldn't exist. If you find yourself unable to discharge it, you should find somewhere you can.
[1]: I like to say that it lets me develop with monotonic forward progress. I even use unit tests during prototype work quite often, after I got sick of the way during prototyping I couldn't count on anything to work, ever, due to changes, and I realized that itself was actually inhibiting my prototyping ability. Sure, sometimes I dump entire subsystems but even then it was usually because the unit tests showed me a fundamental flaw far earlier than the rest of my prototyping would have, and I do so with far more information about the local landscape than I would otherwise have had.
Thanks for this. I have used somewhat similar approach and ended up writing bunch of testing tools for our app over a period. I can see its effectiveness as quite a few people in our team and outside started using them, or made their own copy and modified for their needs. Now I am better off as a developer but appreciation in term monetary compensation / promotion did not come through. However unlike others I do not have option to change jobs due to visa issues etc.
Well, sounds like you're doing your part, it's just the legal system screwing you up.
I wanted to tack on that while the "change jobs for de facto promotion" technique is solid and time tested, doing the sort of stuff I mentioned can help you climb faster. If you're planning on that career path, that's great skill development. You may still advance if you just clock time in before moving on, but you'll find you don't advance as far and that you seem to be getting the same job over and over. (At least, statistically.)
Best of luck with the visa issues. If nothing else, keep an eye on the long term. Development today may still pay off later.
(I mean, don't go crazy. I'm not big into unpaid work. But not all "eight hours" are created equal.)
This is absolutely true. However I just generally dont see the point in 'fighting' superiors... its not that its impossible to fight the system and make things right, its just not a good investment of an individual contributors time and effort. much better investment to move to an organization that is either already aligned with positive processes or actively seeking to improve processes.
I bristle at the sentiment of "things that are bigger than me." Perhaps my aversion to this notion stems from the fact that I've worked (mostly) in small companies. Folks at my now bigger company say things like: "above my paygrade" or "out of my hands" or "I'm too low on the totem pole."
I admit that by straining to fix these hard things, I'm often going outside of the formal role I hold at the company. I'm fine with that. Others may not be at times. For me, the straining is required. We have a cultural goal to "think like an owner" and that is how I typically justify my behavior.
I agree with one caveat, the "leadership" you mention is actually "management". Actual leadership requires you to do a hell of a lot more than just what you're told/serving your own bottom line.
Companies have business needs to not go down and to not lose customer data, both of which are very real consequences of poor engineering practice.
Managers are also acclimated to the business's practices and may find others absurd. The essay is suggesting that hey would be better managers were they to listen to the WTFs from newcomers.
I don't think you have to compromise much. If you have any self-worth, you'll strive for improvement. You don't have even have to that careful about rubbing folks the wrong way. You're not going to get fired or anything.
You won't get fired, but what might happen is that you get a reputation for being "that guy". The "guy who's always complaining about unit tests". Or, "the guy who emphasizes process of 'agility'". What then happens is that the moment you speak up, management tunes out. They know what you're going to say. They know what you're asking for. And they've already told you no. At that point, you're basically screwed, organizationally. You have a reputation as a troublemaker, which will make transfers difficult. You're not going to get the desirable assignments on the product that you're currently working on. So even if you don't get fired, your life is often made so miserable that you leave.
If you're upset that there's lousy test coverage, and you respond by complaining, that's not useful. If you respond by making the test suite run faster; creating better mocks; writing docs; making the test results more visible; teaching others how to write (better) tests... that's a different story.
The trouble comes when you think that pointing to a problem is, in and of itself, valuable. We're all surrounded by problems; pointing out one that we likely know about probably isn't useful.
If you respond by making the test suite run faster; creating better mocks;
writing docs; making the test results more visible; teaching others how to
write (better) tests... that's a different story.
The trouble with that is that you only have a limited amount of time. If you're spending time writing tests (even if those tests save time in the long run), it's going to eat up time in the short run, which cements your reputation as a "troublemaker". Namely, your boss is going to wonder why you're taking extra time to write tests and create mocks when your peers are cranking ahead with features.
Yes, in theory, you get to say, "I told you so," when your peers' features are found to be bug-ridden pieces of crap that have to be reworked two or three times before they can be deployed. In practice, a boss whom you can say, "I told you so," to is a boss who'd have listened to you in the first place, making the whole exercise moot.
My life got much, much easier once I learned to stop straining so hard to fix things that are bigger than me. If you don't like your managers, or the culture, or the business, find another place to work, it's that simple. If you can't do that, you're simply going to have to learn how to compromise.