There's also often a messy transition period where a team goes from a bunch of generalists to more specialists. This can be extremely rocky for all involved as often generalists have lots of historic operational knowledge and strong opinions, but the new specialists come with external knowledge of best practices and how to scale.
It is important that historic knowledge is respected for the context it provides, while outside knowledge is equally respected for the deeper context and data behind it.
Unfortunately, this change often means generalists lose jobs in favor of specialists as a company grows if there isn't a good place for them, or if they don't rise to management. In fact, even early managers can be displaced by outside managers who come in with more management experience. And this is not inherently a bad thing for the company (while it may be for the employee getting bumped). The people that take a company from point A to point B are rarely the same people that take it from point B to point C.
Am I crazy to think that generalists are the best-suited engineers to making the managerial move? You can keep a good broad knowledge base sufficient to be useful in early directional discussions, and sufficient to sniff out a lot of (not all) BS artists and identify which specialists to listen to/loop in, in less time than it would take a specialist to do the same in terms of juggling things across team, due to the nature of the generalism skill.
I think otherwise trying to be a "senior generalist" or "staff generalist" is going to be very hard, but for logical reasons.
It's certainly a factor, but I'd argue that by a very long margin, people management skills are the best indicator for engineers suited to making the managerial move.
You have to know what you're talking about and know when to defer to experts, and being a generalist might prepare you well for that. But once you've reached the threshold on knowledge and critical thinking, the biggest barrier to being a good manager is mastering the mechanics of people wrangling.
Being a tech generalist is a great background for a manager. But there are some caveats.
Firstly, besides understanding the work, a manager also has to be able to actually manage subordinates. This is orthogonal to technical ability.
Secondly, internal promotion of a generalist carries risk of that generalist being too opinionated based on his own work. They are much more likely to be a conservative force. This might be an unnecessary barrier to good changes.
Disclaimer: I would categorize myself as a generalist so I might be biased here
> Unfortunately, this change often means generalists lose jobs in favor of specialists as a company grows
> And this is not inherently a bad thing for the company
Yes, it is not _inherently_ a bad thing, but I would say that it is _generally_ a bad thing. Sure, if you are raising a Series C and everything is about optimizing the hell out of your product, there might not be a place for generalists.
However, I would claim that if you throw out generalists, you throw out some of the core/most valuable people who could help the company establish new products and help you grow into a multi-product company (diversifying your portfolio is generally a good thing). If the company is only left with specialists it will slowly transform into another one of those industry giants that struggle with innovation.
I am not sure if it as bad as described here. Yes, over time the relative part of generalists in a company shrinks. But it does not necessarily mean, that generalists loose their jobs, just because some specialist knocked at the door.
I think it is more like an organical transition where newly hired personnel are more likely to be specialists. And that is totally okay, because when you start a company there are about 0% specialists onboard ;-)
Late reply, but I think there's a second harm to throwing out generalists in addition to what you outline:
Specialists aren't as good as generalists at integrating the work of specialists across domains (almost by definition). A lack of generalists can then lead to a situation where specialists are all making locally good moves, but the overall direction is negative -- something akin to Simpson's paradox (though, in the other direction).
I think companies need _both_. Specialists can miss the forest for the trees, and at some point you need to go into the last 20 of the 80/20 rule, which requires supplementing or specializing your generalists.
And I believe it's entirely possible to be both; to be a better specialist you need to be a better generalist, and vice versa.
>often generalists have lots of historic operational knowledge and strong opinions
*citation needed
Edit: to be clear, in my experience the specialists that aren't also generalists on some level are the least productive on the team because they have trouble participating outside their knowledge domain. Maybe I have worked at the wrong places though...
Starting dev of some product, you might have a lot of generalists. They make the original architecture and design decisions, do the first implementations of features, etc. It's their baby. Later on, the company identifies laser-focus areas where specialists are appropriate, and you contract or hire some. The original devs (the generalists) will have knowledge about why something was built a certain way, and might react strongly when the specialist advises deep changes to the program to help it scale up.
That's the messy part of the transition: There's kind of the implication that past decisions weren't correct. It's easy for people to get emotionally involved and take personal offense.
Example: Backup product. Design started in the late 90s. By late-aughts, multiple CPUs were obviously a serious thing. New guy was brought in, and set on the project of re-architecting the core program into a more asynch data processing model. 6 or 8 months later, the changes were working for base cases, but the senior devs threw an absolute hissy fit focused on how much of their code was changed. We should've banded together and fixed the edge cases. Instead, we spent years fighting over the structure of the program, and that dev transferred to another group.
Bringing in security-focused employees worked out better. Their changes were more around the edges of the various programs, rather than in their cores. Fewer toes stepped on, fewer disagreements.
>"That's the messy part of the transition: There's kind of the implication that past decisions weren't correct. It's easy for people to get emotionally involved and take personal offense."
Thanks for highlighting something I missed in my post. Some of the time yes, the decisions might just be wrong because generalists may not be experienced enough to know the right way to do things. But quite often, they were in fact the right decision at that time and circumstances change.
So any sort of process and culture that helps take the emotion and risk of personal offense out of those situations is really critical to gaining the benefit of historical context in a positive manner.
To clarify, when a company is young, you have a lot of people wearing a lot of hats. They may in fact be quite good at wearing some of those hats. Likewise, having visibility and exposure to multiple domains aspects of the same company gives a perspective that is hard to have if you live in a super specialized silo.
However specialists don't need to be generalists to be productive as long as they come in with the mindset that they are specialists, and therefore may not know everything. And that is especially important, as I highlighted, for respecting historical context of the company and its operations.
There might be VERY good reasons for not doing something the way the rest of the industry does it (which is what the specialist might be inclined to do). However a good specialist should also be adept at factoring that into how they approach problems, and properly weigh the new information against their past experience to determine if in fact this is a special snowflake situation, or if their approach is still the right one. Conflict can often arise when if they push for the latter, so strong communication skills are key (and that is something that never really changes for any role at any stage of company growth).
I get where you are coming from. For me, being a long-term employee having gone through a few transitions from smallish to biggish, and when your storage guy you hired doesn't really know networking basics, or the database administrator has no clue about Linux, etc. it tends to slow everyone down.
"Scale" often seems to equate with SF hotshots from one of the big names that will come in and airdrop a brand new architecture. These things take years and you need those generalists that have the tribal knowledge and the ability to jump in (almost) as deeply as a specialized developer, DBA, Network Admin, etc. I think the SRE movement is good evidence of the type of breadth and depth needed to operate a complicated system at scale.
IMO that's a problem of hiring the wrong specialists, or BS artists in specialist hats (see the article's point about "the tech industry is not serious about hiring" :) ).
In other words, we have to go to war with the army we have, not the army we'd like to have. Sometimes waiting for just the right fit isn't an option, either because of the need to keep adding heads for growth, and sometimes you have situations like a Director (just brought in from BigDealCo to lead the growth) has a former employee that knows is "perfect" for the job. Director leaves after a year, after installing people that don't have skills. Catch-22 sometimes.
It is important that historic knowledge is respected for the context it provides, while outside knowledge is equally respected for the deeper context and data behind it.
Unfortunately, this change often means generalists lose jobs in favor of specialists as a company grows if there isn't a good place for them, or if they don't rise to management. In fact, even early managers can be displaced by outside managers who come in with more management experience. And this is not inherently a bad thing for the company (while it may be for the employee getting bumped). The people that take a company from point A to point B are rarely the same people that take it from point B to point C.