> I can see, that I’m indeed a Systems Architect, and not a Software Engineer anymore.
I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
I prefer how my company career ladder, where there are no architect titles. Each level is 'fractal', containing almost the same responsibilities but at different scales and scope.
So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams. This way architecture is everyone's responsibility, and people have a natural path to grow their skills.
I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
Doesn't this distinction exist in every profession? Experienced people start doing less and less of the day to day tasks, and start doing more strategic work.
Only in software do we think this is a terrible thing, and that an acceptable use of someones 10 or 20 years experiences is for them to be down in the trenches centering divs with the new grads.
Hence terms like "staff engineer", which is a modern architect but with a name that pays lip-service to egalitarian fantasies.
Agreed, just because a particular job function isn't someone's responsibility doesn't mean that they can't learn from those who do. A junior developer isn't going to be making architecture decisions at most organizations and in a lot of cases, architects are in more of the strategy meetings that lesser experienced engineers bemoan having to attend anyway.
Organisations who don't give architecture work to juniors are terrible places to work for juniors. Zero career progression. You can only really learn good architecture through practicing and making a lot of bad design decisions.
I did architecture work when I was a junior. It's what made me a senior. I got asked if I wanted to do architecture work recently. I told them that's what we hire juniors for.
Architecture is piss easy. Any senior thinking juniors can't do it are full of themselves.
It’s more that when push comes to shove a staff engineer will center divs with the new grads, while a ‘software architect’ will say it’s not their job to center divs.
> "The SystemArchitect responsible for designing your system hasn't written a line of code in two years. But they've produced quite a lot of ISO9001-compliant documentation and are quite proud of it."
Just how many architects with decades of coding experience have you actually met? I'm yet to see one (in my 3rd decade of professional programming). Usually it is people who branched out from programming very early.
I'm about 15 years into my career, and I've been writing code the whole time. In hindsight, a lot of my code has held up well. I just heard that code I wrote 10 years ago is flying in space, so that's cool. I've been at my current job 6 years, and since it was a small startup when I joined and it's grown fairly big, I'm one of the most senior employees. About a year ago I was given the title, "software architect", mainly just so that I can officially make unilateral technical decisions in the presence of ambiguity when there's a schedule crunch.
Just in the last few months, I've found it difficult to be personally productive writing much code. Between catching COVID, then a string of other viruses, and having a young daughter and a baby on the way, I just haven't had a lot of quality focus time. Instead to feel like I'm earning my paycheck, I've had success in writing out high level architecture documents and diagrams, then relying on my coworkers to take over ownership and execute on the details. It's actually seemingly working out well, since I'll be going out on paternity leave toward the end of the year.
How anyone could actually BE an architect without a lot of actual developer experience is something I'd like to know. Are you sure you're not just being unfair to the architects around you for disagreeing with you on things?
I cannot fathom an architect who wouldn't be a strong developer if slotted into that role.
I've mostly stayed on engineering tracks where architects are untypical roles so had a fortune observing from a distance. Only did a few year stint with Java + CIM where architects were endemic. Now that you asked I sifted through memories a bit and yeah none of the one I worked with or observed have been coding for more than 3-4 years prior to their ascension.
I just don't see how any architect can ever be very good without a strong background in development. I'm of the opinion that an architect who isn't regularly getting feedback from the implementing teams is probably a bad architect. Their decisions will often make the implementing teams job harder and they need to understand that in order to balance broader needs with implementation smoothness.
> I just don't see how any architect can ever be very good without a strong background in development.
I think the disconnect here is that the majority of architects I've worked with haven't been "good" in a technical sense from a "actually building software" perspective. I'm assuming this is true for a lot of folks here, based on people's responses in this thread.
The day to day of some of these roles has been:
* sit in meetings with high-up product managers and biz leaders
* come up with pretty diagrams of how to totally re-build (sorry, re-architect) the system. Often this is jargon-heavy - protobuf this, envoy that, circuit-breakers the other thing.
* talk engineering management into this plan
What's been meaningfully missing is the details of point A -> point B to achieve their plan. None of the jargon is necessarily wrong but it's not grounded in the reality of the current software systems at all.
This comes about when the technical VPs aren't hands-on with the system anymore either. If non-technical leadership is saying something like "improve uptime" and you go get hire an architect into the company and you'll get a lot of presentations about things the devs on the team may already know about but haven't been able to get prioritized, but it's still on the existing devs to actually build it.
"Paying someone fancy to convince non-technical business leaders to let the devs build what's needed" isn't necessarily a bad decision from a CTO or SVP of Engineering, even! But the devs who have been saying the same shit this whole time only to now see the credit go to the higher-ups and the new architect, after getting the blame for the old status quo, don't exactly have a positive view of the architect's actual engineering competency.
> None of the jargon is necessarily wrong but it's not grounded in the reality of the current software systems at all.
Oddly enough, I'm having this discussion right now with a few other people at a new company. They want new documents to describe the ideal and I want it to describe how things work currently and evolve it over time.
OTOH, I've never gotten the impression that any of them are technically weak, I think they just view their role differently than I do. Much like how a TDD proponent will start imagining unit tests are the point, I think these architects have started to think these documents and diagrams are the point.
My experience is a bit different, the end role for engineers that don't want to land in management is usually becoming an architect, or staying as tech lead forever, unless they branch out into consulting.
From what I've seen a rather typical trajectory is towards a product manager if you are in a tech megacorp or CTO if you are in a small upstart/company (who of course is nothing like CTO at a megacorp). YMMV of course.
I just got the job and have 16 years of coding experience, with complex stuff touching major operations in several companies (not to brag, but to prove :)). Now I am doing still coding of the left-overs, but the new role is really mostly designing systems. So I think your experience is not that universal. But yeah, I've also met several architects that have like two years experience, and then branched out.
I was one, though my job title was even snootier. I still wrote a lot of code but there was some precedent for that in my immediate group, and we were always short of (real) senior engineers. Zooming out, I don’t think the natural born hackers and the people promoted to Architect overlapped too much. But once in a while, yes.
HA said the master craftsman to the apprentice. Doth thou really think a master such as myself would hammer nails? I am in important zoom meetings! Surely it would be distasteful to touch such dirty tools at my level.
> Doth thou really think a master such as myself would hammer nails? I am in important zoom meetings!
Yes because aligning goals, putting together a system,scoping work, and synching business goals with technical milestones adds no value at all. It's the nail-hammerer who believes that JIRA tickets spring from the JIRA ticket tree magically aligned that caries the whole org on their shoulders.
How much nail hammering does a foreman at a construction site do? He's hammered nails before. Hammering different looking ones in a new location is not going to be very useful or educational for him.
And I've seen what happens when a whole team of devs get invited into a zoom meeting like that - they're bored out of their skull, most likely sitting there browsing the social media du jour, and I can't blame them.
At my last job, somebody made everyone custom, "Don't blame me, it was {excuse}" t-shirts, tailored to their roles. My manager's shirt said, "Don't blame me, I was in a meeting".
I think the more accepted title these days instead of architect is "staff engineer". If you read older architecture books like "the software architect elevator" or "12 essential skills for software architects", most of the book is about technical leadership, driving change, personal skills, delegation.
Building some theoretical construct and having other people build it is by itself bad architecture, resulting in bad team culture and bad systems.
Agreed on fractal model with healthy room for mistakes that should be carved in for everyone - it's the only model that can enable healthy growth.
Non-practicing architects are 0, and non-architectign engineerign is 0 too.
This said, as much as I hated the architect idea earlier in my career (because I was architecting regardless of my title) - the concern and ability to architect things is a very real one. It can only be learned by taking responsibility and having room for mistsakes.
I agree that Staff engineer is in practice the closest thing to the old Architect title. I still think this approach is better.
If nothing else it's a shift in the responsibly of architecture. In the most rancid organizations there are silos where devs only code and architects only design. The joke of the Ivory Tower Architect is real. On the other hand as a dev II (between junior and senior) I was asked to design and implement a large scale green field distributed system that interacts with other teams. Our ladder and job titles not only allow this, but it's encouraged and sometimes expected.
In my previous roles in larger orgs, the architect vs principal eng distinction was a breadth-vs-depth thing. Are you planning or aligning broadly across 10 teams, or are you helping both roadmap and do longterm planning but also diving deep into the hardest technical problems for just 1 or 2 teams, say.
If the architect from your second paragraph works closely with the dev team for prototyping and iterates on their feedback I don’t see anything wrong with that approach.
yes exactly. I was referring to what I think the OP was rejecting, a rigid top down approach to software engineering, with the architect and their UML at the top, and the grunts and their editor at the bottom.
> So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service,
This is how you end up with antipattern-laden monstrosities. And I don't mean that in the dogmatic sense, I mean juniors will find "clever" ways of getting the job done, often with far too much abstraction and reinventing core technologies like message-passing, task schedulers, databases, ORMs... It can be a good learning experience but they need a active mentor/architect overseeing every major decision.
If you are going to reinvent something you should at least be aware of it. Juniors just don't have this broad perspective yet, almost by definition.
OTOH, one of the my pet peeves is an "architect" who gives this advice I am giving but doesn't actually have the requisite technical background, i.e. saying "don't reinvent the wheel" while looking at a teacup.
> This is how you end up with antipattern-laden monstrosities.
ah haha I just stepped away from my work computer after creating a new branch and writing a "short term fix" in a particular module instead of fixing the root cause because that would involve talking to multiple teams. Instead, I can just do some simple string manipulation just for the one bug I have...
I am trying to talk myself into or out of creating this pull request...
In my current workplace it’s basically impossible to fail design review - like I’ve never seen it happen, and this is becoming the norm across the industry. At this point design reviews are just a cargo cult ritual
Yeah, code review is one format that can work for this, but my broader point is that trusting juniors to make architectural decisions even on the feature-level is a waste of time and effort. There must be oversight and guidance.
This is where the fractal nature of the role helps.
In your example, a mid level dev would oversee the architecture of the service. This dev would work to the junior to ensure their approach fits the rest of the service, and review their work. In turn a senior eng would oversee the architecture of the system owned by the team (possibly involving several services), and keep tabs with each one.
It might be harmful if it was universal, but it's really just another facet of the normal, perpetual tension between generalization and specialization.
Large organizations working on grand projects benefit from slicing up roles so that experienced specialists can anticipate and prepare solutions for the uncommon challenges that arise in those projects (and so that the bureaucracy of large orgs can be managed at all).
Meanwhile, smaller organizations, or groups working on more modestly scoped projects, benefit from generalists who can hold much of the project in their head and develop highly cohesive solutions that can be delivered efficiently.
A skyscraper requires seasoned architects or its going to crumble under its own weight, but even an extensive residential home renovation can be planned by a general contractor and done way cheaper than if many specialists were brought in.
It sounds like there are some organizations/projects you wouldn't want to be involved with, but that doesn't mean that those organizations/projects are doing something harmful or regressive. They just don't suit the way you want to approach work.
But the existing situation is the reverse of what you described.
The largest tech companies out there (usually? anecdotal) do not have this distinction, it's the smaller and older companies that continue to enforce this arbitrary divide.
A quick Google search of software architect jobs for Google proves otherwise. Like the comment above, large orgs slice up these roles and have architects (like myself) to help guide software teams into cohesion on platforms that require high collaboration between services to support a multitude of products. Anyone who says architecture isn’t needed, isn’t doing anything at scale (or blissfully unaware that their ship is sinking).
Great Architects work closely with the dev teams to give them reference diagrams, proof of concept code, and the art of the possible. Bad architects drop a big requirements doc and walk away. Almost every large organization has architects (enterprise, solutions, software, and cloud).
You will see these kinds of titles in Silicon Valley's IT departments (these are different from Engineering) and in B2B sales related arms. Architect vs. dev is not normally how their core software engineering functions are organized. Everyone is a "software engineer" and everyone has a "design and architecture" competency to meet. It's more emphasized at senior levels, but it's always there.
I see these kinds of titles in product teams. I’m not disputing what you claim, that architects are primarily B2B in Silicon Valley. It’s just not true anywhere else. I’ve been a coder and engineer for a really long time. Now I’m an architect after finding management not to be my thing. I code. I build products. Any organization where the developers are also the architects are just flying by the seat of their pants and again, aren’t doing anything at scale (yet?). It’s extremely important to recognize the different areas of architecture. Enterprise and Solutions architecture is primarily B2B focused. Software Architecture and Systems Architecture can be B2C focused or product focused.
I’ve worked as a software engineer at places where everyone did architecture and it always ended up in a hairball mess that some senior/principal would recommend rewriting after 2 years.
> Any organization where the developers are also the architects are just flying by the seat of their pants and again, aren’t doing anything at scale (yet?).
How did you reach this conclusion? Both Amazon and Google don't structure their software engineering teams around an 'Architect' title.
I also searched the Google Carrera page for Architect positions and the great majority of them are customer facing roles with very few Architect positions assigned to specialized fields.
An architect who codes and builds products is what we would call a senior or staff engineer. We don't have any room on or teams for people who are solely devs/coders; even new grads are expected to grow into small scale design and architecture over time.
I don't know about Google, but where I work we have essentially one single technical line and people are expected to contribute to both development and making architectural decisions (and operations, etc, etc). Obviously, higher ranked staff are expected to make more architectural decisions vs. development, but it is a gradual growth process, and there are no positional distinctions.
I worked for google and never seen anyone with a title of architect or anyone who’d only do designs without getting their hands dirty. You must be looking at “cloud solutions architects” or whatever support-like roles
Why would the architect provide proof of concept code? Feels like a completely different role than large scale planning. This is something I would expect a manager or senior dev to assist a junior with.
> Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
As long as Conway's law holds, system design is going to be at the mercy of the person who designs the team organization. It doesn't really matter what title you give things.
I can tell you from experience that powering through a conways law mismatch doesnt yield good results. "communications overhead" feels abstract, so let me make it more concrete. When the org structure has no mapping to the software structure, the following bad things happen:
- there is no clear rule for who's job it is to fix a bug. You have to always inspect the code and guess.
- same issue for code review, and your code quality standard ends up beholden to the team with the tightest deadlines
- nobody knows which manager to escalate issues to, and the only effective strategy is to haul a bunch of PMs and senior engineers into a conf call to analyze and decide
- the testing and release strategy becomes one size fits all, and cannot make any assumptions about independent subcomponents of software -- you have to test and release the entire product in one go
But you just defined a person with a lot of more experience covering the bigger part of the design and the junior covering the smaller part, because of less experience.
So literally that's "hiding" an actual distinction that it's already there.
I agree, every dev should be able to design systems. With software development becoming a highly desirable career due to money, this is just not true and possible anymore.
The world is full of productive developers that are not skilled enough to design systems.
Designing systems is also incredibly hard, I have rarely seen (in my very limited sample) more than a few devs able to do it in a company.
So we have a category of people who put a lot of effort in becoming good at something, but we don't recognize them because "everybody should know that too". I think that's not positive.
People who want to grow in the design skills are free to do it and resources should be available, but the reality is already that people at higher levels tell you what to do. Isn't that what you would expect, somebody with more experience driving you?
Hardly. As a dev, architects around me have no problem with me getting more insight into their work, and letting me do some things that are 'outside of my lane'. On the flip side, I can say that something is above my paygrade if I want to stay in my lane. Your ramifications statement is more of a reflection on particular company culture.
On the whole, lanes are created by companies to create a promotion flow so that people can always keep trying to climb the ladder. We see more flat organizations today, but as an organization grows to become huge, hierarchy is almost necessary.
<3 Totally! To someone else's point, it is important that when I get involved in an alternate "lane", that I do not create more work for everyone and there is net value being created.
I would argue you acknowledge the point when you say you know when to avoid stepping outside of your lane ("above my paygrade").
lanes are created for specialization but when you do that it's a clear indicator that if you're not specialized in that area you should be deferring to those who are.
You wouldn't want a devops person telling you how to organize the controllers in your application, and that devops person wouldn't want you telling them how to organize the management of the pipelines. That's not culture, that's necessity due to scale.
The point being made is that should be put off as long as possible because when you put it into place suddenly the communication becomes a lot harder and it's not obvious it's always a benefit until there's real pain from not doing so.
---
I would not consider understanding the large context of things to be stepping outside your lane and I suspect most people wouldn't either.
It may surprise you that my very high scale employer in addition to having no architects also has no devops. It has instead an infrastructure group that ships a PaaS. Engineers click their own buttons on the PaaS. Only if things have gone very wrong would you end up talking to or having actions taken on your services by the PaaS owners.
Now obviously to design such a PaaS successfully you should know a thing or two about ops, but the outcome of that expertise is a reusable software platform, not per-product (or worse, per-release) labor.
I don't know what to tell you without de-anonymizing myself on the internet. It's a household name that you've probably used. Again, it is not that product engineers are "doing everything" - the road is paved by a very large platform org - but they just build the road, we have to drive on it.
I don't know any companies that use that word. Normally we are called software engineers. Amazon is the closest, with "software development engineer."
Nor do we use the word devops. Which is funny, because as I understand it, software engineers working on infrastructure automation in the same way that that we work on products are the closest thing to what "devops" envisions. The companies that are doing it don't use the word, and the companies that use the word are the most incredulous that anyone could be getting by without dev and (dev)ops as
substantively separate roles. It would be almost as if the people who most liked to talk about Agile were the ones most invested in very long range estimation and planning. Oh wait..
Whether it is lanes or 'roles and responsibilities' I think everyone benefits when they have a good sense of what they are signing up for. But, in the real world, sometimes you just can't get that kind of clarity. I will get responsibilities throughout the year that I clearly think "I don't really know how to do this"...and then I evaluate whether I can learn to do that thing well enough or if I should defer to somebody else.
In regards to culture and scale and lanes...what I was referring to was having a culture of people who are accepting when others ask if they can help, rather than telling those people to go mind their own business. If employees want to do what they are specialized in...totally fine...but if I am interested in a topic that someone else specializes in, I would like to have an opportunity to explore that(without being a PITA obviously).
The same could be said about the opposite.
No lanes are placed, the junior developer (or intermediate) will design a system that they are not qualified for and the problem will be discovered only at pull-request time, when a huge waste has already occurred. Or worse, the PR will be reviewed by a senior developer that's not experienced in system architecture and will get approved.
It's pretty common even in senior developers to not be experienced in system architecture, so the problem is not resolved with this approach (based again, uniquely on my experience. I didn't run any statistics)
What I would expect is what I actually have: you own the design and architecture of your own work, but you get feedback from more experienced people on your design proposals until they are in line with what the more experienced people would expect.
That's sounds preferable to having a few blessed folks do all the high level planning, completely detached from the lessons being learned from trying to implement those ideas.
Anything that harms those feedback loops is going to create pain.
Absolutely reasonable, however how do you know who are the more experienced folks?
In a small organization, that's doable, but when you get to 50 engineers, knowing who to ask is important.
The legwork, thinking, and writing come from the software engineers who are also going to be building and maintaining the thing. The review simply verifies that their plan is reasonable and perhaps suggests some angles they hadn't thought about. Reviewers are engaged in the same kind of work as the people going up for review - which is how they are able to make useful suggestions. After the meeting, some will be going back to coding, others to investigate the alerts they just got paged for, others to design/UXR workshops, others to go run sprint planning, etc.
I would expect an Architect team to deal directly with the product requirements, stakeholders, etc. and produce a design that they hand off to others to execute, while they move on to the next set of requirements. Architecture is part of the project lifecycle for everyone. This process is simply the architecture complement of code review, only slightly more formal.
What happens in case the design is actually highly incorrect, will the people restart the work from scratch?
Regarding the Architecture team, that's literally what caused problems in so many companies, that's the definition of how to have bad architects: the architects are so disengaged with the system that they cannot make recommendations.
An architect team should be highly integrated, participate in the coding aspect and only once something is shipped, moves on.
We schedule follow up meetings with smaller groups of interested parties about the areas of controversy or deficiency.
>An architect team should be highly integrated, participate in the coding aspect and only once something is shipped, moves on.
At that point, why are they a separate role or team? That sounds pretty close to architecture being one of the responsibilities of the main development team.
Someone who is functionally part of the development team but has more experience than others and some kind of supervisory capacity over technical direction is, in my world, a senior or staff engineer. Do you also have those? How are they different from architects?
How are you going to design database schemas if you don't know Postgres? Controllers and models if you don't know Rails? Components if you don't know React? I don't think design and usage are such separate skillsets.
Put another way, wouldn't you want the Postgres expert to design your Postgres schemas? Or the Rails expert to lay out your Rails application?
Again, I agree on that, my perception of architect is that it's a strongest software developer with a high specialty in designing systems.
It shouldn't be a person out of the software loop, it wouldn't make sense.
I never intended to create a low bar for the title, nor a separate role from "software developer". I'd expect the architect to still be a software developer able to contribute to the existing stack.
There are two stages: an architect defining the big picture and then a technical senior person reviewing the small picture. You describe the later. The big picture you cannot do like you describe.
I guess that depends how you define “big picture.” If you’re talking as a company what kind of infrastructure are we going to buy, what will be the storage platforms, ingress, message brokers, service mesh, sure. That is handled centrally by a platform group, which gives product teams a relatively constrained sandbox to play in. But specific products, features, and services are handled directly between product management and senior engineers, with a design doc reviewed by other senior engineers. No one has the title Architect and the same person generally owns architecture and implementation of the same functionality.
What if the feature/module designed has to interact with other parts of the system?
I saw that all the time, it's really hard to identify the right partitions in a system. It requires a lot of research and experience. Whenever one of those partitions is missed/overlooked, the software gets muddy and hard to work with around that area.
When brought to the extreme where very little boundaries were identified, this is common in CRUD apps where entities are confused with partitions the entire time, every change to the code ripples from the database all the way to the frontend.
In Rails world, this is a highly common problem: a change to database schema affects the UI directly, so every time something is changed in the schema, the following has to change:
- JSON renderers
- React or HTML views
- db schema
Thinking though those kinds of issues is bread-and-butter software engineering, we would not work with anyone past new intern/new-grad level who couldn't handle that.
That would be amazing, but we are faced with the reality that when a lot of people need to be hired, the bar can't be that high, or there is literally a barrier in how many people you can hire.
If you are in a smaller company, totally agree. Get way stronger foundational developers.
Architect, developer, tester, etc. are project roles that are inherent to software development.
The job title may say Staff Engineer, JavaScript Ninja, Principal Engineer, Senior Architect or any combination of the above, but in the end someone has to do the job.
When your company didn’t use the specific title of architect the role of architect was still distributed between different engineering levels. For example you had the role of X Engineer + Architect… this can work well if the project is mostly of technical nature and doesn’t require a lot of technical requirements elicitation, negotiation, synchronization and other people and organization focused work. Because if it does contain such work, you’ll be spending so much time in meetings that you won’t have time to write code any more.
Other companies assign the architect role to a specific person, or have multiple kinds of architects. An important duty of architects is alignment within and outside the organization and mediation between business and engineering.
In large-scale projects such work is especially important and can’t be done by someone with the title of developer, because they would probably get bored out of their mind and also not actually develop anything any more.
Neither of these approaches is wrong. I’ve worked with dev teams that expect to receive a design/spec that they will implement and at the other end of the spectrum with teams that love to design their own architecture. I will provide that spec for the former and I will make sure that their architecture remains cohesive with the platform architecture for the latter.
And it did happen to me that teams would insist on designing their own architecture and then create an over-engineered heap of code. Too bad for them, because they have to fix the bugs, but still ok for the platform, since all the interfaces and platform requirements mesh as they should.
Whatever level the architect is operating at, it’s crucial that they keep their technical skills sharp through building small prototypes, reading and analyzing code, code-review, etc. I can say from my own experience that whenever I neglected that, I started to become detached from the reality of the project and my decisions were poorer as my problem-solving approach became too abstract.
I like how an ex-boss of mine described it in his blog https://bradhenrickson.substack.com/p/establishing-an-archit... where architects "can either be advisory or decision making" and the latter is the old school ivory tower style of architect being alluded to here.
The prevailing sentiment here seems to me to be that we should do away with the title architect and allow senior developers to perform that function. I believe I saw a reference to Hohpe's "The Architect Elevator" as an example of the old school of thought in the comments here. Check out this guest blog of his https://martinfowler.com/articles/architect-elevator.html where he makes the case that senior developers should set the architecture for services and systems but that there is also the need (in larger organizations) for architects to advise on software portfolios and technology radars.
>So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams.
Sure, but at some point "spanning several teams" means you will not be close to the code anymore. You'll spend most of your time in meetings and it increasingly becomes difficult to stay close to the code. Doing "grunt work" is increasingly a waste of your time and a distraction from doing things that have impact at your level. At that point you end up telling people what to build and having them build it.
> I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
You're fooling yourself if you believe that that's not the norm already.
In fact, it boggles the mind how anyone believes that inexperienced people who are taking their first steps writing code and don't know what a software architecture is have as much say in the daily workings of a project than the grey bearded guys with a couple of decades of experience under their belt.
Of course I'm not just architecting everything by myself, and making decisions all day long. Every Software Engineer still makes a lot of architectural decisions on different levels - exactly how you described.
There's just a lot of other activities that Systems Engineer does (and still consults them with the team, but is at the end responsible for doing them). (all the bullet points in "The realization")
> I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
No I don't think so. Where in the article did the author said he is telling other people what to do? What he said is
> proposed software architecture consulted it with key developers, and made sure everyone is onboard
Being an architect, or staff engineer in modern term, means that the person will need to use his experience and expertise to help the team to find appropriate solutions, present to the team, list out pros and cons, let them pick, and drive the change with other stakeholders.
At certain companies that is the way it works. At other companies you're a working leader, but your knowledge is only leveraged for gap-filling. At a marginal amount of companies engineers are empowered to have reports and lead teams. These companies are fascinating places. I'm at one now and I don't intend on ever leaving.
A title is relevant for the person claiming it, and the people around recognizing it. If it helps him feel proud and accomplished, who are we to argue?
We - unaffected bystanders - don't really need to have opinions.
Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO (and from the linked Fridman interview he sounds like a really good one). What is it about tinkerers being put into these positions as a natural progression?
It always seemed to me that people with more of a managerial background would be better managers - is software/system development the only field where masters of their craft ultimately become directors?
Tinkerers are basically technology generalist, and have tinkered in probably everything. So they can map out solutions and have good ideas of what will work and won't.
People with managerial backgrounds have....what? Nothing really. They have to guess at any plan presented by technical people, are always suspicious they are being screwed over on estimations and real problem areas, and are unable to correctly identify when people are doing good work -- thus also being unable to set a healthy engineering culture for success. That's why most managers are demoralizing for engineers. They just don't get it.
It's interesting to note that alot of the most successful startups in SV are not from MBA's but engineers with masters or PhDs....it's not a coincidence I think. They have the practical experience to lead a real world venture to success.
Managers are good at managing departments like insurance claim processing or bad debt collections, which any human can learn in a few weeks fundamentally.
This is an overly cynical view of management. Distrust and "screwed by estimations" are signs that the dynamic needs tweaking, not that all dynamics are like this.
People with managerial backgrounds can become quite adept at helping you:
- Identify blindspots in your biases and behavior that keep you from peak performance
- Avoid working on stuff that's not valuable to your team
- Settle disputes within a group
- Motivate you and keep you engaged/fulfilled with your work
- Get unstuck with personal problems
This is not an exhaustive list and you don't have to have a 'managerial background' to master stuff like this. I am an engineer who has had to learn management as a startup founder. I used to distrust the whole management thing but that kept me from growing as a teammate. Management is not only useful in 'non technical' jobs, it's useful in all human endeavors it's why we study it so much and why it has so much leverage.
A cynical or experienced view? I'm in management and have seen good managers and bad, but the bad ones are quite a few. To address your points, managers with a non technical background:
- Cannot identify blind spots that keep you at peak performance, only another engineer with more experience (tech lead or senior) can do so. They can only identify behaviors that make them look bad or are inconvenient when viewed from the point of view of their peers -- other managers.
- A non technical manager has literally no idea what is valuable to delivering complex technical work. They can only guess, and often guess badly. Again only a senior or tech lead with experience could do this.
- How could a non technical manager motivate any engineer, without an understanding of their difficulties, problems and ways to solve these problems practically? I just don't buy it. "Let's do overtime on the weekend guys..."
- Managers should not be involved with people's personal problems. I've met so many managers that are extroverted and managing sensitive introverted teams, that all they ultimately do is the equivalent of hammer on the aquarium glass. Remember that sign in the pet store: "Don't tap on the glass"? It's true of technical teams that are of a totally different temperament than managers.
> A cynical or experienced view? I'm in management and have seen good managers and bad, but the bad ones are quite a few. To address your points, managers with a non technical background:
Thanks for the detailed breakdown and no name calling :D
> Cannot identify blind spots that keep you at peak performance, only another engineer with more experience (tech lead or senior) can do so. They can only identify behaviors that make them look bad or are inconvenient when viewed from the point of view of their peers -- other managers.
Given you have different levels of skill in the team, a good manager would convince you and the other teammate to help each other out with the learning. Lubricating these interactions given everyone has responsibilities is not always trivial. I'm less experienced on the politics but I believe what you're saying about politics distorting incentives. I'm conveniently side-stepping this issue bc it applies to all positions in a corporate structure.
> A non technical manager has literally no idea what is valuable to delivering complex technical work. They can only guess, and often guess badly. Again only a senior or tech lead with experience could do this.
A non technical manager can know very well what's valuable to the product/company. That they don't believe their team on the value of a specific piece of technical work to enable that seems like something else is at play here (lack of trust).
> How could a non technical manager motivate any engineer, without an understanding of their difficulties, problems and ways to solve these problems practically? I just don't buy it. "Let's do overtime on the weekend guys..."
By reminding/reframing/convincing re:impact their work has on their team, personal growth, customers, society or personal preferences. Doesn't have to be only technical; they can help you deal with any self-inflicted discomfort regardless of the subject matter.
> Managers should not be involved with people's personal problems. I've met so many managers that are extroverted and managing sensitive introverted teams, that all they ultimately do is the equivalent of hammer on the aquarium glass. Remember that sign in the pet store: "Don't tap on the glass"? It's true of technical teams that are of a totally different temperament than managers.
Agree to disagree. I've had great conversations with peers when/if we're open to talking about non-work stuff - both ways not just me 'giving advice'. It's not binary and depends on the relationship. A skilled manager can care for reports beyond work, create genuine bonds and be respectful when they have not been given an opening to engage in these subjects.
> A non technical manager can know very well what's valuable to the product/company. That they don't believe their team on the value of a specific piece of technical work to enable that seems like something else is at play here (lack of trust).
Trust is a difficult commodity to build, a lot of company culture issues stem from lack of trust. It's particularly key to the manager/team relationship.
When you have a non-technical manager directly over technical teams it's particularly difficult to build trust. People, emotionally, want to have someone really understand them. Someone who doesn't, at a fundamental level, understand the actual work you're doing is going to be at a disadvantage as the work is crux of the purpose of the interaction.
Not to say that it is impossible, someone with well above average people reading and listening skills can still build that trust and get it. But it's definitely going to be more difficult than someone who really knows the turf.
As a startup founder you are automatically at the top of the company food-chain.
Managers in the middle of the food-chain are all about power-struggle and social games they play to out-compete each other.
That is especially the case for non-technical managers who don’t have a sense for the underlying technical challenges. They have all their bandwidth available for political positioning and social games.
It would be great if all managers where “serving the team” in the sense of your bullet points. But alas many don’t see it that way.
IMHO many startups are successful because they have a technical person at the top. Who is capable to understand, evaluate and positively reward technical work within their organization.
Part of the reason for that - especially in tech - is we keep hiring people utterly unsuited to manage others. Pretty much every manager I've had is a software engineer in the past. But, only one manager was a true manager who helped me with the technical stuff, career and personal growth. The rest were all great people but totally inadequate as a manager.
In tech and pretty much every company I've worked at, you need to get into a managerial position to be able to have a say in what get built and how much you get paid. Programming fatigue and frustration with being told what to do also sets in after a while. The two together convinces people who are terrible with their people management skills to chase a manager-path career.
Essentially, you're diagnosing the symptom to be the cause.
Just figuring out what is important / not important to the company alone is enormously important.
I’ve seen multiple times my careers were dozens of people no will work for months on something that just isn’t important.
Being able to put things in terms like this feature will cost us $1 million in developer time. We can expect a return of $50,000 over the life of the product. Or vice versa.
I feel that turning to leadership (tech leadership, but also pure engineering management) can be a natural "tinkerer" progression. If you want bigger impact, you turn to more "abstraction", and use tooling and patterns to fill out the details. In this case, you can consider that your role is to give broad directions. They are very important directions, because they shape the "design space" of the problem you are trying to solve.
After that, you have a set of "programming tools" (I don't want it to sound mechanistic, because it is the opposite of that), which are your team and reports, and they will be able to fill in the details (and the details here can be significant pieces of design and architecture by themselves). And your role is to choose the right tools, and allow them to work to their full potential. This means clearing obstacles, clear communication, technical help at times, mentorship, aligning expectations and giving them clear paths for growth.
All these things can be considered engineering at a larger scale. You want to get a really big system shipped and productive? This is the work, these are the skills you need.
> What is it about tinkerers being put into these positions as a natural progression?
Because those tinkerers get to a point where they want to be the ones making the decisions, controlling the culture, technology, and direction of the company. Without position you have no power and without power you can't affect change.
Software isn't the only field where practitioners get promoted to management, but it is one of the fields where technically incompetent managers and executives will kill a company with unsound decisions.
> What is it about tinkerers being put into these positions as a natural progression?
I don't know enough to say whether this is a common pattern or not, but if it is, it could perhaps be that tinkerers tend to gather a vast breadth of knowledge that can be very useful when making strategic decisions. They reach the point where they know enough to understand what questions to ask on many topics, even if they are an expert in only a few (or none) of them.
I'm not the worlds greatest manager - but after you've worked with hundreds or thousands of people and seen seen hundreds of projects come and go you start to develop some insight about how things go and how they can go bad.
I don't know how else you pick up the skills to be an effective technical manager
> It always seemed to me that people with more of a managerial background would be better managers
The distinction dates back to the industrial revolution, where you had manufacturing line workers and line managers. A manager would usually be an owner's relative or someone they trusted – more loyal to the company than the unions.
This distinction perpetuated well into current age, just notice how much implicit bias there is about "programmers don't have people skills" to keep workers accepting a career ceiling. Most managers aren't skilled either, and not respected by the workers due to it, but companies won't keep them from managing because they need someone to be responsible for plans, estimates, OKRs, etc.
I guess in software it's "more" common due to survivorship bias – the business of software is so messed up and people have so little idea how to manage it, that companies without experienced leaders have a smaller chance. Strong companies and teams in the field have leaders with enough hands-on experience to have natural authority.
> Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO
I didn't get the impression that he's your typical CTO and completely hands off with code and development. If anything, it sounded like he's still very much in the trenches but has learned how to delegate work well and pick which problems are worth the time investment.
When it came to his work on Oculus and the work he's about to do in the field of AGI it sounded like he'll definitely be making direct contributions. It's entirely possible that I misunderstood his stance throughout the interview though and he's a more hands off guy now.
In my case my impact plummeted. Sometimes going in to management means having your hands tied -- you are the ones allowing higher ups to scale their impact. It was awful.
I'd love to be able to find folks that can do what I do -- it's probably our #1 issue holding us back -- and direct them to meet some business and technical goals, but I've yet to work at an organization that supports that mode of management.
And here your organization has failed you. In our org we split engineering career at a certain point into three careers: engineering, people and architects. Up to the VP level their grading run in parallel. The architects are optimized for tech and business impact while the people managers are good in managing people. and yes engineering tracks and architecture track optimize certain skills but are not necessarily a hard separation
Who's programming 12 hours a day. He never ever said in that interview, that he's going away from programming. He's been "CTO" in the early days too. You misunderstood the whole conversation if you think he's turning into an "architect" only.
Carmack did end up moving from Meta VR (although he still works there one day a week). I presume it is because he was not doing enough engineering work. I used to read all of his posts internally and he really did seem to be an amazing CTO.
It's not they are tinkerers necessarily - it's that they want their impact to scale larger than what they themselves can create. Solo coder -> tech lead -> CTO is like going from single CPU -> multiple CPU -> distributed systems.
> no matter how many smart people you hire, and how many nice frameworks are used - if the organization and processes are not in order, you will keep losing time and money.
I disagree.
In my experience hiring the right people limits the need for organization and process. Obviously you can't eliminate those completely but if you hire the right people your org will generally succeed in spite of a lack of organization and process.
I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.
> Obviously you can't eliminate those completely but if you hire the right people your org will generally succeed in spite of a lack of organization and process.
This is true. I like Marty Cagan's take on it in his article "Scaling with Process vs. People"[1]
Exactly. There is a difference between "right people" and "smart people". The right people are usually smart but smart people aren't always right for the team. If you have the right people you don't much process or organization imposed from top. They will organize themselves.
Yes but if you hire a lot of smart people and then have a process where they cannot make any decisions but instead someone from sales decides every little thing you will still not get anywhere regardless of how many smart people you have.
But I agree if you have a lot of smart people you can go for a lean process, and let everyone mostly guide themselves. But you do have to provide that freedom.
You are not wrong, my experience comes from what I think might be a niche. A startup doing both hardware and software, and does so with an offshore team.
For 'simple' systems, Systems Engineer is not needed. Typically Software Engineers share architectural responsibilities.
> I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.
Whenever I hear the title "Architect" I have to think about certain segments of the IT department at my company and also the participants at my scrum master training. Both were full of "senior architects" that were good at talking but were so out of touch with reality that they couldn't contribute anything real. It's pretty easy to architect a system if you ignore the little, annoying details that make an implementation difficult. And to know what's hard you need hands-on implementation experience. These guys get away with their nonsense because often the implementation is outsourced to developers who just implement what they are told to do without asking questions. Never mind that the bad design decisions make the system much slower than it has to be and takes way more effort to implement.
I am an architect. Proud of my profession. I code only for fun. I do solution architecture in a scope with 100s of impacted developers. I highly suspect that dozens of them are describing us exactly like you do. However, once they challenge us and we elaborate the problem to them, their solution usually falls apart and they align with our solution. Why: because in that scale of complexity we operate in not every engineer can maintain the necessary knowledge.
I highly respect the approach of architecture in the team/agile environment and seniority in engineering. But at a given scale, someone is there to orchestrate all of it and does the overall plan. That person is a solution architect.
How deeply do you go in details with that kind of scope? Do you regularly discuss architecture decision with all these developers, so they're all aware of impact? Do they actually understand impact? How do you actually verify that created solutions comply to architecture? Do you support yourself with solution architects?
Personally I noticed, that there's always something to analyze more in-depth, something to document better, yet another thing to teach others, however other engineers that I work with don't really care about creating coherent view of solutions they create and they don't care about providing clear description/instructions/guidelines for their solutions.
So do you depend on developers to prepare complete solutions and only verify, if they do everything that you would expect or do you actually have to guide them through details and support them in design, so you can together create something that works according to architecture?
That is the art. Finding the right degree of detail, communicate and verify it correctly. Adapt to changes and refinements of the engineers.
We expect engineers to refine and review the architecture. They are engineers. However, they are busy delivering something while the architects talks with business and define the solution for the next iteration. Solution design takes time, soft skills and expertise. You cannot do that and deliver something.
One of our best solution architects is not capable of programming. He holds a degree on physics. But he is good with finding solutions, managing information and also excellent in communication.
As someone who has held both roles I agree with this point, though the same vice versa can be said about operating with certain complexity. Developers need to deal with a considerable scale of complexity too, with regards to various technologies and practices, that architects can't be expected to be proficient in. The key to success is your architects and developers having an affinity for each others role and responsibilities, and management ensuring appropriate conduits to keep the two engaged.
I've been a solution architect embedded with a team of 4 devs, but was constantly dragged into meetings by more senior architects collaborating on the wider platform we were re-writing. That collaboration was important, but it consumed too much time. At the time I lost connectivity with the developers and they were to a degree twiddling their thumbs while a design was nailed out.
Since then I've come to see the benefit of design and architecture stages including the actual developers as much as possible/as much as they want to be included. By that I mean specifically things like getting developers to pair with you on proof of concept work, where you need some form of discovery that will lead to a design decision; pairing with developers on pieces of analysis and documentation around the current state of the system/application, or business/user workflows (in teams where there has been no business analyst).
With conceptual/high level design you might not need much developer input, but when you get into the detailed design/logical design dev input can be crucial (especially if you don't know a lot about the current state of a system). When I'm putting together a design I like to go as wide as possible first with the scope, and then fill out the scale of each area with details after further analysis. Getting your developers (if they're available), to help fill in some of those gaps can be beneficial.
Getting that early engagement with the actual developers is key, and in my opinion taking a more agile and scrum approach to the architecture is much better than doing all the big design up front with zero developer input and then throwing it over the fence. That said, your development team needs to be geared for this, and some might not be, if all devs are busy delivering feature A while the architect looks ahead at what's next. But if I can get time from at least 1 developer who has a good handle on the current code and tech stack, then that's definitely something I capitalize on.
Gotta love those over-engineered projects where to add a text field to an existing form requires editing more than 15 files. So may layers of indirection that debugging is a nightmare and callstacks look like the full lorem ipsum.
The problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. - Joe Armstrong
I prevent cross-site scripting, I monitor for DDoS attacks, emergency database rollbacks, and faulty transaction handlings. The Internet heard of it? Transfers half a petabyte of data every minute. Do you have any idea how that happens? All those YouPorn ones and zeroes streaming directly to your shitty, little smart phone day after day? Every dipshit who shits his pants if he can't get the new dubstep Skrillex remix in under 12 seconds? It's not magic, it's talent and sweat. People like me, ensuring your packets get delivered, un-sniffed. So what do I do? I make sure that one bad config on one key component doesn't bankrupt the entire fucking company. That's what the fuck I do.
Does anyone know of any good books for getting better at System Design that isn't web focused? Every time I look for "System Design" the books are about creating a highly distributed web app with SQL or NoSQL database, load balancers, and other web centric stuff.
I've spent most of my career fixing bugs or improving existing apps/systems in C++. Every time I interview somewhere they want me to design a (usually offline) program or service and I don't know how best to do that. At this rate I'll be stuck at Senior Engineer for the rest of my career. (I'm already 14 years in).
I learned a lot by writing "prototypes." If you look at system design and architecture from afar, they're all very simple patterns that can be written up in a self contained python or javascript program:
* RPC = function call
* API design = function signature
* data schema = simple structs
* message queue = list
* load balancer = simple facade object
* database = hash table / in memory SQLite
Once you have a rough sketch in place, you can run your prototype and test different behaviours.
It is a bit "tricky" to tackle asynchronous and concurrency, which you should do from the start. I prefer to do everything based on callbacks, and run my simulation from a static input list of events, ticking every component with a tick() method. This way I can generate different scenarios, save them to JSON, replay them.
What a wonderful comment. I've bookmarked this page specifically to refer back to your list of simplifications and would encourage you to please begin on your book.
I've used probability distributions to generate the incoming events, but the simulation is always deterministic. In the netflix case for example, I would generate 40% of users that would be playing things around 8am with a normal distribution, and 40% that would play around 8pm, and then 20% that play at random times of the day, etc...
I also played with pymc3 where you can just place plain python inside your probabilistic model and MCMC the heck out of it. It's slow but you can use some real world input and see what distributions arise at each point of the graph.
You may wish to take a look at the book, Righting Software, by Juval Löwy. It is not specific to any particular application or language, but can be generally applied to different projects and systems.
What insights did you get out of that? I was watching the video version and gave up because it was mostly fluff and self-promotion. Couldn’t even remember what the core idea was that he kept repeating.
Often, decomposition is based on areas of domain or functionality. There are some downsides to doing it this way. The method in the book proposes it be based on areas of volatility instead. As an example, imagine writing a piece of software for FedEx shipping. What would change to use that same software for UPS instead? That may identify some areas of volatility. If you can plan that something might change, you can limit the ripple effect of those changes in future. Just an example of one benefit of doing it this way. The book explains some of how to approach designing this way.
Another point: Architecture is not just about designing the software. It is also designing the project to build that software. This involves things like critical paths, staffing, risk management, etc. Understanding these is important if you want to be able to deliver on time and on budget.
Thanks. Regarding the first point, I remember that he was very adamant that functional decomposition was a catastrophe, but I didn’t reach the part where he presented the solution, which based on your comment seems to be decomposing on volatility.
I suppose it is another tool in the box, however in the video it was about to be introduced as nothing less than a magic bullet.
Regarding the second point, I’m not sure I agree. An architect could get involved in staffing decisions and any project has a list of risks, for example I’ve maintained such lists as part of the architecture description. But I would define these these are side-tasks and not as core tasks.
I suggest learning an architecture framework which guides you on what topics to address as part of your architecture - such as 4+1, Viewpoints and perspectives or arc42 which I’ve learned of recently and seems to resemble the former. You’ll then mix domain-specific details into that framework.
Systems design is for me quintessentially embedded system architecture for a microcontroller, a SoC or combination. It could be a smartphone or console, or medical device, etc.
This is a very broad domain… in my case I work exclusively on non safety-critical SW, so I’ve focused on learning Linux internals and system programming which I combine with an architecture framework. Typically I design middleware and daemons which work together to cover a specific area of a product. I work with different kinds of specialists (HW, kernel, performance, etc) and a lead SW architect for each specific platform. The system architect is a different person that oversees both SW and HW and requires skills in both.
For Linux I cand recommend the classics: Stevens, Kerrisk, Love. I found Chris Simmonds’ book to be a good general intro to embedded programming; it has much info which also helps with embedded system design. I’ve also picked up ideas from Making Embedded Systems by White or Embedded Systems Architecture by Noergaard.
Application design refers to individual applications running on an OS. These are the typical MVC or layered architecture styles. The official OS UI framework will likely define a recommended architecture while headless services are quite flexible and support several architectural styles.
Vol 1 of the Pattern Oriented Software architecture is a good intro. I have Software Architecture Patterns by Richards on my reading list, since it covers more recent styles like microservices.
Because C++ has been used in so many different domains and for so many decades(!), there are an overwhelming variety of architectures that have been deployed to great success in their own problem space. So I think you'll have a hard time finding resources until you narrow down what sort of things you'd like to be designing.
You can spend years learning all about great design patterns that make sense for an IoT or embedded system but have completely backwards ideas about how to organize code for a game that needs to process 1000 unit movements per frame.
But once you figure out some narrower domain that you like, you should be able to find books about it , (if you're good at reading code) can start studying open source approaches to it, and can start prototyping small projects to gain some hands-on experience and muscle memory.
Perhaps you could narrow things down a bit by trying to figure out what key characteristics need to drive your design. I'm in aerospace. Some of the systems I stare at are designed pretty much exclusively based on safety considerations. Others are almost completely devoid of such considerations and more care is given to maintainability due to expected high rate of change, or requirements on throughput etc. Security is another joker with a high potential of impact on design. Also figure out what you consider to be "system design" as opposed to "software design".
I’m currently a principal engineer, but I’ve had architect titles in the past.
One worry I’ve had with taking jobs with that title is that the salary range seems significantly lower on average. Even if the specific job paid well, I worry about pigeonholing myself into those roles.
In my opinion "architect" should probably really be the most senior role of people that are doing infrastructure. It can be applied to more specific software engineering roles but it just seems more appropriate considering the infrastructure is what everyone is using as a tool to run their various projects.
Most folks define their title by what they are paid to do. It follows then that if you are not paid to do anything, then you don't have the title, right? Well, if that's not quite right, then you haven't really become anything but rather you are someone who is mucking about with systems and happen to be being paid for it.
I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
I prefer how my company career ladder, where there are no architect titles. Each level is 'fractal', containing almost the same responsibilities but at different scales and scope.
So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams. This way architecture is everyone's responsibility, and people have a natural path to grow their skills.