Hacker News new | past | comments | ask | show | jobs | submit login
How I operated as a Staff engineer at Heroku (2020) (amyunger.com)
173 points by craigkerstiens on March 31, 2022 | hide | past | favorite | 76 comments



In Amy's list of different staff engineer archetypes and their responsibilities, she puts a lot of emphasis on "what you might be blamed for," which I actually really like as a way of defining what someone's "actual job" is.

Sure, that could be toxic if you take it totally at face value. "Blame" has some nasty connotations.

But let's face it: Staff engineer at Heroku/Salesforce is probably, what... $400k TC? At the very least? You are supposed to be someone who is solving seriously big problems. I think the most coherent way to write a job description is defining what those problems are, and "things you will be blamed for if they do not go right" is a very pragmatic way of defining those problems.

Also:

> First and foremost, I am successful if the folks I work with understand how business decisions tie into their day-to-day work.

Love this. If this were the literal only sentence that a person knew about technical leadership in software engineering, they'd be instantly top quartile of technical leaders.


I was in a meeting once decades ago when the company was going through its annual deck chair rearranging. A woman we didn't know came in and said she was going to watch our meeting for... #reasons. My manager asked over and over what she did and finally after 3-4 rounds of CorporateBizSpeak, he finally just said, "Ok, let's try this. What at <company> would have to fail for you to be fired?"


And what was her answer?


If ever there was a live experience that indicated what "gobsmacked" meant, it was that. She basically couldn't answer. The only thing she could blurt out was "<the person who she reported to that we all knew and was fairly high up> said I should be here".


Nothing, she continued filming them while saying “it’s just a movie that I’m making, no need to be upset”.


I dislike the blame bit because success in creative jobs are to a large extent about doing things nobody else thought of. (And by definition, of course, if you fail to do something nobody else thought of, nobody would blame you for it either.)


> if you fail to do something nobody else thought of, nobody would blame you for it either.

That's a great point. It's definitely not a perfect lens. And I do think that the power of a "Staff Engineer" title or similar is giving really talented, creative engineers the organizational capital to go do wack shit, because hey -- sometimes that wack shit is GMail or JavaScript!

But at the end of the day, we're all getting paid to be here; I think having a clear understanding from the start of "what could I do wrong enough that my job is at stake?" is very, very valuable.

Later in the article, Amy also talks about how the "blame" can come from below, too:

> If the ICs that report to my manager end up feeling like “I told you so” or “We knew this was a bad idea” and that wasn’t surfaced for a discussion, that’s on me.

And I think that's a really healthy way to approach leadership. It is -- and should be! -- a very serious responsibility to be making decisions that affect lots of other people's jobs.


> But at the end of the day, we're all getting paid to be here; I think having a clear understanding from the start of "what could I do wrong enough that my job is at stake?" is very, very valuable.

That might well be. I have a vague, uncomfortable feeling that there's a deeper understanding here that simply eludes me. I feel like maybe five years from now with some more experience I'll be able to read this exchange and facepalm at my naivete!


JavaScript was invented by a new hire as a warmup project. It wasn't organizational capital.


I'd prefer responsible or accountable for to blame, personally, but blame has a way of really getting you to zero in on the important-to-the-business aspect. The stuff people will be mad about if it fails!

So if you come up with an idea nobody else had thought of, and you convince the company to spend millions or more on it in engineer-time-dollars, and it ends up being a total waste: yeah, you might have some questions to answer. There's a difference between doing something nobody else thought of and doing the right thing nobody else thought of, and the more you're responsible for in a company, the more that falls on you if you either keep getting it wrong or get it wrong once in a sufficiently-spectacular fashion. (There are nuances here of course depending on how you sold it to people, just how much everyone was aware of the risk of failure, etc... but it's important to remember freedom to experiment and take risks is different than freedom to do whatever you want.)

(There are certainly places where cushy titles can be sinecures, but whether or not that's the case is going to vary greatly depending on the company and their positioning and needs.)


At staff, or even senior levels, you only have a limited time after joining a company to identify opportunities and initiate projects. Once your projects go live, you're responsible for them. If your initiatives keep failing and getting scrapped, you'd be out of a job soon.


Eek pretty bad take on this. The reality is blame cultures are there such that you can be used as a scapegoat. It's nothing to do with defining problems it's there to have someone to fire when something goes wrong.

In terms of the $400k TC you'll be talking shares, that need to vest, and bonuses and suchlike included in that. Nice way to reduce that is to blame you for problems on a project that were out of your control.

I've worked in companies where the Project Manager always put too short a timescale on any project. This was pushed onto them by an exec but rather than speak to the devs get a reasonable time and try to cut scope or extend time they just put it straight back onto the devs. Often they'll cut some time which I always found weird. The reason that this happens is that these PM's know that there is no way they can ever meet these expectations and so rather than try they make sure it'll be late such that they can blame the devs. That way they don't need to actually manage the project and never get reprimanded as it's always someone elses problem.


Maybe replace "blame" with "responsible for", it has the same meaning but has less finger pointing/negative connotation associated with it.


Something about these articles reads so self congratulatory and aggrandizing...

A lot of talk about how the author lives on the cross section of business, management, technical leadership and architecture - but literally zero mention of any measurable numbers or quantifiable impact. Little to no mention of honest disadvantages and limitations of this role.

I also find it a bit strange how literally every single article on https://amyunger.com/ reminds us that the author is a staff engineer, cares deeply about staff positions, very knowledgeable about the staff engineering role, and that it's oh-so-difficult to achieve this prestigious title...


Once you have a team, it's pretty rare to have a "quantifiable" impact in the way that satisfies your engineer side. By design, your time is pulled to the problems that can't be solved quickly/definitively/numerically, or they'd have been solved by the other members of your team before you. And to the extent that solid metrics can be cited, you want them to be your team's accomplishments. The lead who takes credit for everything is a jerk.

I'm not saying your take is wrong here (some people seem to love this kind of management porn stuff a bit too much and roll around in it), but it is true that as you get more senior, you're expected to do things that don't have that kind of quick payoff.


Lead Engineer and primary designer or implementor of a service that handles 1M queries per second with 100ms 99%ile latency, globally distributed for 100M users.


The linked “StaffEng” stuff is fairly ridiculous:

“In literature, recurring character patterns are called archetypes, such as the "hero" or the "trickster," and the archetype term is helpful for labeling these frequent variants of Staff-plus engineers.”

How does anyone take this seriously?


Not sure. Having been a staff and principal engineer at a few places, my view is a staff engineer is primarily a reviewer and documenter: code reviews, writing and reviewing design documents, architectural and coding standards enforcer, etc. Mostly you are shepherding things along that are too technical to be dealt with by the engineering "people" management. Maybe a couple days a week you get to do some interesting coding.


That sounds like a job for someone who is washing out technically but has some wisdom and institutional knowledge to share.


It depends on the org.


They've read Joseph Campbell and appreciate the analogy.


Jung, surely?


> Something about these articles reads so self congratulatory and aggrandizing...

That's a requirement for staff engineering. This is how you graduate to staff. If you don't understand, there's a couple of books you can buy and educate yourself. Also there's a staff engineering podcast where staff engineers are invited to talk staff engineering and how to staff engineer effectively.

See my previous comment about this:

https://news.ycombinator.com/item?id=30049319


This guy Staffs


This is the language required to get that staff title and salary. It’s all part of the game. It’s all just a game, after all.


Quantifying impact in many of these roles is something that's incredibly difficult, and if you actually do it, most people will say "wow, that's an incredibly arbitrary and abstract way of quantifying impact that feels suspect."

The only thing you can reliably point to is "During my tenure, the company saw X metric go up or down in this area." But anyone in management who claims "I made metric X go up/down" is probably selling snake oil. Are you sure that it was you? Maybe it was a parallel change made by someone else. Maybe it was someone you hired, while you did literally nothing else but hire them. Even if you are sure, how in the world is anyone else supposed to verify that?

If someone can point to actually reliable ways to quantify impact of leaders/managers, please I would love to be shown to be misinformed.


> most people will say "wow, that's an incredibly arbitrary and abstract way of quantifying impact that feels suspect."

As someone who's said that, you've given me reason to ruminate on whether I was being too cynical and not empathetic enough.


Quantifying individual engineer output - lines of code, etc - has many well-known problems.

Quantifying impact of mentorship, technical leadership, management inherits all those problems and then adds new confounding factors in on top of it!

Quantifying business metrics changing is usually going to be misleading at best short of for, say, a CTO, and even then it's gonna also depend on product, sales, marketing, etc.


I like where you went with this.

1. The corporation itself is built on business metrics though. Everyone quantifies this way, it’s table stakes to portray oneself as responsible for a positive change in business metrics. You still must pretend they matter even if you know better.

2. What do you think about quantifying by the free market? Eg. the annual revenue contribution of a new SaaS SKU that you spearheaded.


Listen, jack. Delivering mind numbing corporate software is important.


Now you listen here, Mac. We're delivering valuable designs and solutions at cloud scale over here, so you take your FUD and envy elsewhere.


I've only read this one, but did in a spirit of "happy to be there." I did get a bit of a whiff of "drank deeply of the cool-aid" vibe, however. For a blog meant primarily to be read by your team/pr-vehicle it might make sense.


Even better, she just declared herself Staff because she wanted too.

> I was incredibly lucky to spend 5 amazing years at Heroku. By the end of my time, I was operating in a Staff capacity, although I’m honestly completely unclear which titles at Salesforce actually map to Staff.

> I live-snarked much of the process on Twitter

oh, that explains it. An "influencer" and "thought leader".


[flagged]


Not the person you're insulting, but (1) I don't think there's anything creepy about reading articles that someone publishes and (2) there are four articles total.


https://amyunger.com/blog/2020/09/10/staff-engineer-at-herok...

Based on how time is being spent that sounds an awful lot like an EM role to me?

Like EM responsibilities minus actual humans reporting to you?


Yea, that's essential what high level technical people are. They're making decisions on generalized details with enough confidence that they details can be filled in by lower level engineers.


Yes. Perhaps I’m getting old and cynical, but “staff engineer” is an invention by companies that are no longer growing quickly enough to absorb all of their most promising employees into proper management roles. Instead of having the power of enabling continued employment, you have to “influence” others to do your work.


This is ridiculous - the most promising engineers are not always the best people managers.


A promising engineer rises through the technical track by showing impact. You show impact by getting others to sign on to your vision. You request a budget. You engage with a lot of stakeholders through meetings. Sounds a lot like what we call a "people manager" but without all of the tools available to one.

Maybe a better way of framing it is people managers have too much power. In an alternate reality, "staff engineers" hire and fire and have budget signature powers, and "people managers" are more like HR-plus who manage interpersonal conflicts and deliver performance evaluations. The status quo is a remnant of how society has historically undervalued and infantilized technical workers.


Yes and no. There is a part where you're responsible for the direction of the team but the real trick is knowing when to step away and let the teams executing own it and discuss with management what metrics are needed to track success.

There is also another part where you are free to work on activities that are 12-24 months out. So, prototyping new ideas, setting up the pilots and mentoring sr. s/w engg. resources to be able to execute on them.

The little secret no one mentions is that most of the time you're not needed. If you are then you're too much in the weeds and cannot be an effective staff engineer.


>> The little secret no one mentions is that most of the time you're not needed.

That should be true of every individual.

If you mean role (i.e., "what if we had no staff engineers (or equivalents) at all?"), ye-es, but only for a time, else the company will likely be, at best, inefficient.

Every higher level role on the team, be it a staff eng, a tech lead, an EM, etc, is a multiplier role; they should behave as basically a glorified plate spinner. When the plates are all spinning they can step away and you'll never notice their absence. When the plates are wobbly, you should feel their presence more. The ideal workflow is a series of barely perceptible touches to add a little more inertia across a variety of plates.


I refuse to take anything besides software engineer and senior software engineer seriously. Even that varies by a huge amount at different companies.

I don't care if your company called you an Elite Ascended Architect Principle Rockstar. Tell me some concrete things you've actually done. Sitting in meetings with the C-Suite might feel good, but frankly doesn't impress me.


Well said. The original post makes it clear how Heroku and Salesforce at large haven't shipped anything of note for 10+ years.


Valuable Staff Engineering is basically the bridge between Product and the rest of Engineering. Product decides what the organization ought to build (based on discussions with customers etc.), while Staff Engineers break up the problem into buildable chunks and come up with a detailed plan for Engineering to execute, including a true understanding of costs and time estimates. Over the life of the plan (from paper to executed reality), you serve as the point of contact for any questions from junior to senior level engineers on how to execute, as well as the point of contact for Product to make changes to the plan as the business changes according to the market.

You are distinguished from Architects, who are specialists in writing said plans, but do not themselves spend much time coordinating between Product and Engineering. If the plan is especially complicated, a Staff Engineer may serve as the technical manager for Architects, who build out and maintain the more complicated documentation, but typically Staff Engineers will write the plan themselves. Staff Engineers will also often write the code for prototypes or proofs of concept, to help illustrate the plan to Product.

You are distinguished from Project Managers and Engineering Managers, because your role does not require actually running after people and ensuring that the work is getting done on-time. You are responsible for your estimates, and you are held accountable if they are wildly off, so it is important for you to be well familiar with the Engineering teams and what they are largely capable of, but you're not responsible for the actual execution or for the professional development of the engineers managed by Engineering Managers.

Good Staff Engineers are rare because they need to have a firm grasp on the company's code, systems, Engineering teams, and the company's business needs in order to be successful.


This isn't really directly related, but close enough that I feel it isn't off topic: what are the 3-5 best books a new staff engineer should read? I realize that reading 5 books on the subject generally puts you 4-5 books ahead of just about everybody else, but I like books.

Also: this needs (2020) in the title.


Either The New Economics or Out of the Crisis or both, by Deming.

I could recommend twenty more books that I think are essential to be a good leader (in various areas like ownership, communication, continuous improvement, planning, innovation, anthropology, visual control, calibrated estimation, etc.), but Deming stands way above everything else.


Well ... Go on then! What are the other 20 that you recommend, sorted by your personal favourites?


Some authors take multiple books to fully flesh out the ideas outlined in the first, so there are some doubles recommended here.

I also found it impossible to rank books, so instead I took roughly the top 20, grouped them into themes, and then ranked the themes based on how important I think they are.

1. Deming is the primary writer of the type of management we need to be competitive: Deming, The New Economics & Out of the Crisis.

2. There is a wide variety of ways for humans to organise. Nobody is forcing the default capitalist bureaucracy on you, so be creative: Graeber, Bullshit Jobs & Dawn of Everything & The Democracy Project.

3. How people are motivated: Pink, Drive.

4. How to create a high-trust, less authoritarian, open, collaborative environment within a traditional hierarchy: Marquet, Leadership is Language.

5. Communication: Voss, Never Split the Difference. Faber, How to Talk So Kids Will Listen.

6. How to help organisations who don't want to admit what their problem is: Weinberg, Secrets of Consulting.

7. How to structure organisations and processes to deliver maximum value: Reinertsen, Principles of Product Development Flow. Ward, Lean Product and Process Development. Womack, Machine That Changed the World.

8. How to innovate cheaply: Westrum, Sidewinder.

9. How to estimate things properly: Hubbard, How to Measure Anything. Tetlock, Superforecasting. Savage, The Flaw of Averages.

10. How to train people to be experts at things: Hoffman, Accelerated Learning.

11. How people fail at decisions: Kahneman, Thinking, Fast and Slow & Noise.

I had to leave out a few about statistics, causality inference, project portfolio investment that didn't make the top 20 cut.


love it! I've got some audible to listen to :) thanks


Yep, I'd love to get the full list too.


I did manage to write up a list in a response to the grandparent comment.


staff engineer by Will Larson.

recently read that book. it's funny that the book wanted to show a diverse group of staff engineers and what they do. but i ended up thinking, oh wow, they are all pretty much saying the same thing. (there were a few exceptions)

but at least the book gives you an idea.


What about for becoming a Sr SWE or Tech Lead?


The Manager's Path by Camille Fournier is my go to recommendations for all new tech leads or those getting into people management on my team.


What about Debugging Teams?


Or Peopleware


I've actually read Peopleware and got a lot out of it. It's been a while, so I might be due for a re-read.


I translated a few important parts:

> If you’re not making your delivery timelines and this is a surprise to your organization, you and your manager have a problem

When the bullshit arbitrary estimates you gave became bullshit arbitrary deadlines, we didn't like it when reality happened.

> If product has a dream and no one knows what it would take to build it (time, resources, architecture), you have a problem

Some MBA cock landed a customer on lies about features that won't and can't exist and now it's somehow our fault.

> you are basically unfireable because you know so much

We are at the mercy of assholes. Our company rewards generating needless complexity.


Nope. Way too cynical of a take.

> If you’re not making your delivery timelines and this is a surprise to your organization, you and your manager have a problem

If you're not communicating the state of the thing you're working on with the people who need to know then you have a problem.

> If product has a dream and no one knows what it would take to build it (time, resources, architecture), you have a problem

Your job is to effectively communicate what is and is not possible, and at what cost.

> you are basically unfireable because you know so much

Sometimes things are complex and you know why they are complex and how they got that way and how to avoid those mistakes in the future.


> If you're not communicating the state of the thing you're working on with the people who need to know then you have a problem.

You're acting like it matters that you're communicating that there has been a setback or delay. Your estimate is a deadline and failing to make the deadline will be a disappointment.

> Your job is to effectively communicate what is and is not possible, and at what cost.

No one cares about any of this. All the people who actually run the company care about is fulfilling what they are now contractually obligated to do -- an obligation they made without consulting you, so everyone can move on to the next round of customer commits.

> Sometimes things are complex and you know why they are complex and how they got that way and how to avoid those mistakes in the future.

That sounds like everyone working on any large project. That doesn't make anyone unfireable.


I fully believe that there are companies out there for which this is true. I also know that there is at least one organization out there for which it is not true, at least in several parts of the organization, because I've spent the last fifteen years working there. (It's a big org; there are probably parts that do have this kind of toxicity but I've been fortunate enough to avoid them). My work slides to the right all the time, but because I keep my managers appraised of the risk, give them as much warning as possible when something unanticipated interferes with progress, and work with them to consider alternatives and figure out how to respond, I have gotten consistently positive reviews and a wide selection of projects for me to choose from whenever I have availability.

Maybe I just got super lucky--it's a good fit for me and it's a huge part of why I've stayed so long--but to say that all companies and upper management are as you've described? That's painting with too broad a brush.


This is what I've seen. The blame culture is so those above you are shielded from failure but rewarded from success.

The point being if you're an engineer working on a project and it is done quicker than the timeline there will be nothing extra accept a thank you. If your project it late you could be fired.


I have noticed an uptick of “staff X” as the new prestigious goal for technical careers.

Is this a recent phenomenon? Does anyone know why there are so many upvoted stories of people just recounting their daily work?


People with this title are making 400k+


Previous discussion: https://news.ycombinator.com/item?id=24437715

(edit: fixed the link)


Doh, I submitted this twice, fully unintentionally. Came across it on Amy's feed again and well... forgot I'd submitted.


Reposts are ok after about a year! so no worries

https://news.ycombinator.com/newsfaq.html


Not to hijack the thread, but anyone knows any book on what a director of engineering does?


A director is there to get shit from VPs about EMs and get shit from EMs about VPs. FTEs think you're not an engineer anymore and forgot everything once your title changed and "business people" think you're too much of an engineer and don't understand the rest of the business.

Also most of your contribution to the business is ignored if you can't hire.


This is a fantastically accurate description. I'm tempted to send it as a card to my director along with a sympathy bouquet and a box of chocolates. Well done.


They can wipe their tears with stacks of hundred dollar bills :)


The Manager’s Path by Camille Fournier has a chapter or two in senior engineering management, and regardless of that is just a great book about being an effective manager in technical organisations.


A director is the interface between the higher ups and the lower levels of the org. Basically taking strategy and vision by articulating it to product deliverables. They’re also the first to fall on their sword when things don’t work out and heads need to roll.


Exactly. VP is the level that spends most of their time concerned with how their function strategically fits with other functions. It's the entry-level position for C-level corporate politics.

If Director is the most senior you can be without being that, you're generally the highest execution-focused role that lets your VP do their thing. Somewhat equivalent to an XO in the military. At the same time, you're usually the one who has to translate the relevant parts of the strategy into something relevant and tactical for everyone below.


This is the first I've heard of the "Deep Diver" role. How common is this at companies of varying sizes?


Not very common as a 'role', unless you're in the Navy. Because it's made up and it's an oddly specific description of some person the author observed. Everyone can become a 'deep diver' as appropriate. It's not a role, at best it's an archetype which evolves naturally if the organization allows for it. Nobody hires 'deep divers' (unless of course, we're talking about the Navy) or has a business need for 'deep divers'. All of this is just malarkey.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: