I worked on an internal platform for a large engineering org and was responsible for choosing what features we put in.
We had the technical means to track everything from commits to reviews, jiras, deploys, etc. Some of our most celebrated and impactful features were reporting on Accelerate metrics and related. E.g. deploy frequency, size of changes, duration of stages in the dev cycle and such.
I set a very inflexible red line: we don’t expose data more granular than at team level. Ever. No excuse.
Quite a few line managers would come to me asking for “personal metrics”, and even engineers in the team were super interested in building that (“with all this data we could…”).
My argument was that these metrics are toxic and invite misuse. A second factor was that this misuse would generate mistrust from engs against the platform and damage adoption.
Instrumenting an organization thought of as a system is fine. You want to see where are bottlenecks, you want to have measurable targets for how technical work is done and how it correlates to business goals/KPIs.
You want to offer teams metrics of their delivery my process so they can take the info and implement improvements whenever they see fit, and have a data driven conversation with the business (e.g. about the right setting for the carefulness knob)
But teams are the minimum unit of ownership, we stop the instrumentation there. Sure, a team’s performance ultimately links to individuals, but that is the manager’s job to figure out.
Interestingly:
* only line managers asked for this info, nobody in a director/vp/cxo role
* the most annoyed by me saying no were engineers in the team who wanted to do these features
Agreed. And "personal metrics" can also end up having a lot of consequences that were not planned for. Incentives are tricky that way. There's always a long tail of things that need to get done on occasion that doesn't show up in these types of metrics and it becomes difficult to find takers when everyone is optimizing themselves around the core metrics as bonuses, promotions, and even keeping your job in today's climate could be determined by those. It also makes allocating load to play to contributors strengths (which is often what they enjoy most) far more difficult.
We wanted the metrics to create the right set of incentives to make people improve the right parts of the system.
For example, we did present deploy frequency prominently. This gets people to see it, managers to want their team to be in the upper percentiles, etc. which drives a set of practises that, in general, and backed by research, are beneficial.
One of my favourite features was putting two graphs together: size of PRs vs. time to review. Time to review went up more or less linearly to size of PRs, but past a certain threshold (different per team!) time to review dropped sharply with larger size of PRs. This made for a good conversation to topic with teams that sets the right incentives for smaller PRs, iterative development, etc. (and it happens to correlate with deploy frequency).
Might also suggest a size limit for PRs. Theres always that Golden Child who gets away with things because they are prolific. But they tend to screw up architecture because they make too-big moves that discourage feedback and negotiation.
This is a great comment. My thought, after reading it: Why do line managers want this info? Do you think they have someone in mind for promotion, and they are looking for metrics of accomplishment?
And, cynically, I would say that senior managers don't care... because to them, most hands-on engineers/devs are fungible. What is your view about why the upper levels never ask for it?
> Why do line managers want this info? Do you think they have someone in mind for promotion, and they are looking for metrics of accomplishment?
Nah, you don't need to assume malice :)
Most times it was managers with good intentions, not realizing that those metrics were either pointless (e.g. how much code / commits does $person do), or toxic, in the sense that it leads the team to game metrics, that it prevents the manager to actually understand why the values are what they are, that it opens the risk to link them to promotions and perf reviews etc.
Explaining them all this was generally enough!
> And, cynically, I would say that senior managers don't care... because to them, most hands-on engineers/devs are fungible. What is your view about why the upper levels never ask for it?
Actually that wasn't the case. AFAIK (I left at some point, but kept a bit in touch with former colleagues) upper management started using some of those metrics to set organizational objectives.
Again, same argument. You don't need to assume malice. Management has a legit interest in engineering productivity. What happens most of the time is that they don't know how to measure it in an effective way, or how to use it to drive organizational change. Providing guidance is part of your job as a Platform eng.
So, it's a bit more than that. Let's say I've identified someone that I can't figure out what they're doing. They're just going slower than I expect given what I understand of the work. (I was a professional programmer for over 15 years before management, so this is based on the expectations of a former practitioner.)
Now, why is that? The first possibility is that they ran into a string of tickets that were just harder than we expected at the outset, and it's a statistical anomaly. A second possibility is that this person takes the hardest of the tickets, and anyone would struggle. A third possibility is they are taking tickets past their capabilities as a developer and aren't getting the help they need. In this scenario, they're a capable performer, but are need help to get to the next level. Maybe they're junior, or they're a new employee. If they keep taking these tickets, they will grow and their performance will jump. In these cases, you just keep monitoring.
In the middle case, they may have transitioned to a team or project that is a bad fit, and would thrive on another team or an adjustment to what they're asked to work on. They may be a front end specialist that was expected to pick up back end tickets and struggled from the start. Or, they may be in a temporary dip due to personal circumstances.
Then, you have the negative cases. Their work ethic might be insufficient. They may just not be good at their job and may not ever be able to match the performance at the pay grade and seniority they were hired. Or, you may have a good bullshitter on your team and your numbers tell a different story.
The numbers will tell you if this is a temporary dip. The numbers will tell you if their current output is in line with the rest of the team. From there, the hard work starts.
If you don't have those numbers, it starts looking like a lot of status updates and micromanagement.
Note: remember I am condensing the real work into a few paragraphs. Of course I know it's much more nuanced than I am making it here. This is a surface level treatment.
Also note: Most managers end up doing a lot of part time jobs, of which performance management is a relatively small part. If my primary job was performance and task management, that would be a very bad use of resources for the company.
As a former line manager, there have been two cases when I use metrics: I'm promoting someone, and I like having numbers that back up how good they are, or I'm firing someone, and I like having numbers that back up how bad they are.
I generally agree with OP, but there are times where as a manager you know exactly what is going on with your team, but numbers are still helpful.
I’ve helped build out or steer these sorts of systems a number of times and usually management behaves themselves during the adoption and honeymoon phases but then erode the trust later on by trying to use the system to determine PIP or promotion.
Devs who have seen this behavior before tend to push back hard on adoption, and then invest the absolute minimum effort in using these tools. The tools tend to be built wrong often enough to encourage that slide into toxicity. There’s an amount of using a tool where it improves your work experience, and then an additional amount that improves the team experience, and then beyond that it’s doing your manager’s job for them and self-reporting/narcing.
I’m trying to build a tool at the moment that has had three focuses due to different sources of inspiration. First it was going to be Trac (a project management tool and wiki that is so simple it shouldn’t work, but somehow does) with bits of Jenkins. Bamboo has thousands of features and integrations where it should have hundreds. All those integrations make reproducing a failed build locally difficult or impossible. The bones of a build process should be in the build scripts and dependencies, so you can run them locally. The build tool should schedule builds, report status changes, collect some release notes data, and track trends with grafana charts and that’s about it. I also want something running in each dev’s machine to boost system performance and do some of the teamcity features for trunk based dev like push-on-green-build. I just miss how distilled Trac was, but it had some problems with plugins and git support.
That sat on the Some Day Shelf behind two other projects until I read Cal Newport’s Deep Work, and then Slow Productivity. Then it became more user oriented. Atlassian has about three per-user dashboards that I’m responsible for juggling all day, and that is tragicomically stupid. I’m terrible with this sort of juggling but have coworkers who don’t check the PR list ever - you have to pester them to look at PRs, week in and week out. If I’m doing deep work I don’t want preemptions except in specific circumstances (like I broke trunk). But when I come up for air I need a cheap gestalt of what I’ve missed and what people are expecting from me. Show me all the PRs, and my red builds and open tasks in a single view. Allow some low priority alerts through. And that can be facilitated by building a pomodoro straight into the dashboard and information hiding during deep focus moments.
I have some family that was recently diagnosed as neurodivergent, and the thing about YouTube is that your suggested videos get influenced by what other people in the house are watching (particularly if you’re not logged in on a device). ND people of all types have a lot of coping mechanisms to function within other people’s expectations (eg work and responsibilities) and to mask. They get both barrels when it comes to being judged poorly by toxic management tools because their variability is so high. Best performer one day, second worst the next. And this is nowhere more true than with ADHD. And the worst of it is that almost nobody will go harder and farther than an ADHD person during a crisis. They can hyperfocus during rare moments but most reliably due to an emergency (self created, like a paper due tomorrow, or externally driven, like servers on fire). They also innovate by drawing connections nobody else sees between different disciplines.
But they’re the first on the block when toxic metrics kick in. And the productivity tools they objectively need more than anyone else on the team seals their fates, and thus they either don’t use them or use their own, which are similarly poorly integrated, which leads to more plate spinning which they are terrible at.
So what finally got me ass-in-chair in front of a keyboard was realizing that if this is two tools instead of one, you can keep some of the productivity data on the employee’s machine where it can help them with time management and self-improvement but not self-incrimination. Then you can cherry pick and clean up the narrative of your day or week before you self report. Have private tasks on your machine that may look embarrassing to others (like remember to drink fluids, eat lunch, call the dentist, tasks you’re skunkworking).
We had the technical means to track everything from commits to reviews, jiras, deploys, etc. Some of our most celebrated and impactful features were reporting on Accelerate metrics and related. E.g. deploy frequency, size of changes, duration of stages in the dev cycle and such.
I set a very inflexible red line: we don’t expose data more granular than at team level. Ever. No excuse.
Quite a few line managers would come to me asking for “personal metrics”, and even engineers in the team were super interested in building that (“with all this data we could…”).
My argument was that these metrics are toxic and invite misuse. A second factor was that this misuse would generate mistrust from engs against the platform and damage adoption.
Instrumenting an organization thought of as a system is fine. You want to see where are bottlenecks, you want to have measurable targets for how technical work is done and how it correlates to business goals/KPIs.
You want to offer teams metrics of their delivery my process so they can take the info and implement improvements whenever they see fit, and have a data driven conversation with the business (e.g. about the right setting for the carefulness knob)
But teams are the minimum unit of ownership, we stop the instrumentation there. Sure, a team’s performance ultimately links to individuals, but that is the manager’s job to figure out.
Interestingly:
* only line managers asked for this info, nobody in a director/vp/cxo role * the most annoyed by me saying no were engineers in the team who wanted to do these features