> My current organization has 8000 staff members, and I used to glance over the usage metrics of the dashboards my old team produced. I believe three dashboards were used, out of somewhere around 50 - 100. Of those being visited, it usually turned out they were being visited by members of our team checking to see if the data had refreshed for the day. I'd estimate the cost of maintaining our team was approximately $1,000,000, for an organization that supposedly can't afford to waste that money.
I’m not sure how to explain this, but often when I find posts on HN linking to an engineer discussing how management is doing it wrong (especially if it’s someone who’s not running a successful company themselves), there’s usually a hyperfocus on really insignificant numbers as evidence of their point.
This company has 8000 employees. And the cost of maintaining those dashboards is about $1mm according to the post.
It only takes someone in management to identify a change which improved employee efficiency by $125 annually to completely pay for the $1mm spent to maintain the dashboard.
Identify a new $125 efficiency improvement every year, and by year 5 your $1mm annual expense is saving you $5mm every year.
It's slightly more complicated, and the numbers are somewhat obfuscated to hide my actual employer. You're right that these numbers are not a huge deal - that much is obvious, or we wouldn't be wasting that money.
What is more relevant to the average reader is that we have a large department in this space, and almost all (not all) the work is pointless because the numbers just don't matter at this scale. We have a core business that generates amounts of money large enough to fund departments that do nothing other than reaffirm that we care about whatever that department is supposed to do. That's totally fine (though I doubt it's a deliberate strategy) unless you waste time thinking they want to make people read the dashboards. It turns out that it's totally irrelevant to anything - they just want to have the team there to say they care about data, and you'll go insane if you try to do your job better.
But again - isn't the antithesis of this that the company never does the work to look into any of this and, in doing so, misses a huge but non-obvious problem that these tools might reveal? So you need to build it to do the work and check, and then you need to maintain it (because you already built it and the possibility still exists, etc, etc)....
Like, absolutely, there's some impossible to define line below which the org does not care about the data. But also there's probably a point where they would care! Being data driven is not about spending all of your energy optimizing for exactly the scenario you have data to optimize for (see: the post-covid logistic crisis).
To me - it seems like you need to "waste" a certain amount of money tracking down problems that don't happen to exist - but you can't know for sure they didn't exist before you tracked them down.
I guess like...do you have the impression the data should drive people in a direction they are not going? Like...if the systems you work on are saying things are ok aren't they ok?
Author here once again. I have the worst possible answer for this - most data teams that I've seen (and my network is large enough through my own socialising + my blog to be reasonably confident on this) are working on dashboards that could not conceivably be acted upon. As in, the nature of the thing they are reporting on is not susceptible to top-down intervention.
You are however totally correct that some places will have actually valid use cases for this kind of reporting. The article is very much not for those people, as they don't have these kinds of concerns for the most part. This is for people that are watching their organization talk about behaving one way, then acting in a totally different way.
This. It's like arguing that we are wasting time with application logs because 99% of devs aren't reading them. They aren't there for your pleasure, they are there because, the one time you actually need them, you'll be sorry they aren't there.
I think this highlights the big difference between being Data Driven versus Data Reactive. If a dashboard surfaces a strategy problem; you're reacting to data. If you're forming strategy based on a dashboard; you're being driven by data.
That’s true from a top-down corporate perspective, but this blog post is addressing the perspective of an individual on the team, who is repeatedly told that the company really cares about data, and then can’t seem to understand why serious problems with the data are just ignored.
I don't think that's actually what it says though! The blog post talks about how no one looks at the data (implying that the data is not used) and it separately talks about the inconsistent and disconnected ways that companies allocate funds. It does not, that I could tell, actually say the metrics work has revealed data the company is ignoring. Obviously if they are that's different from what I said!
> It does not, that I could tell, actually say the metrics work has revealed data the company is ignoring.
I guess I wasn’t clear: The metrics work isn’t _revealing_ anything new, the data requested is what is produced. The fact that the data is _ignored_ means that it can have serious errors, but then nobody seems worried about correcting it except for our frustrated conscientious developer because producing the reports is a performative act. Nobody really uses the data for anything.
Are you hiring? I'm looking for a job that will allow me to explore my passions. My passions are: video games, anime, and watching youtube videos about video games and anime.
If you enjoy anime I can't think you'd get much enjoyment out of the horrifying low quality of anime YouTubers. They certainly don't seem to like it much.
One of the most popular channels is called Trash Taste in an attempt to act like it's all foreign oriental nonsense they're wallowing in.
(Although to be fair, the current anime trend is "guy who hates women gets transported to a fantasy novel where he gets a sex slave", and the previous trend was "incest romance".)
>I can't think you'd get much enjoyment out of the horrifying low quality of anime YouTubers. They certainly don't seem to like it much.
I love game reviews and they give me very different lenses on how to view video games. Even the ones that absolutely hate a game.
I have yet to really find a useful anime reviewer. There are a few videos from a few youtubers that offer some interesting insight, but seasonal anime reviewers are either poor writers/critics or are simply trying to appeal to an audience who wants to hear their opinions regurgitated. So they focus less on insightful interesting anime and more on hating [current anime trend].
>the previous trend was "incest romance"
it's still around in manga/light novels. I think the "issue" is that these days most anime use it as a small trope instead of the entire premise. Heck, even in most of the infamous examples the element wasnt omnipresent.
- 75% of OreImo is not focused at all about the romance of the siblings. In fact, the male lead dates someone else entirely in the 2nd season.
- Kiss X Sis opens up to a more general harem in the middle of the anime and that MC dates the teacher for a while (SPOILERS I guess, for anime only. But that anime is a decade+ out and the manga ended 2 years ago. You're not getting that arc animated).
- Domestic girlfriend is the most recent and probably tamest example. Yes, I guess it's technically incest to date your new 17YO stepsister that you had sex with before you even knew your parents were getting married. We're well past Westermarck effect though.
It only takes someone in management to realize that you increase the revenue by selling more of an existing product or raising its price. You reduce your costs by firing people or renegotiating the prices with suppliers.
"Increasing efficiency" is mostly a theater meant to demonstrate effort during low/no growth periods and without viable products in development to validate such dynamic.
> "Increasing efficiency" is mostly a theater meant to demonstrate effort during low/no growth periods and without viable products in development to validate such dynamic.
That really depends on the business. I worked with a manufacturing client a few years back, and millisecond process improvements could translate into millions of dollars increased annual profits. They manufactured parts that would sell for about $0.75 a piece at razor thin margins.
I once worked at a place where management kept going on and on about reducing toil, but wouldn't let us use something to automate a bunch of extremely manual workloads.
It eventually occurred to me that we didn't have to, because the team was the mechanism for automating those workloads. They couldn't tell us they weren't that important, so the easiest option was to just talk about automation while paying us a bunch to keep troubleshooting the gross scripts.
Again - that is totally fine, as long as you know what's going on. Then you can make a clear decision around whether you want to stay.
Or increase sales from $1B to $1.001B, or increase efficiency of 1,000 of those employees by $1,000. If literally nobody is viewing a dashboard then it's probably useless, but even a small number of views could have very high value.
Other possibility: I had a non-programmer friend who worked at BigCo and worked hard for 2 months out of the year, doing nothing the other 10 months. He asked his boss for stuff to do and the boss said "just try to look busy". Seems like a good case for outsourcing, but that 2 months of work was pretty important and in-house employees will generally be more reliable and knowledgeable (about the company specifically). So really they were being paid (not much btw) for the 2 months of work and spend the rest of the time being prepared. This was an extreme and obvious case but I suspect this type of thing is pretty common, just usually people get busy-work (eg "make this BI dashboard") rather than being told "try to look busy".
I think there is something to be said for having sufficient staff to handle peak capacity. If you run a lean ship, you are likely to run into situations where existing resources cannot handle the demand and/or headcount attrition leaves you with a gap. It takes months to onboard someone and make them productive.
The flip side of that is the peons are not told the company over-hired and the headcount are being paid "just in case". Which leads to corporate shenanigans where BS work materializes to make people feel busy.
Can anyone here look at "identify a change which improved employee efficiency by $125 annually" and think it isn't going to be complete bullshit? That's in the region of 50c/day. "I told employees to reuse paperclips which I predict will save the company $1M per year"[1] PR/marketing/self-promotion - surely nobody believes it?. And that saving 50 cents per day is supposed to come from a manager who can influence all 8000 employees, but who doesn't think to axe the useless reporting department to save $1M/year in one go? And then the company which keeps a $1M department just to sound good is going to identify $1M in piecemeal efficiency improvements year on year, every year?
Who believes this?
Although I don't completely disagree, see[2][3] "Another reason to have in-house expertise in various areas is that they easily pay for themselves, which is a special case of the generic argument that large companies should be larger than most people expect because tiny percentage gains are worth a large amount in absolute dollars. If, in the lifetime of the specialist team like the kernel team, a single person found something that persistently reduced TCO by 0.5%, that would pay for the team in perpetuity, and Twitter’s kernel team has found many such changes."
But then again, at the bottom of [4] "At multiple companies that I've worked for, if you tally up the claimed revenue or user growth wins [of people claiming they improved things X amount per year] and compare them to actual revenue or user growth, you can see that there's some funny business going on since the total claimed wins are much larger than the observed total." which is pretty much my criticism of your comment; people will embellish their claims, especially when nobody can easily prove or disprove a claimed 50cent/day efficiency improvement over 8000 employees.
> "usually a hyperfocus on really insignificant numbers as evidence of their point."
$1M is roughly an average UK household income, after tax, for a lifetime. This just reads like dismissive boasting rather than a useful part of your comment. If it's "really insignificant" why do you then argue that the company will be motivated to save it in a more difficult piecemeal fashion, year after year five times over, if they aren't motivated to save it once in a big lump?
Firstly, I absolutely agree, the argument you're responding to gave me pause for a moment, then I realized it's probably nonsense in most cases and it also isn't disagreeing with me at all.
If the thing I am working on doesn't matter because it's a rounding error to the organization, but they keep telling me that it does matter and we should be driving for perfection, bla bla bla, then the original poster is agreeing with me, I think. I'm not arguing about whether it makes sense for the organization, I'm making an argument against believing the organization.
EDIT: actually, re-reading the parent post after posting this, I'm now not sure if I'm agreeing with him in arguing with the GP or disagreeing with him. But either way, I'll leave my comment here to try to explain the thinking behind $1m being insignificant to a company.
The point isn't whether $1m is significant to an individual employee. It would make a massive difference to almost any employee on a personal level (except maybe for C level in massive companies), but in a company with 8000 employees, $1m is a rounding error. Even looking at the example you quoted for the average UK income (currently £33400), for 8000 employees that's £267m or $323m just in salaries. When you consider all the other costs of running a business on top of that like buildings, light, heating, expenses (generally accepted wisdom is that a salary is about 50% of the overall cost for each employee), then factor in the material costs for whatever the company is doing, that $1m really is insignificant in the grand scheme of things from the company's perspective.
Average US salaries are higher than the UK at around $59k, and tech salaries are obviously higher, and in the US even more disproportionately so, making this $1m saving even less significant to the company.
The middle section of your post "If, in the lifetime of the specialist team like the kernel team, a single person found something that persistently reduced TCO by 0.5%, that would pay for the team in perpetuity" seems to be arguing a different point. It's not about whether saving the company $1m a year is worth bothering with or not, it's whether paying for a team that might cost $500k per year and can produce $1m per year in savings and/or revenue is worthwhile. Obviously, it is worth keeping the team as the ROI on their work is 100%. However the decision on whether to create that team in the first place is harder to justify because there is probably a more lucrative opportunity for the company to find - if the company is currently making e.g. 7% profit doing what it's doing, it probably makes far more sense to expand production capacity to increase total revenue as that will be be more significant to the bottom line than a small project that saves $1m.
I agree with pretty much everything you say here. But a thought hits me on the dan luu kernel issue - let's say we fund a team of kernel devs and they reduce TCO by .5% - the nature of FOSS means that the reduction in cost is illusory - all their competitors get the same TCO.
The funding of the kernel team is similar to paying taxes to fund science research ... it's just the more I look around the more I see a vast socialist society we live in that we think is some red in tooth and claw capitalism.
The main value of capitalism seems to be the creative destruction- replacing old inefficient companies with ones newly formed, built
differently.
It's just that if we found a way to close down companies after say ten years and forced them to build new, would that solve as many issues as schumpter?
I’m not sure how to explain this, but often when I find posts on HN linking to an engineer discussing how management is doing it wrong (especially if it’s someone who’s not running a successful company themselves), there’s usually a hyperfocus on really insignificant numbers as evidence of their point.
This company has 8000 employees. And the cost of maintaining those dashboards is about $1mm according to the post.
It only takes someone in management to identify a change which improved employee efficiency by $125 annually to completely pay for the $1mm spent to maintain the dashboard.
Identify a new $125 efficiency improvement every year, and by year 5 your $1mm annual expense is saving you $5mm every year.