Hi HN! I’m Arnaud, founder of Echoes HQ (https://echoeshq.com). We build dashboards on the activity of engineering teams, focusing on the value of engineering work.
I’m passionate about developer empowerment and building engineering organizations. This is what got me into engineering management in the first place, and why I later joined Docker to lead the core team. Throughout my career I’ve come to realize that organizations are often held back not by a lack of developer productivity or talent, but by the lack of proper context to achieving good results (a theme commonly discussed here in comments).
To help companies solve this, we surface why engineers are doing what they do in the simplest way we could think of. You define within Echoes what you’re investing efforts into (e.g., the categories of work, ongoing initiatives, or OKRs relevant to your organization), which Echoes publishes across all of your GitHub or GitLab repositories as labels. Applying these labels on pull requests is all it takes to surface how teams are truly allocating their efforts over time. You can later connect intents to external measurements, showing you whether efforts are actually making a difference.
Engineering managers can use this data to inform decisions on priorities, confront the plans to the reality, and communicate on the activity to their CEO / bosses / business partners with the right level of detail. One of the first use case we're using to illustrate what Echoes can do is the very common challenge of balancing the amount of efforts that should go into features versus technical work (https://www.echoeshq.com/recipes/managing-technical-debt).
I can see this being very useful for the managers, if the engineers reliably add the labels. In your experience, how do you overcome the problem of "laziness" where the engineers skip over the labeling step?
Great question :) There are several answers to it.
1. We integrate with the GitHub Checks API and surface missing labels as a failure (similar to failed tests), which acts as a reminder to add the labels. GitLab doesn't have an equivalent to our knowledge, but we have a Docker image and a snippet of yml you can include as a build stage for a similar result.
2. We had customers ask for a JIRA integration which we are about to ship that can help with that. It creates a custom field on your JIRA instance which gets populated with your configured intents, just like GitHub labels. GitHub pull requests which reference a JIRA issue will automatically inherit its labels, meaning that if the intent was expressed at planning time, then there's no additional work to do for these.
3. When discussing with organizations who request that every pull request be linked to a ticket for the sake of reporting, it's a no brainer: would you rather file a ticket for every commit or add a label?
4. Remaining untagged pull requests can be examined and labeled directly from the product itself (making it easy to erase the pesky leftovers).
Finally, the product is indeed targeted at managers at this time, but we have plans to make it more directly useful for the engineers too.
Hi Arnaud. Looking good! Looking at the pricing I don't think it would work for us. While we have 10 engineers, we have many more people who commit very irregularly. Some months we'd certainly go over the "active contributor" limit, but we wouldn't be getting any value out of tracking those additional contributions. Do you have an allow-list of contributors?
Thank you, and yes! We only account for contributors who are dispatched into teams within Echoes configuration. This is also meant for open source projects who don’t want to track contributions from the community, or for very large organizations who want to ramp-up progressively on the product.
This is quite interesting, I didn't think of this type of approach. When I was at Instacart I could have seen this being really useful. I always had IC eng ask me "why does my work matter".
Thank you JJ! Indeed, most engineers care about their impact and how their work contribute to the big picture. Unfortunately, the incentives structure in many companies in not designed to encourage that. That's why we're trying to create this missing link between engineering work and business results.
Congratulations, Arnaud! I really like the idea of your tool. I was using many tools to track dev productivity in the past – with all kind of charts and plots. Somehow, I never got answer to the question "what do we really spend time on? Is this mostly bug fixing, delivering new features, and how does it affect our KPIs?". I like that Echoes focuses exactly on that.
I've got one question – would it be possible in the future to generate some kind of alerts for the managers when for example the technological debt is growing above some threshold?
Thank you! We do get asked about alerts, both on metrics (as in your example) and on allocation (for example when the activity is significantly and durably diverging from the current expectations).
We haven’t started work on this but it’s very likely to happen at some point indeed.
Love the approach of keeping the planning very close to the execution. It's very similar to what we did at Open Listings (though the labeling system was maybe a little more complex and applied to both PRs and Issues).
Anything you can share about a grand vision here? In 5 years is Echoes a tool for product managers (OKR alignment), engineering managers (IC performance management), a replacement for one of those functions?
Thank you! We have tons of ideas of where this product could go, but I can stay with good confidence that IC performance management is not one of them: I don’t believe that this is something that can nor should be automated. If you look closely, we’re actually shifting the focus _away_ from IC performance and more into the effectiveness of the organization as a whole (which in a sense says more of the performance of the management).
Our hope is to break down silos between engineering, product, and “business”, which are at the source of so many organizational inefficiencies. We need this shared context where CEO/CFO understand where we’re headed, where engineers understand how they work fit in the bigger picture, where product can focus on market research.
I’m obviously extremely biased here, but being thoughtful and deliberate about the way we allocate our efforts just sound fundamental to me, and it’s not the sole focus of any particular function.
You are correct that you could achieve similar activity reports with JIRA epics but it requires a level of rigor and homogeneity that I believe is hard to achieve in practice.
1. JIRA most often captures what we _plan_ to do rather than what we _actually_ do. You cannot build exhaustive activity reports from JIRA unless you request all contributions to be linked to issues. This is especially true for technical work which tends to not be tracked and become invisible.
2. Most sizable organizations have an inherent diversity of processes and tools across teams which makes producing consolidated dashboards extremely hard (e.g., one team using JIRA, a second using Linear, and a third using GitHub issues).
For these reasons, our approach is to capture work where it happens rather than where it is described, and to use a central definition of intents as the ontology. Finally, capturing efforts is only one part of the equation: we allow you to associate intents with metrics to evaluate impact.
Thank you! It is important: there's so much potential wasted in suboptimal organizations, and no amount of engineering productivity can compensate for that.
We don’t filter on branches, so any merged work counts.
By default all pull requests are considered equally weighted, but there’s a set of labels that allow you to optionally influence that weighting (using basic XS to XL t-shirt sizing), so you could already tag it XS and have it count for very little. We actually had that question from a user yesterday, and we might add a way to ignore a pull request entirely (i.e., give it a weight of 0).
I’m passionate about developer empowerment and building engineering organizations. This is what got me into engineering management in the first place, and why I later joined Docker to lead the core team. Throughout my career I’ve come to realize that organizations are often held back not by a lack of developer productivity or talent, but by the lack of proper context to achieving good results (a theme commonly discussed here in comments).
To help companies solve this, we surface why engineers are doing what they do in the simplest way we could think of. You define within Echoes what you’re investing efforts into (e.g., the categories of work, ongoing initiatives, or OKRs relevant to your organization), which Echoes publishes across all of your GitHub or GitLab repositories as labels. Applying these labels on pull requests is all it takes to surface how teams are truly allocating their efforts over time. You can later connect intents to external measurements, showing you whether efforts are actually making a difference.
Engineering managers can use this data to inform decisions on priorities, confront the plans to the reality, and communicate on the activity to their CEO / bosses / business partners with the right level of detail. One of the first use case we're using to illustrate what Echoes can do is the very common challenge of balancing the amount of efforts that should go into features versus technical work (https://www.echoeshq.com/recipes/managing-technical-debt).
If you'd like to learn more: we have a two-minute demo video (https://youtu.be/3ZRGdZq7v24), or I'd be happy to discuss and run you through a live demo (https://calendly.com/arnaudp/echoes-demo). We look forward to hearing your feedback and answering your questions!