One mistake I (and no doubt other founders) made was assuming you'd "get it right" because you were scratching your own itch. My cofounder and I started our company [0] literally based on the pain points we were feeling and still, when the first user tests on the first version of the product came around, users didn't get it. Or they didn't see the value. It was a tough lesson - we were building the thing we thought we wanted and others didn't want it. But we persevered, pivoted a little here, rewrote a lot there, and ended up with something that users not only liked a lot more, but also worked better for us too.
Moral of the story is scratching your own itch makes you qualified to talk about the problem, it doesn't make you an expert on a good solution.
0: https://kitemaker.co, the super fast, hotkey-driven product management tool/issue tracker that has deep integrations to GitHub, Figma, Slack, etc.
Yeah, this is a good point and something I learned the first time round.
Just because I (as a manager) had previously been willing to adopt bleeding edge tech with no hoops to jump through to eradicate hundreds of man-hours, doesn't mean that the average customer is.
If you're not representative of the typical customer/user, you need to find someone who is to use as a yardstick.
It doesn't automatically mean you're headed in the complete wrong direction, but it does mean a lot of your initial assumptions (pricing, deal size, sales cycle, etc) are probably way-off.
completely agree with this. This is why the idea behind Cagan's "Empowered Product Teams" [https://svpg.com/empowered-product-teams/] is so important. Founders are often too blinded by their initial approach of solving their own pain that they are blinded to superior alternatives that could be more easily seen by fresh eyes. Key is to empower those people to solve the problem, not just implement a solution.
We're big Cagan fans at Kitemaker. In fact, most of our thinking around the product is tied into empowered product teams. Cross-functional, autonomous, self-organizing and impact-focused. We want Kitemaker to be the best tool out there for such teams.
"..makes you qualified to talk about the problem..." I like that take, you are in a position to empathize but not necessarily know the solution or even the real problem as it manifests differently in different settings outside of your experience
But, that's still better than 'trying to create a product that may solve the problem of others', right? I mean, feeling the pain seems a valid start point?
Great read and bang-on (I'm running product on the fourth software business I've started - the first three were sold). But there's a missing piece to the puzzle though. The author talks about using data so that you don't fall victim to bias, but he doesn't talk about how to do that.
In my experience to ensure data-driven product decisions you need both a system and process to:
1. Collect feedback
2. Organize it to be useful
3. Analyze and prioritize customer problems
4. Use your feedback to validate problems and solutions directly with customers
5. Communicate status to your team
6. Close the loop with customers when you build their requests
At the early stages you can do this with a spreadsheet or Trello board. Eventually you'll graduate to a tool or in-house solution.
Here's a piece that dives into the details about how to build your own system to collect and organize feedback so you can eliminate bias and drive product decisions based on data:
That’s what qualitative data is for. At the very least, talking to customers and reading their feedback. Preferably followed by organizing that data somehow, and using it to better understand the quantitative data you have.
It’s fair comment, though, that undifferentiated ‘engagement’ is rarely a good metric.
Also, hopefully the people in charge are thinking closely about what time engaged means, for that product. I imagine breaking user actions down to intent, if possible, would help quite a bit with that.
For a product that is supposed to be simpler and save people time compared to the alternative, increasing time using the service might mean people like it and are getting more done so using it more... or it might just mean that it's getting harder to use. Then again, maybe they've graduated from doing simple things with it to complex things, and even though interactions take longer, they're still saving more time and effort overall compared to the alternative and are happy.
I guess the bottom line is I think you have to slice and dice the data a lot of ways and think about what it actually means for your product to be using that data effectively.
Absolutely. I think that’s why it’s so vital to combine both qualitative and quantitative data.
I doubt there are too many people in tech making this mistake, but without numbers there’s a good chance you fall prey to people’s inaccurate perceptions or explanations for their own behavior.
On the other hand, without those first-hand accounts, it’s all too easy to tell a mistaken story about your numbers. In particular, your users’ sentiment towards their time spent in your app is a guess, unless they tell you.
I think timescale is another crucial dimension to evaluating time-in-app, and whether it’s a positive indicator for your business.
For instance, driving up engagement has doubtless been good for Facebook’s financials in the short- to medium-term. Arguably though, that relentless focus has led to the present political climate where they’re fighting off regulation. Whether that’s an existential threat is yet to be seen (one can only hope), but it certainly casts the metric in a different light.
As you say, it comes down to the fact that there is no silver bullet metric; there’s no substitute for thinking about and dissecting the data you collect.
Author here. Great feedback and completely agree with your comments.
Taking a structure/organized approach to collecting, indexing, and sharing the qualitative data isn't something I covered here but absolutely essential to make sure that a research project scales. My experience has been that the first mile is where founders get the most lost.
We went all-in on Enchant [0] with a set of screen mockups and potential users telling us that it was exactly what they needed.
That didn't work out so well.
It wasn't what they needed.
They didn't buy the product.
But we kept talking to those users and understanding what they thought they were getting and the problems they were hoping to solve. Those conversations gave us a better idea of what would work and helped build a realistic roadmap.
Executing on that roadmap has worked for us. We continue to evolve that roadmap based on customer demand - When somebody asks for something that's on our roadmap, we count their vote.. and generally prioritize work in order of demand.
So ya, talk to your users... it's not about what they ask you build, but what problems they're trying to solve.
Advice like this is fantastic, but I hate when it's positioned as the one-true-way to do product development.
Consider some of the greatest inventions of all time: the printing press, the telephone, the airplane, the Internet, and the web. Considering the notion that they would have been conceived of this way is absurd, so clearly there's more than one way to make something people want.
Mix in the fact that contrarians often disrupt things the most, and when you see a process like this that itself has become considered the consensus approach, it seems like a high point of potential leverage to just try a different method, especially if that one has also been proven to work or is more in tune with your own talents, experience, and intuition.
I don’t see why those things couldn’t have been invented by this process?
The article specifically states that you focus on your potential customer’s problems, not the solutions they want or would use. Ie, you’re not asking if they want a better horse, but you are asking what sucks about the one they have today.
From there better shoes, saddles, or even a car/plane can come out of that discussion.
I don't think the kinds of 'problems' people may have articulated that these things solve would have been revelatory enough to argue they fit into this process. At best, the problems would have been so general like "I am uneducated about the world" or "it takes a long time to travel between places" that they are effectively immaterial when it comes to the process that led to these innovations.
For example, if someone told you today "I don't like that I am going to die one day", would you be able to iterate your way via customer development to stop death? Would the fact that people don't want to die be incorporated in the understanding of the process that led to any solution that gets created? No. Some problems are either so obvious or so non-obvious that the spark of insight that leads to solutions is barely influenced by the articulation of the problem or lack thereof by those who are suffering under it, often completely oblivious of it.
Thanks, I agree that the default path of "be like Steve Jobs" is a trap that I've fallen into enough times to realize that there are more ways to build where you don't have to have the insights up front.
The key in my opinion is to be able to identify market opportunities and problems to solve, then be REALLY flexible about how you go about solving them b/c your first product solution ideas are very unlikely to work.
I agree this is definitely a great way to solve problems, it's one of hill climbing basically, where the tactics and strategy really need to be carefully managed since it's very easy to get the gradient or step size wrong.
Another way to see it is it's a form of de-risking an already risky endeavor. Anytime you reduce risk by adopting a methodology or formula that has known benefits, you're basically trading options (in the financial markets sense.) In this case, you're (probably) reducing downside risk, while increasing the odds of success, but at a cost of reducing the overall maximum upside, since your odds of creating a once-in-a-generation disruption goes down if you're using a hill climbing methodology to create wealth.
Sure, we've all known founders who've created products that were successful without talking to potential users, but it's very rare. What we've all encountered more often: founders who stubbornly don't talk to users as some sort of half-understood adherence to the biography of Steve Jobs. And then their product fails. If they had only tried a different way -- I don't know, talk to the people who might pay them? -- they could have avoided complete failure. It's not as glamorous as magically stumbling on a good idea, perhaps, and requires extra work, but it's often a more viable path for most of us.
I think what gets overlooked about Jobs, even among his admirers, is that he did start with an audience. Particularly by the macOS X and iOS days, Jobs was fundamentally driving the product design towards what he wanted with the implicit idea being that if he liked it and used it, others would too.
This has its own pitfalls, but he was fundamentally "talking to users", he had just narrowed his focus group to a user of one. He used his own products as far as we know as well.
The thing is that most tech doesn't dogfood quite so easily. Most founders can't replicate that as they are generally selling outside of the consumer market that they can just adopt themselves.
So, maybe cast another way, every product must have a target demographic and you need to interact with that demographic to get feedback on what you're building. Jobs just happened to have a shortcut that isn't generally applicable.
And of course, Jobs built the rest of Apple around this culture.
I read Ken Kocienda's book about working on the original iPhone keyboard [1]. He says development revolved around demos: first to each other, then to senior leadership, then to Jobs.
2000s Apple tried to hire people with good taste, then constantly tested against that taste. It was all internal, but it was testing.
The cited article pulled out a catchy headline from an extrinsically motivated context like Telemarketing. Not surprising that people who fail from a rote task get de-motivated and quit, learning nothing.
I don't believe that study has any relevance to the world of creative work -- like building products or becoming a Jedi ;) -- that is driven by intrinsic motivation to iterate and improve. Success rarely requires the same level of self-evaluation and reflection in creative contexts.
> I had listened to what they said — exactly as they said it, but I did not realize until much later that I failed to actually understand what they meant.
Two things come to mind:
1) Wants are not needs. To pun Ford a bit, "I want a bigger, faster more reliable horse to get to work. Something safer for my family" is what the clearly stated want is. Remove horse and you're at root need. The Need. Too often, The Need is never distilled from the want.
2) In Covey's classic book "7 Habits of Highly Effective People" he says something along the lines of: Seek first to understand, then be understood.
Yes, listen with intent. But before you write a single line of code, draw - yes, free-hand - a prototype of (what you think) you heard and share it with the other person - the user, the customer, even a colleague.
Never assume. Never assume a want is The Need. Never assume you understand until you've vetted that understanding.
Solid post. Any advice for a founder on how to start engaging with users after a few years of _not_ doing that? Luckily we've had success despite our inability to get user feedback, but obviously we need to do better.
We did concierge onboarding and set up shared slack channels with many of our early teams. This made it really easy and expected to ask for more feedback and to have users give it proactively.
Community channels are key as you grow. Frankly this is an area where we've underinvested as we've been heads down building/shipping product and is going to be a huge area of focus for us in 2021. The job of the founder is to really to give people who love your product a venue to share their ideas, questions with other users. Customer advisory boards work well too.
If you're SaaS I'd start with sending automated emails to ask for feedback at key points in your customer journey. You might find this article I wrote to be helpful[1]. It covers those key points and also includes examples to make it easy!
This sounds very similar to what Design Thinking suggests. Connect with your (potential) customers to get their habits and preferences so you've more ideas and then prototype them through mockups or small models and once you've the feedback from the customers, decide to either go ahead with the version 2, pivot it, or shelf it.
Design thinking talks about being empathetic to your users/customers. Which also sounds like the essence of this post.
"Your ideas are not nearly as important as your process — and the best process starts with understanding what the customers you wish to serve already do to solve their problems today and even more importantly, understanding why."
Ideas matter, if your ideas do not include a reflection of current ways people are solving problems you have bad ideas and would be better off thinking and researching before talking to anyone.
For sure, ideas matter A LOT. But we over-index on product solution ideas by default.
I think ideas that matter most for is for founders are the ones around a strong conviction on the market opportunity and then be flexible about the solution that is built to capture that opportunity.
The latter is where I think process is more important than the product solution idea having seen my product solution ideas fail hundreds of times without changing my market vision ideas.
This doesn't mean our ideas are not more important, just that people have too much confidence in how good their ideas are. Its much harder to have a good idea then to create something to a good quality standard, the latter can be done by any expert which means it only requires capital, good ideas on the other hand can't really be bought and there are no sure methods to come up with them.
Thanks Scott. I remember back to our convos in the Grain office from a year back that helped to inspire the structure of this post. Was cool to see you invest in it at Tipe and reap the rewards!
Moral of the story is scratching your own itch makes you qualified to talk about the problem, it doesn't make you an expert on a good solution.
0: https://kitemaker.co, the super fast, hotkey-driven product management tool/issue tracker that has deep integrations to GitHub, Figma, Slack, etc.