About half a million years ago I sat through a Nolan Bushnell (Atari, Chuck E Cheese) speech.
He said a sentence that has stuck to me forever. "The only question that matters is when you say 'What is your Visa card number?'" Meaning that until the moment of truth when money leaves your customer's wallet, everything your prospective customer says is bullshit and not worthy of building a business on. He was completely dismissive of market surveys, etc. Waste of time and money.
He brought this thinking into physical products, too. His basic "build a business" model was:
1. Make a flyer. See if you can get an order. (Similar to the "dry test" idea). AKA the landing page.
2. Iterate until . . .
3. . . . you get nibbles and interest. Then make a mock-up. (You've gone from two dimensions to three dimensions).
4. Continue iteration on the marketing front until you get an order. (I.e., the customer gives you his/her Visa card).
5. Then and only then manufacture ONE of your products, even if it costs you $5,000. It's cheaper than manufacturing a bunch of units to get your unit cost down below $200.
6. Iterate based on experience, refine the product, get another order, etc. etc.
Even antique guys are smart. :-)
Sincere apologies to N. Bushnell if I mis-remember this. It was a long time ago. But the "What's your Visa card number?" question stuck. Life is either bullshit or jellybeans, and in business I tend to treat things as bullshit until a prospective customer puts his/her wallet on the line.
I have frankly come to believe that the landing page only approach is a non-solution to validating the idea. In a landing page, you are testing how well you can convince visitors to contact you, signup or download a whitepaper.
Real relationship with a customer start when he actually gets to try out the service you are offering. That is the real reason why trial -> paid conversions are usually low. In a landing page MVP, you are not validating your idea, you are validating your ability to convince for a particular idea. It is quite easy to get false positive or false negatives here.
Rather, what I prefer is to get hold of people and email/call them to actually hear about their pain points. Old school but works like a charm.
But if you get the consept validated it's great. You only have to focus on execusion. If you're focusing on execusion without a valid reason, you're wasting time.
I was having this discussion with a my partner not too long ago. Basically, we launched an MVP (a downloadable ebook about a niche topic) about 5 months ago. It has done pretty well considering we have spent almost no time on it since then (we've been busy with jobs and such).
We are planning to start working on really improving the ebook, the site, and adding other features that we think people who buy the ebook would be interested in. We have three ideas that we want to try out, and my partner is in favor of the Dry Test Method for each of these three, while I am leaning towards trying to build an MVP of one or two of these features/products and seeing which gain the most traction.
Anyone have any thoughts on what they think is the best approach? Note: Our audience is mainly international university students and young professionals.
What ideas? People reading the e-book likely want more information. You could build a landing page for another book, with a ToC and see if people are interested. And if they are, you write.
I advocated for a "Dry Test" with our company's SEM campaign, for features that we will build. My co-workers thought it was deceiving and would flounder customers' trust. What do you guys think?
Keep in mind in SEM you are reaching out to potential customers, not existing ones.
"Dry testing" in software is traditionally called "vaporware". It's been found to be an excellent way of taking sales away from competitors with actual products on the market, who are hampered by not being dishonest scumbags and only being able to get away with claims that are feasible.
I'm probably a minority, but your reputation with me is toast if I find out you ever did dry testing.
I understand the sentiment, but I'm curious how you personally judge an expensive idea without spending the 6 months to develop it? A simple web app may take a weekend, but that's not the only type of minimum viable product, unfortunately.
Market research still works when you admit to your audience that's what you're doing. They say it's somewhat less accurate that way, but even if that's true it doesn't entitle anyone to throw chaff into the system that allocates resources to us.
I spent some time at a startup in 2006 that was trying to build a business around what the OP is calling dry testing. It's very easy to cross the line into deception. If you're implying that the product or feature exists, I think that's deceptive. Why not just explain that you're testing to decide what you should develop, ask for the relevant feedback, and let people sign up to be notified if you build it?
Because feedback in that form is generally worthless. It's like going to a meet-up and tossing around your idea. You'll get a LOT of 'feedback' all of it as completely worthless as what your thinking already.
The whole idea behind a MVP is to get worthwhile feedback about the product. Feedback in the form of customers actions. Will people actually buy this? Will they respond to what I'm doing? Can I get traction?
None of those things can be answered with a submission form on a landing page.
I have long agreed with the author here. While there are answers to be found in dry testing... it takes a prototype to really solicit any truly meaningful answers about a product. At least in my experience.
I don't disagree. The landing page tests I saw were largely worthless. Most of the time there wasn't significant data, and even then it was so far from obvious how to interpret it that people simply confirmed pre-existing biases. But perhaps others have figured out how to make effective use of such data. My point was different: if you are going to do it, eschew being deceptive. Deception destroys more value than testing creates.
Was the startup building solutions around generating content for dry tests? Like a skeleton landing page generator? That thought crossed my mind as I read through some of these articles.
No, the idea was to run tests and build the products that showed uptake, and then to repeat the whole process incrementally as a feedback loop. But it didn't work for a number of reasons, two of which I cited above.
Posts that talk of their new startup's "MVP", which is nothing more than a landing page in front of a registration page
That is not what MVP is. MVP is the simplest version of a product that will allow you to iterate. That is something that can be used.
The landing page, the MVP, the iteration, the improvement, the pivot etc.. are all part of the Customer Development Methodology which Eries Ries is also known for.
The author is saying that other HN posts are claiming their lone registration page is an "MVP" (hence quotes). He wants people to stop calling these MVPs, because they are not, just as you say. He doesn't say that the landing page is bad, per se, but that it's 1. not an MVP and 2. not able to test the same things.
He said a sentence that has stuck to me forever. "The only question that matters is when you say 'What is your Visa card number?'" Meaning that until the moment of truth when money leaves your customer's wallet, everything your prospective customer says is bullshit and not worthy of building a business on. He was completely dismissive of market surveys, etc. Waste of time and money.
He brought this thinking into physical products, too. His basic "build a business" model was:
1. Make a flyer. See if you can get an order. (Similar to the "dry test" idea). AKA the landing page.
2. Iterate until . . .
3. . . . you get nibbles and interest. Then make a mock-up. (You've gone from two dimensions to three dimensions).
4. Continue iteration on the marketing front until you get an order. (I.e., the customer gives you his/her Visa card).
5. Then and only then manufacture ONE of your products, even if it costs you $5,000. It's cheaper than manufacturing a bunch of units to get your unit cost down below $200.
6. Iterate based on experience, refine the product, get another order, etc. etc.
Even antique guys are smart. :-)
Sincere apologies to N. Bushnell if I mis-remember this. It was a long time ago. But the "What's your Visa card number?" question stuck. Life is either bullshit or jellybeans, and in business I tend to treat things as bullshit until a prospective customer puts his/her wallet on the line.
/Phil