I don't know if I agree with everything here.. is the goal to capture emails/signups, or to get users that will use your product and provide meaningful feedback?
I know it's a chicken and egg problem, you want a large enough sample base to get users using the product but with vague messaging you are attracting people that wont use your product (people that you shouldn't target) and people that WILL use your product but don't understand the vague messaging. I would suggest taking all versions of the page and signup methods to test usability of the product. Optimizing on the initial signup is great, but if your users don't use the product after they signup bc it's so vague.. then you've just wasted all that optimization effort. Unless the goal is to create a huge list of signups.
Also, where is traffic coming from? Paid, organic search, social media, press release, tc, etc? That has a huge difference on your results. And what tools are you using to run the tests and what is the sample size? I think readers would love to hear those tidbits too
You are absolutely right and our goal is to get engaged users. We are doing a lot of other things to drive engagement and I’ll blog about that soon.
All of our acquisition methods specifically target sports fans and are zero cost. For instance, we have partnerships with some sports content sites that run our ads for free. We also drive quite a bit of traffic by posting links to Sidelines in team fan forms.
The on-boarding process for Sidelines takes a few steps. One of the things we do, is as soon as you sign-up via the landing page we ask you to select your teams. This gives the user something to do and we find that 95% who sign up complete this step. In the sign up flow, we automatically subscribe new users to other users on the site who are active and follow the same teams, and this is explained clearly to the user in the sign up flow. Once the user completes the sign up flow, he lands on the feed, which shows interesting stuff the people he is following have shared about his teams. Since the feed is such a universally well-understood UI construct, users get it, and we see a good percentage of our first time users reshare an item or comment on an item on their first visit.
We also send out a welcome email (which we’ve been experimenting with) which explains more details about the site and how to get the most out of it. We’ve had fairly good success in bringing users back to the site with the welcome email.
We’re currently in the process of implementing a tour which will walk the new users through the features of the site, that should further drive engagement on the first visit.
Hi, I'm the CTO of Sidelines (startup mentioned in the blog post).
For our landing page tests we use Google Analytics which now includes Google Website Optimizer. We track engagement by logging activity time stamps per user in Redis. We also track how many other friends a user invites as a means to measure our k-factor and we are definitely biased towards targeting users that are more likely to be engaged and bring other users to the site.
Yeah, I actually have the immediate response of bouncing back from that type of landing page. I have no idea what it does, so I have no incentive to sign up. I suspect that some may sign up just to find out what it is. I wonder what the effect would be of having that as well as some button or link to "Find out more"?
I think the results here speak more than any guesses we might make, but I'm just astounded with the results.
Side note: I was able to click "Sign In", and then from that page access the "About" page from the footer. That told me at least a little bit more about what the service is actually for.
Yup, the "Find out more" link is a great idea and something we're in the process of trying. We're doing an experiment with creating a 30 second video about the benefits of the site. We're planning to have a link on the landing page which says something like "Watch a quick video to find out more" which opens up the video for those users who are curious but not curious enough to sign up right away.
I find a couple of the conclusions of this article very misleading. A couple of the lesson are great, but I feel the need to point out a few conclusions that are not quite spot-on:
1. Random photos are better than screenshots
Everything is better than straight screenshots. You didn't show what you were using before, but at a dismal 5% conversion I am guessing it was straight screenshots that weren't "posed" on a device or anything? Photos that have relevant content and even a minor emotional effect will say more than your screenshots ever will. I don't think I would call a football and baseball image "random" on a sports site, though. They are more relevant to a sports fan than a screenshot.
2. Large Background Photos Convert Really Well
That is not universally true. The two screenshots you have provided demonstrate the effect of focus in a design, not the effect of background images. You have used a background that is not only relevant, but is high-contrast colors and blurred, which creates a "foreground" out of your form. That will make the user's eyes go directly to the form rather than getting lost in your boring headlines, etc.
3. Inciting curiosity works better than clarity
It's interesting that you interpreted this as "curiosity" rather than benefits vs. features. Your original headline was a classic mistake. "A social network for sports fans" is a useless statement. No benefit, no motivation, no reason to click. "Follow sports together" is inherently more appealing because it describes the benefit of the service and has a social motivation built in. THAT is what you have done. Curiosity is not relevant.
4. Focus on the value proposition for brand new users, not advanced users
Duh. I don't want to be disrespectful, but this is a pure numbers rationale. In any market there will always be a small minority of advanced users compared to the vast majority of basic users. This will be true for every service, ever, unless the only people that could possibly understand the offer are advanced users.
5. Allowing users to sign up with their email increases FB/Twitter sign-ups
This was a good one. It is usually less effective to give people an ultimatum (register or leave) than to give them options to compare (would you rather register with FB or email?). Users will focus on the choice rather than the "yes or no" if you give them useful options to choose from. I once read an example test where more people were willing to pay for an a set of partially broken dishes if they were compared to an incomplete set than if they were sold alone. Same principle.
6. Figure out and optimize for the target demographics that convert the best
In the age of analytics, it is futile to define target audiences before statistics are available. It is much better to do as you have done: launch it and let your target audience identify itself. Good lesson.
1.)“ You didn't show what you were using before, but at a dismal 5% conversion I am guessing it was straight screenshots that weren't "posed" on a device or anything?“
I looked hard for a screenshot of our original landing page but couldn’t dig one up. The screenshots were of the parts of our site that were resonating the most with our most engaged users, and we got good feedback on both the screenshots themselves and the visual design. However, we found that the additional complexity for our average user in having to parse and understand a bunch of screenshots is not as effective as a sports-related photo with a simple, powerful tagline and explanation.
“I don't think I would call a football and baseball image "random" on a sports site, though.”
You’re right, random wasn’t the right word. We meant that we didn’t experiment much with the photo itself or ask for feedback from a wide list of sources.
2.)“Your original headline was a classic mistake.”
We are not just talking about taglines or even comparing the two taglines we showed here. In fact, “A Social Network for Sports Fans” is almost as vague as “Follow Sports Together”.
We experimented with a lot of other options for messaging and many which were more detailed and clear (3 bullet points on the key benefits of the site, for instance).
In these experiments, we consistently, we found that simple but curiosity-inciting messaging works better than more detailed and clearer messaging.
4.) This lesson is obvious to us now, but was not obvious from the start. By sharing our results we are hoping that others will avoid our mistakes, even the ones that may be obvious to you.
5.) That sounds like a cool study. I'd love to read about it even you can find the link.
>... people were willing to pay for an a set of partially broken dishes if they were compared to an incomplete set than if they were sold alone...
I think this is similar to what is described by Dan Ariely in his book, Predictably Irrational, as the Decoy Effect. Relevant excerpt from Wikipedia [1]:
"People not only compare things, but also compare things that are easily comparable. For example, if given the following options for a honeymoon - Paris (with free breakfast), Rome (with free breakfast), and Rome (no breakfast included), most people would probably choose Rome with the free breakfast. The rationale is that it is easier to compare the two options for Rome than it is to compare Paris and Rome. Ariely also explains the role of the decoy effect (or asymmetric dominance effect) in the decision process. The decoy effect is the phenomenon whereby consumers will tend to have a specific change in preference between two options when also presented with a third option that is asymmetrically dominated. This effect is the "secret agent" in many decisions."
In your case the third choice - email - is the decoy or secret agent - making the other two options more attractive for ease of use.
What I took away is that women like to share things and men follow women. This isnt snarky, but an observation that might be applied to other sites that are not obviously women oriented.
I'd be curious to hear about how you onboard new users - from all appearances, the vagueness/single page sort of force people to register if they want to learn about the product. What kind of onboarding or lifecycle setup have you implemented - and any preliminary stats you can share?
That is a great question. We spent a lot of time designing our signup flow and are currently on the third major iteration. The one thing that we find works consistently well is to give the user something to do as soon as they sign up. For Sidelines, that something is selecting your teams. The drop rate on that screen is extremely low and we speculate it will approach zero as soon as we add more sports. We are also working on implementing a "tour" of the site as part of onboarding, showcasing the various features because we did get the feedback that things can be confusing initially.
Why do you have a hooded figure that's making the gesture I wonder, at least for soccer fans it evokes the notion that it's a hooligan - if you're trying to get the user to relate. I'd go for hood off and see some non-gender specific hair. YMMV.
PLEASE, do NOT follow this advice, PLEASE:
1."Random photos are better than screenshots"
3."semi-vague messaging on a landing page is more powerful than clear and detailed messaging"
No, the advice is not "random photos" and "vague messaging" but "Do A/B tests ASAP" - as soon as you have enough traffic to do them (yes, there's a threshold, under it you're playing dice, not A/B testing).
OK, this question is coming from someone who works in marketing as his full time job.
So you're saying providing details is bad and vagueness is good. Vagueness "incites curiosity" which increases signups.
Not sure if I like that. Feels a bit unethical. You are intentionally holding back information for the sole reason of increasing signups. If you showed the product, or listed the features, or provided more details, signups goes down? That's a sign you need to improve your product.
We're seeing more signups, more engagement, and more activity. Not sure how that is a bad thing?
"So you're saying providing details is bad and vagueness is good. Vagueness "incites curiosity" which increases signups."
Not really, we're saying that for certain kinds of free consumer apps, curiosity can be a very strong motivator for someone to try a product. (This kind of tactic would never work for a paid product for instance or a product targeted at business users.)
Vagueness doesn't necessarily incite curiosity, but asking first time users to parse detailed messaging often kills it prematurely, and you've got to find the right balance that optimizes conversion. This is what worked well for us.
Finally, there are many other tactics that we're successfully using to turn the users who just signed up into active, engaged users and they are mentioned elsewhere on this thread.
It's hard to measure since we've seen engagement levels consistently rise over the past month, but we've also been doing a lot of OTHER work to increase engagement and bring people back to the site. For instance we've been driving community/discussions around specific teams on the site, sending out email notifications/newsletters, FB notifications etc, optimizing FB open graph- all of which have been pretty effective for engagement. It's still unclear whether there is a direct correlation between clarity on the landing page and longer-term engagement.
This was extremely interesting. I'll definitely have to take the time to make one of those newfangled "huge bg photo" landing pages with my signup form.
You can do many things to spark curiosity in visitors to a landing page. Some may grant you a few extra minutes to follow their curiosity... but then what? Most of them will get an answer to their questions and then stop using the service. We see this on checkout flows where people will put something in their cart because they are simply curious about shipping costs and taxes. Then the site owners wonder why there is such high cart abandonment.
(I'm a co-founder at Sidelines.) LanceJones, you're exactly right: A key lesson from our landing page experiment is that vague messaging incites curiosity in users, and curiosity can be a powerful motivator to get users to spend an extra few minutes to sign up for a service. This same user may have decided to bounce off the landing page if it went into the details of the value proposition (this is what we did with the initial versions of Sidelines' landing page), so the vague messaging has a better chance of getting the user through the door. However, whether or not the site is able to keep the user engaged post-signup depends on a number of different factors: The engagement numbers for sidelines have consistently gone up as we keep optimizing our sign up flow once the user logs in, the first run experience of the product, the welcome email, email/FB notifications, newsletters, etc.
We are not using anything fancy, it's just Google Analytics, redis and postgres. If we send anything out to the world (email, link to partners, open graph action) we tag it either with a google analytics campaign or a unique identifier so we can trace it back to our database. Every 12 hours, using a bunch of SQL commands + API calls we get a report in the mail.
http://quicksprints.com/post/32792397474/how-we-increased-la...