Doing your own tests is good, but I think the username/password is too simple to benefit from the madlibs style. I also thought the wording was unusual, "I'd like to use this word ...... as my password." It could be that the strange wording was what caused the drop-off.
I thought the username and password was a good choice for this test. There really wasn't that much of a difference in phrasing between Patrick's form and the Huffduffer form. The most likely explanation for the drop off is the difference in the audience these sites are made for.
As far as design decisions like these go, the data doesn't say anything on an absolute scale about the merits of one form versus the other, but it does tell us which is more effective for a particular audience in a particular context.
When you are done reading the Madlibs, it should read like a sentence. What he did was surround regular text fields with strange sentences. When you are done filling it out, you can't just read it like a regular sentence.
It should have been:
I would like to make bingo cards! Please save them for me under the name [ Your Name ], and and my email address is [ Email address] in case you need to contact me. But please [do / don't] contact me twice with hints, and [do / don't] send me a newsletter.
See, when you are done, you can read it like a sentence.
All he did was replace "Username" with longer wording. It didn't seem "fun" the way you did it, just odd and off-putting.
I think its possible its the wide difference in font. Because the madlib text looks very different (as opposed to bold input labels) it looks frail and slapped together. Larger fonts, bolder text, and more space could probably help it more visually.
Your version of the madlibs form also looks much worse than your usual signup form. I don't know how much that affects user signup percentage, but personally I'd have an easier time trusting your standard form because if it's design, not because of the "style"
I am skeptical that "looks pretty" drives conversions. I think customers pretty much don't care, based on "ugly" versions winning in many, many of my tests. Remember: the customer doesn't get to see the pretty alternative laid out side by side with the ugly alternative.
Example: form submit buttons in your Internet browser of choice look fairly pretty, right? (Or at least comfortingly familiar.) I would not describe http://www.bingocardcreator.com/images/bcc.net/next-step.png as pretty -- it is what I could throw together in ten minutes. But it clocks "pretty" buttons with the same text on it.
You're welcome to try it yourself if you think pixel-perfect alignment will make a difference. Please post results.
I'm as far from design as possible most of the time... and I simply don't feel invited by your madlibs form implementation. On the other hand Huffduffer form just looks amazing. You missed some elements which I think are crucial for what they've done...
In the Huffduffer example the keywords are exposed, the fields look clear and spacing is just the way it feels nice. There's also a rhythm - keyword, field, keyword, field... even though it's not a standard form - you just know what to do with it. LukeW's implementation is not as pleasing, but still looks good, because there's some whitespace and no clutter.
I don't think you actually compared good standard form to good madlibs form. You compared it to a hacky, badly implemented madlibs and wrote off the whole idea ("Anyhow: test ended, not touching the madlibs idea again").
It isn't a matter of looking 'pretty', it is how does the brain visually understand it as simple or not.
I agree that there designers go overboard with 'it doesn't have the right padding', etc.
But you can't ignore HUGE test like red makes people hungry more (McDonalds) etc.
That sort of design does translate, and the mad-libs form just looks confusing.
I would press that button just to make it go away. ;)
Not that it matters -- the data matter -- but I can certainly rationalize why such a button would be more effective than a standard form submit button: My peripheral vision has seen a hundred thousand form submit buttons by now. I'm not quite as blind to them as I am to standard-sized non-animated web ads, but it's close.
And it seems to me that the problem with being a designer is that it's physically impossible to design a web form in less time than the average user will spend looking at it. You will necessarily spend orders of magnitude more time looking at your forms than a visitor will: Visitors don't look at forms for fun. They click through them fast, or they click away from them even faster. If I had to stare at that button for an hour, as opposed to simply finding it and clicking it as fast as I could, I'd design it differently. Which might very well screw it up.
It's remarkably difficult to evaluate your own reasons for making decisions. Just consider advertising. It works very well, but nobody thinks it works on them.
In general, buttons that stand out more get clicked more. Ugly things tend to stand out better. In fact, 'standing out' may be a passable proxy for measuring 'ugly.' In a similar trend, annoying TV ads were found to be more effective than non-annoying ones because people remembered them better.
The design of something is the first indicator I use to evaluate how trustworthy a site is. The better the design the more I know that these people put alot of thought into it and spent money and time on it. Now, overly clever stuff is definitely annoying and bad.. but ugly is a big red flag.
Running tests is definitely good advice. But execution definitely matters in these things. Just looking at the two forms tested side by side, the original looks more well spaced out and balanced. The madlibs has the two text fields actually touching. Compare that to the original madlibs form (http://www.lukew.com/ff/entry.asp?1007) and you'll see that the original madlibs form just looks a lot better.
While everyone is piling on with the untested hypotheses, here's mine:
The madlibs form is just too wordy. Whereas the username/password field pair is not only less wordy, it's an idiom: I've seen those fields, in that arrangement and that order, thousands of times over the last decade. I barely even have to read the captions at all.
Not that this hypothesis, or any of the others, matters. The point of this article isn't to design a better form. It's to show us how to design a better form.
If I were to play pop psychologist, I think we have some fairly simple fixes that we want to work because they're cool/sexy/etc, and some simple fixes which we don't want to work because they're "stupid" or sound marketing-y.
For example, in the original madlibs test, where seven things changed, we want to attribute the change to the madlib element (which was sexy and creative) rather than to e.g. the text on the conversion button changing. I've gotten 20% lifts out of "stupid little things" like that before, and seen good writeups of the same from other people. But e.g. designers do not want to hear that the thing they spend 98% of their time on is essentially meaningless and the elements of the page which they scrutinize least carefully -- like microcopy or button text or (in one memorable test) which way the stock photo model is facing dominate the measurable results. This is like how doctors do not want to hear that the difference in clinical outcomes due to professional experience is totally dwarfed by the difference in clinical outcomes due to following checklists. (On the plus side, designers don't kill anybody for their professional pride.)
In general, I think the point of A/B testing is that you should ruthlessly soulcrush what you want with what the data actually tells you. I respect HN folks enormously, but I'm not hearing "I tested it and it worked" -- I'm hearing "I've got a great theory on why your data does not conform to reality. Reality is, of course, what I predict it to be."
Yeah, it fantastically proves the point: do your own A/B tests. I am guessing the implementation can create a huge difference. Maybe you can ask for help from some of the designers here on HN and see if a beautiful implementation makes a difference. Plus your two checkboxes can also go into the "malibs" form.
Seeing as we have two totally different results from different people, my conclusion is that even though the madlibs style can improve your conversion rate, it's too easy to get it wrong and actually reduce your conversion rate.
The majority of websites have used the standard sign up form, and it's known and recongized by most users. It's risky to to go with something like this that could have a strong negative impact if you do it wrong.
The madlibs form has a "sign in as guest" link, while the original sign-in form does not. The article doesn't specify whether a guest sign-in is counted as a conversion or not.
P.S. If you have good eyes you’ll spot the other A/B test ongoing on this page. I’m using the traditional way of mitigating cross-test interaction… ignoring the possibility of it
Both forms have a 50/50 chance of having the guest link on them, and guests don't count as trial conversions (because I have several months of stats that suggests they give me money infrequently and they cause more support issues than all trialers and paying customers put together). The thrust of that A/B test is to see whether aggregate sales go down if the guests option is taken away. If they don't, I will drop it.
I noticed something in sprites2.png referenced in your A/Bingo page: the bottom row of pixels on the third blue background image is duplicated from the second blue background image.
