How does this actually indicate it had to do with the redesign? Sure the trend may be going up, but the site is increasing in popularity so that makes sense that it is going up. There are more people there to interact with and more communities people can join that they may be interested in following and discussing things in.
I am just wondering how this rise is attributed to the CSS change? It may have had some effect, but I'm confused how this single graph says that it is because of a CSS change?
In fact, from my reading the rate of signups was increasing before hand and continued increasing afterwards, implying the design made no difference. That's what I can see from the trendline anyway, and the rest of the graph is too uneven to draw any conclusions.
They are wrong to draw the trendline the way they did. Since what they are claiming is not steady growth (then the trendline would make sense) but a shift in behaviour they should instead have drawn one line for average before the change, and one line for average after the change.
It is quite obvious that something has changed. Before the change there are five days above 7.5%, and after there are more than 10 days above 7.5%. The valleys are much higher too.
Yup, but that's not an 8% increase in AR's. That's maybe an 8% increase in the rate of increase in AR's. That could be attributable to a LOT of things though (there isn't even enough context to control for network & seasonal effects, let alone whatever increased attention there might have been to sign ups after the rollout).
Ironically, probably the best way to isolate the impact to the launch is to narrow the time window and resolution to the minutes before and after the redesign launched.
More days above 7.5% would be true even if the underlying trendline was going up anyway.
They really should plot the moving average and include a longer timeseries - plotting the trendline before and after would be trying to prove what they are trying to show, rather than trying to work out whether what they are trying to show (our fancy new design wasn't a complete waste of time) is actually true.
The ugly truth is, the reason for Reddit's popularity growth in long term is /r/gonewild and its children. It's so integral part of Reddit, not even the femi-nazi-caterpillartrack-crazies dare to even attempt SJW'ing it.
Quote from a previous article (so I wouldn't attribute all 8% to bots):
We’re experimentally removing CAPTCHA and using other methods for spambot detection.
We know it’s annoying for humans, and are starting to worry the bots are actually enjoying it.
Plus, our visual CAPTCHA was failing on accessibility for non-visual redditors.
Sorry, to be clear, I wasn't attributing the lack of CAPTCHA to bots. Rather the improved usability and increased human sign-up rate to the removal of a painful CAPTCHA.
In addition to plenty of stuff I cant think of, you could see:
1) High percentage posts flagged as 'reported'
2) High percentage posts flagged as 'spam'
3) High volume of posts containing URLs
4) Posts containing identified spam messages (like how an email spam filter may work)
5) Multiple accounts from similar/same IPs
Almost all spam in a subreddit I mod for is from accounts < week old. (I believe there may already be a limit on how much you can post after registration though)
I mod a subreddit with >100k subscribers. We use AutoModerator and have it set to remove anything posted by an account less than a day old.
It catches some false positives from people who sign up just to post something, but it's also drastically decreased the spam. Most spambots are still at the "sign up and then blast shotgun-loads of posts before the algorithms catch you" level of behavior.
Most people never notice. The ones who do get an explanation of what happened when they ask about it in modmail.
It can be rough on a brand-new account that wanted to jump into a discussion right away, but at the same time the quantity of spam hitting your subreddit once you pass the 100k mark is just ludicrous, and this is by far the most effective way to stop it.
IIRC they probably mean to detect them through how they operate on the website, rather than how they register (tracking navigation, how they upvote/downvote, and what content they post).
Simply testing for javascript still works pretty well for drive-by bots. For ones targeting Reddit specifically the captcha probably wasn't doing much anyway.
Yeah, I hope internally they projected how many people were dying on the CAPTCHA, how many were believed to be bots, and then looked at the results through those filters.
I agree. While it does show an average increase of ~7.5% after the line, it also showed that before the line! It's hard to tell that anything happened.
It's refreshing to see what is to my eyes clearly a cleaner and more modern design actually performing better than what looks like a kludgy FrontPage design. Too often these analyses purport to show the opposite.
However, the Nielsen Norman Group (NNG) recommend against using placeholder text in form fields. They list 7 main reasons. This is taken from their website
1. Disappearing placeholder text strains users' short-term memory
2. Without labels, users cannot check their work before submitting a form
3. When error messages occur, people don’t know how to fix the problem
4. Placeholder text that disappears when the cursor is placed in a form field is irritating for users navigating with the keyboard
5. Fields with stuff in them are less noticeable.
6. Users may mistake a placeholder for data that was automatically filled in
7. Occasionally users have to delete placeholder text manually
Obviously, not all of these will apply to the Reddit sign-up form (or to your own website). It's worth reading the full NNG article to get a different but valid perspective based on their research:
I wish I had a screenshot of the original version. It was absolutely awful. It seemed like the code wasn't touched in at least 5 years.
Nice job with the redesigned version, but in all seriousness... I wonder what took so long? Every few months a company should take a spin as a new user and find ways to refine the new-user process.
Use this as a reminder to take another look at your registration sign-up process in 2015.
This is a typically poor example of ignoring the complexities inherent in analysing new user acquisition.
Suppose you implement a change that causes the registration form to crash for some subset of people? I am sorry to say for a while I was responsible for causing such issues at a very large service ;) Depending where the crash occurs and where you are counting signups you may actually get a massive boost in your metrics, as the victims retry with every attempt met by failure.
Suppose you correct an issue making it easier for people to login? Then lower registrations - the key here is to make sure you have a good metric to look at (e.g. accounts created that are interacting well with the site after as long a window as possible).
An account registration is basically like a hit metric.
The creation of an account does not imply that it will be used or the person who created it has not created many more either accidentally or for example because they could not log in.
IMO the design of Reddit's website could use an overhaul. I can't even snap it to one side of my screen without totally screwing up the front page and forget about using the site on mobile. The sidebars contain way too much unnecessary information as well. I do love me some reddit though.
any way to check which ones are real people and which ones are bots creating accounts for some reason ? maybe their activity posting vs subreddit popularity... I guess you may be surprised.
My sarcasm detector is having trouble. Was your comment sarcasm? Was the original article and accompanying graph sarcasm? Is it sarcasm to say that unix time is incrementing at the same rate in 2015 as it did in 2014, and therefore it being 2015 is responsible for the increase in unix time?
I am just wondering how this rise is attributed to the CSS change? It may have had some effect, but I'm confused how this single graph says that it is because of a CSS change?