Improving sign ups by 125%

As we’ve been continuing our work on Raygun it’s become important to look at improving the conversion flow from visitor to sign up to paying customer. Whenever you create something new you start at ground zero – you’ve often sacrificed some conventional wisdom to simply get a product out the door. You start collecting metrics and then you work hard on improving those metrics.

I wanted to share one of our recent tests and show how little a change can be for it to have a dramatic impact on conversions.

One of the recent tests we ran was on optimizing the signup page. We were getting good click throughs to sign up but a lot of people were then leaving. That seemed quite strange – by the time you’ve clicked to signup you are showing intent to actually sign up. Fall off from this page should be relatively small based on my experience.

Here’s our control, the current design of the page:

Here’s the 3 versions I create to test against.

1. Changing the title to ‘Sign up in seconds’

2. Changing the title to ‘Sign up in under 1 minute!’

3. Changing the button to say ‘Create my account now!’

That’s all. I didn’t change anything else. A common mistake that is made with A/B Testing is testing too many changes at once so you don’t actually know which change impacted the conversion rate (there’s another topic, multi-variate testing but that’s beyond the scope of this post).

Here are the results

“Sign up in seconds” headline 125% higher conversion to a trial user.

“Create my account now” button 100% higher conversion to a trial user.

“Start a free trial in under a minute” 62% higher conversion to a trial user.

Several things are worth noting here:

• The winner, the ‘Sign up in seconds’ version, has a confidence level of 96%. It could be better but it’s good enough for now.
• After rolling ‘Sign up in seconds’ out permanently I’m going to run another test with the button change and without it. I’m curious to see if it boosts the conversion rate even higher by combining the two. Throwing a button change version in was just something I thought could yield an interesting result vs. my control.

I personally am amazed at the level of impact simply changing the heading was. You may be the same – I wouldn’t consciously think I’d change my behaviour based on something so small, but it seems we all do!

There’s plenty more we can add to this page to keep improving it. Here’s some of the ideas I have for now and I’d appreciate your thoughts in the comments on what we could test too!

• Add a note that no credit card is required for the trial. I hate services that do this but we should spell it out. Perhaps people think the next step might be to ask for a credit card?
• Add some copy about the benefits of the service to this sign up – remind the user why they clicked the button in the first place.
• Mentioning something about security, reassuring the user that their details are safe.

That’s just one test we’ve run and we’re running plenty of others. I hope you’ve found it interesting getting insight into the process we’re using.

Oh, and if you’re looking to do your own A/B Testing, I highly recommend Visual Website Optimizer. We’ve used it for years on the Mindscape website and it’s a pleasure to use. So simple and easy to use and yet some really powerful result. It also helps ensure you’re seeing a statistically significant result without having to do the math yourself.