How A/B Split Testing Increased Our Conversion Rate by more than 100 Percent

“Unlock the secret to landing page optimization”… “6 things you aren’t doing that are killing your conversion rate…” “Do this one trick to get your viewers to do anything, and I mean anything…”

I got pretty tired of reading countless blogs and forums that touted the alleged “secrets” of landing page optimization. The Internet is full of people advertising short cuts and quick wins, and following those tips didn’t really seem to help our business. We needed meaningful website optimization that actually helped us get more paying customers.

We had just launched Vendevor.com, a shopping cart and payment processing solution that you can add into your website and Facebook page, and we were in a scramble to not waste our initial marketing dollars on a terribly-converting website. Our team had to learn to effectively split test and optimize our site, and we had to do it fast if we were going to close our next round of funding that would take us to the next level.

This is an honest narrative on what we did (and are still doing) at Vendevor to really impact our conversion rates, drop our cost per acquisition, and grow our paying user base.

Our “aha!” moment in website optimization wasn’t changing a call-to-action button color like you read about online, or some tiny text change (although you’ll see those definitely helped), but rather creating intuitive pages that naturally direct the viewer to perform the action you want.

I’d like to walk you through what we did. And it all starts with this first rule:

Optimize with a specific goal in mind.

By definition, if you’re optimizing a webpage, you must be optimizing for something. That’s the whole point of the process. At first, we were optimizing for a simple conversion rate: the percentage of people that hit our site would sign up for a free trial of Vendevor. And here is the landing page we started with (in all of its terrible-ness):

Screenshot 1

We spent some time (okay, a LOT of time) at Panera Bread Company pouring over that landing page, trying to come up with a clever, pithy slogan to convince the world that we were going to be the easiest eCommerce plugin they’d ever seen.

What we got was that people were severely confused when they saw our site. And they left.

People couldn’t gather what we did from that one page, and there wasn’t even a call to action or place to learn more. In fact, we were asking for more information about that visitor (even if it was just an email address) before even giving them anything or any information in exchange.

This taught us our first lesson about what to optimize for. We were hardly monitoring the conversion rate of a viewer into a free trial. We were convinced that the page we had was brilliant because it made sense to us.

So, after about two months of no traction we finally admitted that we had to try new designs and also laser-focus on two or three metrics that actually mattered when split testing our homepage. We knew that the ultimate metric was visitor-to-signup conversion rate.

We also decided that there were a few secondary, but still very important, metrics to track that would weigh in our decision of deciding the winner of each A/B split test. For example, while we were optimizing for the conversion rate of visitor-to-signup, a major contributing factor of that conversion rate was the bounce rate on our site (percentage of people that only view one page of the site and then leave). If we couldn’t get people to stay on the site or look around, there was a pretty bad chance that visitor was going to turn into a free trial member, much less a paying customer. So the bounce rate would be important to minimize.

Lastly, the average number of page views was the third metric we would consider. Why is this important? As an example, we later learned that if we could get a visitor to view the pricing page, they were 40% more likely to sign up for a trial. So, the more pages we could get the visitor on, the more likely they were to sign up.

We had our three metrics we wanted to track and solely focus on, but we weren’t entirely sure what to change about our site. We had some guesses, but we learned the hard way that our first guess on design and text for our homepage was completely wrong.

So, we took the advice of a fellow entrepreneur and decided to implement the following incredibly valuable lesson in A/B testing, which became our second major realization in optimization:

Imitate before you innovate.

Our drag and drop shopping cart technology was a relatively new model, and so we were initially dead set on explaining our company in a new way (hence the almost clever “selling online seem complicated?” line). And that got us nowhere, except $25k down the drain in poorly performing ads.

Rather than innovating the way we communicated our value proposition on the site, we then began to mimic some of the major players in our industry. The way they laid out their page, where their call to action was, and some of the wording that they used. This was following the assumption that these companies have infinitely more resources to optimize their pages with, and so we could be confident that what the landing pages they were using at the time were converting, and converting well.

(Tip: want to tell if your competitor is split testing their page? Visit their site very often and occasionally you will see additional text in the URL that contains some sort of “A” or “B” page (sometimes numbered 1, 2, 3, etc.) and you can type in the next logical number in that progression and you’ll often get their next split test. We did this with BigCommerce)

So, we began to mimic the landing page layout that our competitor was using at the time. Here was the result:

Screenshot 2

Thirty-nine percent improvement. Not bad, now we were getting somewhere. It was a start, at least. Lesson learned – If you don’t know where to start, imitate before you innovate.

This test proved that a concise and to-the-point headline, as well as some social proof (showing the number of signups we had at the time) helped communicate what Vendevor had to offer and that other people found value in it. These aren’t optimization “secrets”, but rather just intuitive principles of persuasion in our value proposition, mixed with minimal text and a simple call to action.

Side note: If you aren’t currently using CrazyEgg.com or any other popular heat map tracking tool, it’s well worth the dollars. About this time in the optimization process we started implementing click tracking and heat mapping tools to know 1) where the majority of visitors were clicking on each page, and 2) how far down the page visitors would typically go. This would help us know where we should spend the most time split testing pages based on where people were clicking and how much of the page they were actually viewing. There isn’t any point in split testing a part of the page that only 10% of your viewers end up seeing. Time is precious, so split test the most-viewed pages and sections first.

After this last split test, we looked at our average number of page views and average time spent on the site. From the high bounce rate, we decided to try highlighting the headline text to bring more attention to our value proposition, as well as adding a video that concisely explained Vendevor. We figured that if we could get people to understand what Vendevor did in 60 seconds or less, then we could get them to look at more pages of the site. The net result of these experiments was interesting:

Screenshot 3

From this we learned two lessons in optimization that we have implemented on other pages as well:

Get to your value proposition as quickly as possible.

The big video link that explains Vendevor in less than one minute was in the middle of the page where you couldn’t miss it, and if you watched the video, you most likely understood what Vendevor had to offer. The text was highlighted so that it grabbed your attention, and in one sentence you knew what Vendevor would allow you to do (that was the goal, anyway).

The trend we were starting to realize was that the optimization process wasn’t fundamentally based on button colors, etc, but rather creating intuitive pages that concisely explain your value proposition, allow the viewer to navigate your website in a natural progression, and convinces them to sign up. We were realizing that optimization was more about intuitive design than unlocking “secrets” of A/B testing that could blow up a conversion rate. We started to follow this checklist:

  • Will the viewer be able to understand what Vendevor is and does in less than sixty seconds?
  • Are there any dead-ends on a page where we don’t prompt them with the next logical action? (sign up, link to another page, etc.)
  • Are we using at least three principles of persuasion on each page? (Social proof, Reciprocity, Authority, Scarcity, Commitment, Liking. Read more here: http://www.influenceatwork.com/wp-content/uploads/2012/02/E_Brand_principles.pdf)

Lastly, we began adding in examples of our product in action, so that the words of our website explaining what Vendevor had to offer was directly supported by images. We asked ourselves, what if instead of a smiling face and a description of Vendevor, we had a description of Vendevor with a photo that matched that description? Seems intuitive, right? The results yet again were interesting.

Screenshot 4

We’re still gathering the data, but as of now but we’re seeing anywhere between a 15-100% increase in conversion rate, 30-50% more page views per visit, and 50% longer average visit on the website just from this one above the fold change.

Once we were getting better at optimizing our front-end sales pages, we were still having trouble converting many of our initial sign-ups into paying accounts. After sign up, we needed a little more information to be able to process credit card payments on customer’s behalf, and gathering that information seemed to be a dead end for about half of our sign ups.

We tried split testing the required form right after sign up by adding testimonials, security icons, and more, and none of it worked. We actually got a 10-20% lower conversion rate by adding these elements. All we could figure was that these persuasion elements were creating a red flag in the user’s mind that they were trying to be sold on something that they didn’t yet understand. The fact that we were having to convince them to fill it out the form by containing security icons and testimonials was reason enough for them to not fill out the form at all.

We couldn’t seem to run a winning split test on this form to save our lives. We were so concerned with the design of this form rather than reconsidering the timing of the form itself. This is when we learned another breakthrough lesson in optimization:

Ask for as little information as possible, as late as possible.

We were asking for personal information without ever giving the user anything in return.

So, we decided to run a test and make this initial form optional. You could click “I’ll fill this out later, I’m just looking around for now.” They would still be prompted for this information before they could process sales, but not before.

What happened? A 30% increase in conversion rate in completing this form.

Essentially this taught us to wait to ask for more information from the user until it is absolutely necessary to continue. This gives the user a chance to commit to the product before they are asked for personal information.

Twenty percent here and thirty percent there might not sound like breakthrough optimization (like the “1800% improvement” articles you see online), but as you gradually split test and optimize each part of the sales funnel, the results compound on one another. It doesn’t take more than a few of these improvements to double or triple your end-to-end conversion rate in a relatively short amount of time.

So there you have it, the story of our early days of optimizing Vendevor’s sales funnel. To us, it hasn’t been unlocking “secrets” but more about creating an intuitive website flow that concisely conveys our value proposition and creates the fewest barriers possible in converting a viewer into a paying customer.

Bonus tips:

Have you split tested your pricing lately? Several months back we added another premium plan (our most expensive yet) to see how that would influence our clients’ decision to pick a certain payment plan. The simple effect of having a “vanity” plan to bring pricing context to your other plans is worth testing, as we discovered. $29 might sound like a lot if it’s your top tier, but $29 doesn’t sound like a lot if it’s sitting next to a $79 and $139 plan. Here’s what we found:

Original pricing:

Basic: $9/month
Pro: $29/month
Enterprise: $79/month

New Pricing:

Basic: $9/month
Pro: $29/month
Enterprise: $79/month
Executive: $199/month

The result was staggering. When we added the Executive plan into the pricing chart, the percentage of users choosing the Pro plan increased by 18%. This increased our average revenue per user with almost zero effort on our part. Yet another thing to split test!

Helpful tools to get you started: