Split testing (sometimes called A/B testing or multivariate testing) is a method of gathering data to help you iterate on ideas and designs. You might use split testing to improve your website, book cover, or app icon, to name a few. Because split testing can often be confusing, and because so many terms are used interchangeably, let’s go over in detail everything you need to know to run an effective split test.
A traditional split test directs a portion of your website traffic to one version of a web page and directs another portion of your website traffic to a different version of that page. In doing so, you can compare how website visitors in each group behave on each web page.
Split testing and A/B testing are used interchangeably. Both these terms mean that you are testing two versions of a website (Version A and Version B). If you add a third and fourth option, it’s now an A/B/C/D test.
Multivariate testing uses the same mechanism of an A/B test but compares more variables. A true A/B test only tests one variable (for instance, a blue call-to-action button vs. a red call-to-action button). But in a multivariate test, the headline, copy, call-to-action, and images on the website might all be different.
Even though A/B tests and multivariate tests are different, the term split testing is commonly used to describe either one.
Using split testing, you can see how various website layouts, imagery, copy, offers, or calls-to-action affect user behavior on your website. For instance, you could take your existing landing page (the control), and test it against a totally redesigned landing page. Then, using the data of the split test, you can answer questions such as
Generally, to conduct a split test, you direct half your traffic to Version A of your website, and the other half of your traffic to version B of your website. You control the landing page URL through a software such as Google Ads, Facebook Ads, or a dedicated landing page software. If you’re technically savvy, you could also direct portions of your website traffic programmatically.
Traditional website split testing has its limitations. For one, in order to gather enough data to make informed decisions, your website has to have quite a bit of traffic to test. If you’re using ads on Google or Facebook to drive this traffic, the effort to split test can become expensive. Split testing also takes quite a bit of time.
Steve Chou, who runs the e-commerce site Bumblee Linens, knows these limitations first hand. He would publish a listing on his site, wait several days, swap out the images for new ones, wait again, and then compare the results. The problem with split testing, he said, “is that it takes forever. Every test that I run usually takes at least three weeks or more. And I’d say that nine times out of ten, my tests are inconclusive.”
Besides the time required, a conversion pixel or some kind of tracking mechanism is needed, adding complexity and hassle to the tests, especially when selling on sites like Amazon, eBay, or Etsy. “Not only is this a major pain in the butt if you have multiple listings,” he said, “but if you’re lazy like me, you’re never going to do it.”
So Steve tried an alternative: PickFu. “I thought I’d give it a try just for fun,” he said.
He ran a poll on PickFu to test two featured product photos. The results took less than 20 minutes to come in. After featuring the winning photo from the PickFu poll on his site, sales of that listing increased by 209%.
Even though traditional split tests and PickFu polls differ in these important ways, it’s common parlance to use the terms split testing and A/B testing when referring to PickFu polling. Here at PickFu, we also call our offerings “split testing” or “A/B testing,” but we acknowledge the differences we’ve outlined here.
Using the wider meaning of split testing described in the section above, what you can split test becomes much broader than only versions of websites. With PickFu, you can split test any creative idea. Some examples of what you can split test include the following:
Using PickFu to split test your ideas, it’s quick and easy to iterate new and improved versions of those ideas. The written comments help you understand how customers react. PickFu provides both quantitative data and qualitative feedback.
The feedback you receive has the potential to pay for itself. As Manny Coats, CEO of Helium10 notes, “PickFu.com is where I go when I want to quickly split test Amazon product images to actual Amazon Prime customers, and know which images customers prefer, before I make my product listing live. I get results within one hour with detailed ‘reasons why’ they voted a certain way. This has the potential to save thousands of dollars in lost sales from choosing the wrong image.”
In addition, PickFu can surface concerns of customers that you might not have foreseen. Author Mike Fishbein said, “I had a radical idea to use a 1970s-style motif for the [book] title and cover.” After brainstorming many ideas, he was most excited to title his book Pimp Your Book: How to Self-Publish a Bestseller on Amazon. “I thought it would add character and be attention-grabbing, [but] when I tested it on PickFu, I learned that readers found it unappealing and tacky.”
PickFu is faster than other split testing alternatives. As technical product manager Isabella Patton said, “We chose PickFu because of the quality of the users and the speed at which we get results! Other online survey services take up to a week to complete, and we just don’t have that time… Since it takes less than 15 minutes to get the survey results, we were able to alter our designs immediately and re-test them.”
Depending on your method of testing, costs vary. For instance, if you’re using a platform such as Google Ads or Facebook Ads to split test, you’ll pay for clicks. You also need to consider the time and effort required to set up a split test on your website.
On PickFu, a 50-response poll starts at $50. You can use available features and options to customize your poll and make it more valuable to your business. For instance, with audience targeting, you can only poll people who match certain demographic or behavioral traits, such as women, iOS users, Amazon Prime members, homeowners, or a combination of up to three traits. Use our Pricing Calculator to determine the cost of your poll.
Visit PickFu.com now to create your first split test. It only takes a minute to set up, and you’ll start receiving unbiased answers to your questions immediately!