What’s the Best Amazon A/B Testing Tool for You?

An Amazon A/B testing tool helps e-sellers figure out what exactly about their product listing drives customers to act. In a traditional Amazon split test, sellers run variants of their product listing, collecting data over time to figure out which listing garners more conversions and sales.

These four tools can help you carry out an Amazon A/B test. The first three gather data from live tests on the Amazon marketplace. The data comes from actual customers who browse your products—although they do not know their shopping behavior is being monitored for a test.

The fourth helps you choose which variants (such as title, images, and description) customers will like best before you go live with your product and outside of the Amazon marketplace, giving you more control.

Contents

  1. Splitly: a focused A/B testing tool
    1. Benefits of Splitly
    2. Disadvantages of Splitly
    3. Who should use Splitly as an Amazon A/B testing tool?
  2. Cashcowpro: an A/B testing tool with other bells and whistles
    1. Benefits of Cashcowpro
    2. Disadvantages of Cashcowpro
    3. Who should use Cashcowpro as an Amazon A/B testing tool?
  3. Listing Dojo: a free Amazon A/B testing tool
    1. Benefits of Listing Dojo
    2. Disadvantages of Listing Dojo
    3. Who should use Listing Dojo as an Amazon A/B testing tool?
  4. PickFu: an A/B testing tool with personalized feedback
    1. Benefits of PickFu
    2. Disadvantages of PickFu
    3. Who should use PickFu as an Amazon A/B testing tool?

Splitly: a focused A/B testing tool

Splitly enables you to create variants of your listing, then it automatically rotates the variants live on your Amazon listing. Using Splitly, you can split test

  • Price
  • Product features
  • Images, including image order
  • Descriptions
  • Back-end keywords

Splitly runs one product variation at a time. After several days, it begins reporting results back to the seller, and when it finds a statistically significant winner, it ends the test, leaving the winner as your listing.

Benefits of Splitly

If you’re a visual learner, you’ll love the graphs Splitly creates to show you how your products are performing. Splitly tracks your average daily sessions, average daily sales, conversion rate, and average daily profit.

Splitly is an Amazon A/B testing tool
Source: Splitly

Beyond A/B testing, Splitly helps you track how you rank for keywords on Amazon. On the higher plans, you can also experiment with pricing changes that react to competitor events such as going out of stock or price drops.

Splitly starts at $47/month. On the Starter plan, you can run up to three concurrent tests and track up to 100 keywords. Plans increase in price and allowances from there, with the $497/month Enterprise Level offering the most concurrent tests and tracked keywords.

If you run into trouble while using Splitly, it offers email support. According to the site, the average response time is 2 hours and 15 minutes. Splitly’s documentation has plenty of resources about A/B testing and what it calls a “very relaxed refund policy” if you aren’t satisfied with your experience.

Disadvantages of Splitly

Splitly is AI-driven software, so you won’t get any feedback from customers about what exactly made them love a product variant. This forces you to trust data as-is and not know the reason one variant attracted more customers than another.

Splitly takes pains to tout that a winner is never declared until statistical significance is over 90%, but this also means that results are often inconclusive.

Splitly does not offer free trials, so you have to commit to at least a month on the Starter plan to use it.

Who should use Splitly as an Amazon A/B testing tool?

Splitly is best for Amazon sellers who don’t want the hassle of running tests or tracking keywords manually.

Cashcowpro: an A/B testing tool with other bells and whistles

Like Splitly, Cashcowpro analyzes data by automatically running tests on variations of your existing Amazon product listing. Whereas Splitly rotates your listing variants daily at midnight, Cashcowpro alternates them more frequently.

Benefits of Cashcowpro

This Amazon A/B testing tool offers a 10-day free trial on its services, after which you can decide to cancel your subscription, pay $99.70 a month, or pay $997.00 for the year.

Unlike Splitly, Cashcowpro does not have plan tiers. Whether you subscribe monthly or yearly, you get everything Cashcowpro has to offer: sales data, keyword tracking, automated feedback collection, A/B testing, inventory monitoring, and more. Whereas Splitly owns several different software brands, including Jungle Scout and Jump Send, Cashcowpro aims to be an all-in-one Amazon solution.

This means that A/B tests aren’t necessarily the selling point of Cashcowpro. They may even seem like an afterthought, coming last on this list on the homepage:

Cashcowpro includes an Amazon AB testing tool, but it feels like an afterthought

The A/B test results on Cashcowpro are not presented in charts and graphs as they are in Splitly, but more of a list or spreadsheet.

Cashcowpro's Amazon AB testing tool shows results like this
Source: Splitly

One thing to note about Cashcowpro’s A/B tests: Unlike Splitly, which claims its algorithm takes into account outliers, PPC spend, and profit margin, the winner on Cashcowpro is determined on sales and sales alone.

Disadvantages of Cashcowpro

Like Splitly, the reasons for a variant winning can be indiscernible. Shoppers may behave differently on Monday than they do on Tuesday, but the algorithm does not take that into account. You have to have some faith that the winning variant will continue to perform as it did during the test.

Cashcowpro’s A/B test isn’t as comprehensive as Splitly’s. Cashcowpro only enables you to test the main product image (not the subsequent images or image order) and only up to 100 characters of your product description.

There are no pricing or keyword variants in Cashcowpro.

Who should use Cashcowpro as an Amazon A/B testing tool?

Cashcowpro is ideal for users who want a wide range of features beyond A/B testing to help them run their Amazon business.

Listing Dojo: a free Amazon A/B testing tool

Unlike Cashcowpro and Splitly, Listing Dojo devotes its full attention to Amazon A/B testing. It is the one and only service it offers. And, oh yeah, it’s free.

Listing Dojo dashboard
Source: Listing Dojo

Benefits of Listing Dojo

The most significant benefit of Listing Dojo is that it is 100% free. You don’t even enter a credit card.

Like the other Amazon A/B testing tools, Listing Dojo enables you to select variations and then sit back, relax, and watch the results come in. You can test up to seven variants including price, title, images, and description.

The results will show you the changes in clicks, sessions, conversions, units ordered, gross revenue, and net revenue.

Disadvantages of Listing Dojo

Listing Dojo does not use an algorithm to declare a winner for you, nor does it determine a statistically insignificant test. Instead, you have to figure out your A/B test’s winner on your own after looking at the data. This can create muddy results that might not help you improve your product listing at all.

You cannot test keyword variants with Listing Dojo.

Finally, all free software comes with a catch — when the software is free, you (and your data) are the product. Listing Dojo is offered by Viral Launch, a company with a suite of paid Amazon seller tools. Listing Dojo is essentially a lead generator for Viral Launch’s other offerings, albeit a smart one.

Who should use Listing Dojo as an Amazon A/B testing tool?

Amazon sellers just dipping their toe in the split testing waters could benefit the most from using Listing Dojo, since it’s free. However, be careful to understand how Listing Dojo calculates the data so that you interpret results correctly.

PickFu: an A/B testing tool with personalized feedback

One major drawback of any live test on Amazon is that it can negatively impact a product’s sales. If your variant performs worse than your original listing, you lose potential sales only to get frustrating answers.

In addition, live tests take days and weeks to see results. That’s all time lost on potentially lessened sales.

Enter PickFu, a polling software popular with Amazon sellers as an A/B testing tool. PickFu enables you to quickly test your creative ideas with audiences that match your customer profile and target demographics.

Benefits of PickFu

PickFu runs entirely outside of the Amazon marketplace. Your live listings are not affected.

This is a huge difference from traditional A/B testing, in which customers only see one variant. Since they don’t know that their behavior is being monitored, they aren’t active participants in the test and cannot compare versions to choose a favorite. Rather than rotating your variants on Amazon, you create a poll that includes your variants. PickFu then shows them to a group of respondents for feedback.

Results come back to you on the same day — often in as little as 15 minutes!

With PickFu, you can select specific audiences to poll in order to get the most accurate and helpful results. A popular audience is Amazon Prime members. You can also choose audiences based on their age, gender, exercise habits, and more.

Each respondent not only chooses their favorite option, but they also provide a written comment explaining why they chose it. PickFu is an effective way to test product titles, main images, and product descriptions.

After you discover which variants of your product PickFu respondents like best — and why — you can use the best versions on Amazon. Instead of potentially losing sales with an unsure live test, you can post your listing with confidence.

Disadvantages of PickFu

PickFu differs from other Amazon A/B testing tools in that its polls are run outside of the Amazon marketplace. Splitly, Cashcowpro, and Listing Dojo use live Amazon listings to gather data. With PickFu, you won’t compare click-through rates, impressions, or revenue.

You cannot test price or keyword variants or track rankings using PickFu. In order to affect your performance with the Amazon algorithm, you might need to run a live test in addition to a PickFu poll. The two testing styles can work together to give you comprehensive results that include both data- and human-driven feedback.

Who should use PickFu as an Amazon A/B testing tool?

Any Amazon seller can use PickFu to optimize product listings or solicit feedback about their products. PickFu is the only Amazon A/B testing tool that provides qualitative data from real people in addition to information about which product variant rises above the other.

Steve Chou, the founder of online company BumbleBee Linens, ran a PickFu poll because he was tired of how long and often inconclusive his live split tests were.

He tested two featured photos with an audience of 50 females.

Within 20 minutes, Steve received comprehensive feedback and a clear winner: Option B. He listed the winning image on his site and on Amazon and saw the sales of that listing increase by 209%.

You too can run an A/B test on PickFu and find out what audiences have to say about your product. If you’re struggling with sales, an Amazon A/B test can help pinpoint problems with your product listing and highlight its strengths.

PickFu offers customer service through email correspondence and live web chats. Polls start at $50 for 50 respondents.

Start your experience with Amazon A/B testing tools by exploring PickFu’s gallery of polls for more examples of e-commerce A/B tests. Then set your business up for success by running your own Amazon A/B test today!


Learn More:Product vendors and eCommerce sellers use PickFu to optimize product listing pages by testing design concepts, photos, and descriptions with Amazon Prime member audiences.

Get 50 instant poll responses for $50.

Know what sells — even before selling it. 

Start Your Poll Now
Answers in minutes

Be the first to write a comment.

Your feedback