An Amazon A/B testing tool helps e-sellers figure out exactly what element in their product listing drives customers to act. In a traditional Amazon split test, sellers run variants of their product listing, collecting data over time to figure out which listing garners more conversions and sales.
These four tools can help you carry out an Amazon A/B test. The first three gather data from live tests on the Amazon marketplace. The data comes from actual customers who browse your products, although they do not know their shopping behavior is being monitored for a test.
The fourth helps you choose which variants (such as title, images, and description) customers will like best before you go live with your product and outside of the Amazon marketplace, giving you more control.
Table of contents
- Splitly: a focused A/B testing tool
- Cashcowpro: an A/B testing tool with other bells and whistles
- Listing Dojo: a free Amazon A/B testing tool
- PickFu: an A/B testing tool with personalized feedback
- 3 more Amazon A/B testing tools to consider
- What happened to Splitly?
- Frequently asked questions
Splitly: a focused A/B testing tool
Splitly enables you to create variants of your listing, then automatically rotates the variants live on your Amazon listing. Using Splitly, you can split-test the following elements:
- Product features
- Images, including image order
- Back-end keywords
Splitly runs one product variation at a time. After several days, it begins reporting results back to you, and when it finds a statistically significant winner, it ends the test, leaving the winner as your listing.
Editor’s note: Splitly was discontinued on Sept. 15, 2021.
Benefits of Splitly
If you’re a visual learner, you’ll love the graphs Splitly creates to show you how your products are performing. Splitly tracks your average daily sessions, average daily sales, conversion rate, and average daily profit.
Beyond A/B testing, Splitly helps you track how you rank for keywords in Amazon’s product algorithm. On the higher plans, you can also experiment with pricing changes that react to competitor events such as going out of stock or price drops.
Splitly starts at $47/month. On the Starter plan, you can run up to three concurrent tests and track up to 100 keywords. Plans increase in price and allowances from there, with the $497/month Enterprise Level offering the most concurrent tests and tracked keywords.
If you run into trouble while using Splitly, it offers email support. According to the site, the average response time is 2 hours and 15 minutes. Splitly’s documentation has plenty of resources about A/B testing and what it calls a “very relaxed refund policy” if you aren’t satisfied with your experience.
Disadvantages of Splitly
Splitly is AI-driven software, so you won’t get any feedback from customers about exactly what made them love a product variant. This forces you to trust data as-is and not know the reason one variant attracted more customers than another.
Splitly takes pains to tout that a winner is never declared until statistical significance is over 90%, but this also means that results are often inconclusive or take a long time.
Splitly does not offer free trials, so you have to commit to at least a month on the Starter plan to use it.
Who should use Splitly as an Amazon A/B testing tool?
Splitly is best for Amazon sellers who don’t want the hassle of running tests or tracking keywords manually.
Cashcowpro: an A/B testing tool with other bells and whistles
Like Splitly, Cashcowpro analyzes data by automatically running tests on variations of your existing Amazon product listing. Whereas Splitly rotates your listing variants daily at midnight, Cashcowpro alternates them more frequently.
Benefits of Cashcowpro
This Amazon A/B testing tool offers a 10-day free trial on its services, after which you can decide to cancel your subscription, pay $99.70 a month, or pay $997.00 for the year.
Unlike Splitly, Cashcowpro does not have plan tiers. Whether you subscribe monthly or yearly, you get everything Cashcowpro has to offer: sales data, keyword tracking, automated feedback collection, A/B testing, inventory monitoring, and more. Whereas Splitly owns several different software brands, including Jungle Scout and Jump Send, Cashcowpro aims to be an all-in-one Amazon solution.
This means that A/B tests aren’t necessarily the selling point of Cashcowpro. They may even seem like an afterthought, coming last on this list on the homepage:
The A/B test results on Cashcowpro are not presented in charts and graphs as they are in Splitly, but more of a list or spreadsheet.
One thing to note about Cashcowpro’s A/B tests: Unlike Splitly, which claims its algorithm takes into account outliers, PPC spend, and profit margin, the winner on Cashcowpro is determined on sales and sales alone.
Disadvantages of Cashcowpro
Like Splitly, the reasons for a variant winning can be indiscernible. Shoppers may behave differently on Monday than they do on Tuesday, but the algorithm does not take that into account. You have to have some faith that the winning variant will continue to perform as it did during the test.
Cashcowpro’s A/B test isn’t as comprehensive as Splitly’s. Cashcowpro only enables you to test the main product image (not the subsequent images or image order) and only up to 100 characters of your product description.
There are no pricing or keyword variants in Cashcowpro.
Who should use Cashcowpro as an Amazon A/B testing tool?
Cashcowpro is ideal for users who want a wide range of features beyond A/B testing to help them run their Amazon business.
Listing Dojo: a free Amazon A/B testing tool
Unlike Cashcowpro and Splitly, Listing Dojo devotes its full attention to Amazon A/B testing. It is the one and only service it offers. And, oh yeah, it’s free.
Benefits of Listing Dojo
Listing Dojo is part of Viral Launch’s suite of seller tools, which includes Amazon keyword research and competitor tracking. The most significant benefit of Listing Dojo is that it is 100% free. You don’t even enter a credit card.
Naturally, Listing Dojo is a hook for Viral Launch. The cost of Viral Launch depends on the plan you choose and the tools that are included. You can take advantage of plenty of other resources on its site, including e-books and free courses on Amazon PPC ads.
Like the other Amazon A/B testing tools, Listing Dojo enables you to select variations and then sit back, relax, and watch the results come in. You can test up to seven variants including price, title, images, and description.
The results come back in detailed metrics reports with easy-to-read charts and graphs. You can see the changes in clicks, sessions, conversions, units ordered, gross revenue, and net revenue.
Disadvantages of Listing Dojo
Listing Dojo does not use an algorithm to declare a winner for you, nor does it determine whether your test was statistically significant. Instead, you have to figure out your A/B test’s winner on your own after looking at the data. This can create muddy results that might not help you improve your product listing at all.
You cannot test keyword variants with Listing Dojo.
Finally, all free software comes with a catch — when the software is free, you (and your data) are the product. Listing Dojo is offered by Viral Launch, a company with a suite of paid Amazon seller tools. Listing Dojo is essentially a lead generator for Viral Launch’s other offerings, albeit a smart one. The pricing for Viral Launch’s paid plans ranges from $21 to $166.
Who should use Listing Dojo as an Amazon A/B testing tool?
Amazon sellers just dipping their toe in the split testing waters could benefit the most from using Listing Dojo, since it’s free. However, be careful to understand how Listing Dojo calculates the data so that you interpret results correctly.
It helps if you already have a certain amount of traffic and sales and you’re motivated to do the work digging into the data to uncover insights that’ll increase your listing’s profitability.
PickFu: an A/B testing tool with personalized feedback
One major drawback of any live test on Amazon is that it can negatively impact a product’s sales. If your variant performs worse than your original listing, you lose potential sales only to get frustrating answers.
In addition, live tests take days and weeks to see results. That’s all time lost on potentially lessened sales.
Enter PickFu, a polling software popular with Amazon sellers as an A/B testing tool. PickFu enables you to quickly test your creative ideas with audiences that match your customer profile and target demographics.
Benefits of PickFu
PickFu runs entirely outside of the Amazon marketplace. Your live listings are not affected.
This is a huge difference from traditional A/B testing, in which customers only see one variant. Since they don’t know that their behavior is being monitored, they aren’t active participants in the test and cannot compare versions to choose a favorite. Rather than rotating your variants on Amazon, you create a poll that includes your variants. PickFu then shows all of the choices to a group of respondents for feedback.
Results come back to you on the same day — often in as little as 15 minutes!
With PickFu, you can select specific audiences to poll in order to get the most accurate and helpful results. A popular audience is Amazon Prime members. You can also choose audiences based on their age, gender, exercise habits, and more.
Each respondent not only chooses their favorite option, but they also provide a written comment explaining why they chose it. PickFu is an effective way to test product titles, main images, and product descriptions.
After you discover which variants of your product PickFu respondents like best — and why — you can use the best versions on Amazon. Instead of potentially losing sales with an unsure live test, you can post your listing with confidence.
Disadvantages of PickFu
PickFu differs from other Amazon A/B testing tools in that its polls are run outside of the Amazon marketplace. Splitly, Cashcowpro, and Listing Dojo use live Amazon listings to gather data. With PickFu, you won’t compare click-through rates, impressions, or revenue.
You cannot test price or keyword variants or track rankings using PickFu. In order to affect your performance with Amazon’s product algorithm, you might need to run a live test in addition to a PickFu poll. The two testing styles can work together to give you comprehensive results that include both data- and human-driven feedback.
Who should use PickFu as an Amazon A/B testing tool?
Any Amazon seller can use PickFu to optimize product listings or solicit feedback about their products. PickFu is the only Amazon A/B testing tool that provides qualitative data from real people in addition to information about which product variant rises above the other, while allowing you to target specific demographics.
Steve Chou, the founder of online company BumbleBee Linens, ran a PickFu poll because he was tired of how long and often inconclusive his live split tests were.
He tested two featured photos with an audience of 50 females.
Within 20 minutes, Steve received comprehensive feedback and a clear winner: Option B. He listed the winning image on his site and on Amazon and saw the sales of that listing increase by 209%.
You, too, can run an A/B test on PickFu and find out what audiences have to say about your product. If you’re struggling with sales, an Amazon A/B test can help pinpoint problems with your product listing and highlight its strengths.
PickFu offers customer service through email correspondence and live web chats. Polls start at $50 for 50 respondents.
Start your experience with Amazon A/B testing tools by exploring PickFu’s poll gallery for more examples of e-commerce A/B tests. Then set your business up for success by running your own Amazon A/B test today!
3 more Amazon A/B testing tools to consider
FigPii offers A/B testing along with three other distinct tools for gathering site visitor data: heatmaps, video recordings of sessions, and online polls. With FigPii, you can test one page of your site or multiple pages. The platform determines a winner using Bayesian statistics but requires you to manually declare it in your dashboard to end a test.
FigPii offers a free 14-day trial for each of its paid plans, which are tailored toward how many monthly visitors you have. A/B testing is included in each of the four plans.
Trendle‘s offerings range from reverse ASIN to customer review alerts to split testing. Trendle’s platform lets you run up to 50 split tests at one time across all Amazon marketplaces. You can pop in at any point during a test to view sessions, conversions, and sales in action.
Trendle’s three main plans include all features, starting with the $10-a-month Newbie plan. As with competing tools, Trendle offers a free 14-day trial.
Manage Your Experiments
Manage Your Experiments is Amazon’s own split testing feature, which it launched in 2019. Up until that point, sellers could only use third-party tools such as the ones described in this article to split test inside the Amazon marketplace.
Amazon allows only a small subset of sellers who are brand owners with high-traffic ASINs to use Manage Your Experiments.
The program lets sellers simultaneously test two versions of published A+ Content on their listings. Amazon recommends that sellers run an experiment for 10 weeks in order to get the most out of the data.
What happened to Splitly?
JungleScout, which owns Splitly, shut down the split testing tool in September 2021. As JungleScout explained on its website, “we empower a community of more than 500,000 entrepreneurs and brands to launch and grow successful e-commerce businesses with our powerful, all-in-one solution. We wanted to focus our efforts wholeheartedly on that mission.”
Frequently asked questions
Split testing software makes A/B testing possible. E-commerce sellers invest in split testing software to get valuable data with the ultimate goal of optimizing their listings and improving click-through and conversion rates.
When you add variations to your listings — say, different colors or flavors of a product — it makes it that much easier for your customers to find what they want without having to click out and search elsewhere. Amazon Seller Central takes you through the steps of creating a new variation listing as well as adding to an existing listing.
In a word: no. You need to let an A/B test run its course with the product variations you’re testing. You’ll skew the results otherwise.
You can perform a live A/B test by manually rotating your Amazon listings or using one of the tools we’ve described above. You can also test outside of Amazon with a service like PickFu.
For the most reliable data, you should run a traditional A/B for at least two weeks. Keep in mind that you need a healthy amount of traffic in order to collect enough data. Anything less than two weeks won’t accurately account for daily fluctuations in sales and other factors.
Viral Launch’s paid plans start at $21 a month and go up from there. Listing Dojo, Viral Launch’s split testing tool, is free to use.
Related reading: Must-have tools for Amazon product listings and Amazon feedback tools