Editor’s note: Thanks to Danny Carlson for this guest post on Amazon A/B testing.
As Amazon grows more and more competitive, a seller’s need for an edge gets greater every month. What if there was a strategy you could implement that made Amazon Ads cheaper, product launches more effective, and click-through and conversion rates higher? There is! It’s Amazon A/B testing.
Unlike your own e-commerce website, Amazon’s platform doesn’t allow you to install split testing plugins or tools. And messing with a listing too much can make the Amazon algorithm go haywire, putting your existing ranking at risk.
So how do we do it? In this article, I’ll dive into how to split test your listing, what to test, and how to gather the best data without messing up your Amazon indexing or ranking.
What is Amazon A/B testing?
In the most basic terms, the goal of a split test is to improve click-through rate, the percentage of shoppers who click on your product after finding it in search results, and conversion rate, the percentage of shoppers who buy your product after viewing the listing.
Improved click-through rates will boost traffic to your page. Improved conversion rates will boost your bottom line and make more efficient use of your traffic. This has an added benefit: on Amazon, every sale for you is one less sale for a competitor, boosting your keyword ranking instead of theirs. Amazon’s algorithm rewards listings with a high conversion rate with a better position on its search results.
How to split test on Amazon
Unlike a traditional split test you might run on your own website, software such as Hotjar or Google Analytics can’t be installed onto your Amazon pages. So you have two options to split test your Amazon listings:
- Manually rotate or use software like Listing Dojo to update your variations in a live test (Amazon does offer an internal tool called Manage Your Experiments for split testing A+ Content, but it’s only available to certain sellers)
- Run surveys outside of Amazon
Let’s discuss the pros and cons of each.
Live Amazon split tests
The upside of testing live on your Amazon listing is that the data comes from real shoppers making real purchases. At the end of the day, live tests are the only way to get live data.
However, to get trustworthy data, you have to alternate your A and B versions every day for at least two weeks. Why two weeks? Because unless you run your A/B test for a long time, your split test won’t take into account daily fluctuations in sales or events outside of your control.
Secondly, by frequently changing your listing’s text and images, you run the risk of messing with your keyword indexing and confusing shoppers who came back to look for your product again.
For this reason, I do not recommend live split tests for text, especially the title or any fields containing your main keywords. You might stop indexing for that keyword! Yikes.
Off-Amazon surveys
Using off-Amazon surveys has the upside of not messing with your Amazon listing at all. You can run multiple split tests simultaneously to get your data more quickly, then use that data to change your live listing.
Using a service like PickFu, you can target which demographic groups can respond to your survey. This way, your respondent pool is as close as possible to your actual customers.
Just keep in mind that because they’re not actual customers, their behavior might not perfectly reflect a shopper. Clicking an option on a web page is different than actually buying something. Often, people say one thing but when it comes to buying… crickets. Survey data is still useful, but don’t lean too heavily on it.
Highest-impact Amazon A/B tests
What are the first things every Amazon seller should test? Certain fields get much more attention, so making small improvements there can lead to big improvements in sales and ranking over time. To make the best use of your time, start with the most important elements — the ones that can get you five or 10 times the benefit and improve your bottom line.
Main image split tests
Without a doubt, the most important element to A/B test is your main image. It’s the biggest factor affecting click-through rates from search results pages, Amazon Ads, and your competitor’s listings (in sections like Customers Who Viewed This Item Also Viewed).
If your main image sucks, it almost doesn’t matter how good the rest of your listing is because no one will see it. So how do we optimize the main image?
I made a comprehensive step-by-step guide here, but I’ll cover the key points below:
- Make at least four variations that are as different as possible from each other, then, based on the winners, create more variations similar to those.
- Have the product take up the maximum amount of space possible.
- Position your product at 15 degrees. A 15-degree angle almost always performs better than a direct top-down view because it shows the product in three dimensions instead of two.
- Add a graphic reflection effect to make the product stand out.
- Have the background digitally removed so that it looks seamless.
- If your product has several pieces, shoot photos of each component separately, then place them together later in Photoshop. This enables you to more precisely and easily create layout variations. Just be sure to shoot each piece at the same angle or else they will look weird when placed together.
The title’s first 75 characters
After your main image, the next most important field to A/B test is your product’s title. Shoppers won’t click if they can’t quickly and easily know your product is the one they’re looking for.
Use your main search phrase in exact match at the beginning of your product title. This method helps for keyword ranking purposes, but it also clearly states what your product is. To find your main search phrase, look for a keyword that’s both high in search volume and can stand alone to explain your product. (This complex topic is one I cover deeply in this blog post).
After your main search phrase, include the most important descriptors about your product. What key details does your target customer need to see in order to know your product is the one they’re looking for? For some products, these important descriptors may be “BPA Free,” “Pack of 25,” or “Bluetooth 4 Compatible.” It varies depending on your product.
To identify what’s essential about your product, PickFu polls will offer useful insights. Try split testing four or five product title variants with survey respondents in your target market. This way, you don’t change your Amazon listing and risk its keyword indexing until you know it’s worth doing so.
Live Amazon A/B testing with PickFu data
Once you’ve done a few tests and gotten some good data from PickFu, it’s time to implement changes on your Amazon listing to see how it performs. Upload the poll’s winning image or text to your listing and make note of the date and current metrics in a spreadsheet so you know what your baseline is.
Check back the next day just to make sure nothing went totally haywire, but as long as your keyword indexing is fine and conversion rate didn’t tank, leave it to gather data.
A week later, check your business reports and find your Unit Session Percentage. Did it go up? Down? Stay the same? How does it compare to your PickFu survey results?
Note: Refer to Unit Session Percentage instead of what Amazon calls the conversion rate. Unit Session Percentage varies slightly from your conversion rate because Amazon directly calculates the percentage of units sold to listing impressions. This means that orders with multiple units will skew the percentage to look higher than your true conversion rate, but generally not enough to worry about.
You’ll also want to check your click-through rates by downloading a Search Term Report from Amazon Ads. Download a Search Term Report for the week leading up to your change, then one for the week following. Run a basic calculation to calculate the average click-through rates from each period. To get good data here, you need to have run ads for a while with at least $3,000-$5000 per month in ad spend to see any marked difference.
If you meet these ad criteria, ask yourself the same questions about the click-through rate: Did it go up? Down? Stay the same? How does it compare to your PickFu survey results?
Summary
Not many strategies can have such a profound impact as split testing can. Amazon A/B testing may not be as sexy as the new marketing Manychat bot or Google Ads funnel, but slow and steady results will make a big difference.
Think of testing’s ROI like compound interest. At first, the small improvements don’t add up to much, but over time improvements stack on each other and the difference grows exponentially bigger.
Danny Carlson is a 27-year-old entrepreneur, podcaster, and lifestyle architect. In 2014, he started his first business producing extreme downhill longboarding videos. He has since grown the Amazon FBA Agency Kenji ROI to more than 10 team members. Kenji ROI has served over 500 Amazon sellers by creating photos, video, copywriting, and graphics for over 1,240 listings.
Danny is also the host of the Actualize Freedom Podcast and the Danny Carlson Podcast. He has completed more than 60 interviews with names like Daniel DiPiazza, Steve Sims, Manny Coats, and Kevin King.
Residing in Bali, his off time is spent doing standing acrobatics, ripping sport motorcycles, and training at Nirvana Strength.
