Amazon Split Testing

With almost 500 million products on Amazon, how do you stand out from the crowd? Split testing is one of the most effective means to optimize your Amazon product listings and increase engagement and sales on your products.

What is split testing on Amazon?

Split testing on Amazon is an experiment between two or more variations to see which variation in the set performs best. For Amazon split testing, you might test variations of your product name, its featured photo, or the product description. Split testing is also known as A/B testing because there is a version called A and a version called B. (If you add a third and fourth version, it’s called an A/B/C/D test).

Amazon split testing tool

Traditional split tests direct a portion of your Amazon traffic to one test variation and another portion to a different variation. After the test, you compare how shoppers behaved with each variation. For instance, on average, did one set of visitors browse for longer? Did one set convert at a higher rate?

Many software tools exist to help sellers run split testing on Amazon, including Splitly, Listing Dojo, and Cashcowpro.

However, the traditional Amazon split tests that these tools offer have some limitations:

  • For one, in order to gather enough data to make an informed decision, you have to accrue enough traffic to test.
  • In order to do this, you’ll probably need to pay to promote your products on Amazon, which can become expensive.
  • The tests also may take quite a bit of time, depending on the traffic levels.
  • Further, because you’re using live traffic for your test, you risk seeing an adverse effect on sales if a test variation performs worse than your current listing.

Split testing on Amazon using PickFu

PickFu is a polling software that enables you to quickly test elements of your Amazon listing with audiences that match your customer profile and target demographics.

Differences between live split testing and PickFu polling

  • Traditional split tests use Amazon shoppers as they are shopping as test subjects.
    • The PickFu Panel is comprised of U.S.-based respondents who are paid a small stipend to answer your poll. A popular audience available to test on PickFu is Amazon Prime members. However, a PickFu split test is conducted outside of the Amazon marketplace.
  • When running a traditional split test, Amazon shoppers are not aware that their behavior is being monitored. They do not know that there are other versions of your listing other than the one they are seeing.
    • On PickFu, the Panel sees all variations in order to compare them and provide their feedback. Not only does each respondent vote on the variation they like best, but they each also provide a written comment explaining why the chose their preferred option. These written comments provide insight into what works with shoppers to help guide your decisions.
  • Traditional split tests often take days or weeks to complete.
    • Using PickFu, most polls complete in less than 15 minutes.
  • The cost of traditional split testing on Amazon is often unpredictable. You need to pay for the testing software you’re using to split test, as well as the cost of the ads you’re running to build traffic. In addition, you risk losing revenue with a live test if one version has an adverse effect on sales.
    • With PickFu, the cost is a flat fee. Polling starts at $50 but changes with different options you can use to customize your poll.
  • Traditional split tests use live Amazon listings.
    • On PickFu, your product does not need to be live on Amazon. You can test product variations such as color options or design before going into production. Get a sense of what shoppers want before committing to inventory.
  • Traditional split tests can only compare variations of your own Amazon listings.
    • Using PickFu, you can perform competitive analysis by polling your listing against competing listings. Gain insight into how your product fares against category leaders.

What can you split test?

Using PickFu, you can test every element of your Amazon listings, such as

  • product names
  • photography, especially the listing’s featured photo
  • product descriptions
  • packaging design
  • product videos
  • product bonuses or add-ons
  • ad headlines and copy

Examples of Amazon split tests

Split tests of product photos

amazon split testing product photos
A split test with 50 general population respondents that took 26 minutes. See full results here
amazon product photo split test
A split test conducted with 50 Amazon Prime members. Results took 30 minutes.
Amazon split testing product photos
Split test of exercise equipment with 50 females. Can you guess which one won?

Split tests of physical product design

Split test of subscription box design
50-respondent poll for product design. Results took 24 minutes.
4-option product design poll with 50 females. Results came back in under an hour.
50-respondent poll for an olive oil dispenser. Results completed in only 13 minutes.

Split tests of product names and brand names

Split test with 50 females who were asked to rank three potential names for a natural skin care brand
Split test with 50 females asked to rank three potential names for a natural skin care brand
Split test of 4 names for an e-commerce store
50-respondent poll to rank names of an e-commerce store
50 respondents who live in suburbs voted in thishead-to-head poll for a high-end baby brand

You can also test product descriptions and whole product listings by including links in your polls. Want to see more e-commerce split tests? Explore our gallery of polls, then get started with your own Amazon split test!