Why we built Synthetic Audiences, and when you still need real consumers

AI made it cheap to create options. You can generate 40 product image variations or 60 titles and taglines in a minute. But choosing the best one is still hard.

AI can only give you its best guess at what will perform. Live A/B testing can take weeks to collect enough data, and the whole time you’re losing out on conversions by showing a worse option to half your audience. Even testing with real consumers (our bread and butter at PickFu) can quickly add up in time and budget when you’re testing and analyzing dozens of options across hundreds of real people.

This is the problem my team has been working on for the better part of this year, and it's why we built Synthetic Audiences.

Synthetic Audiences gives you a fast prediction of what real people would choose, based on years of actual PickFu panelists’ survey responses. It’s now available to all PickFu users – free while it’s in beta, so try it out and let us know what you think.

This is a big shift in what we offer at PickFu, and I know a lot of our customers will have questions about it. Below I’ll dive into why we built Synthetic Audiences, how it works, how much you should trust it, and when we'd tell you to use real people instead.

TL;DR

• Synthetic Audiences gives you fast directional feedback on your concepts or options, built from over a decade of real PickFu respondents' past choices.
• Use it to narrow a long list or get an early read on a concept. Then confirm the decision with real people.
• Results come back in under a minute, and it's free while it's in beta.
• To try it, ask the PickFu Agent to run a synthetic test.

Related guide

Synthetic respondents vs. real consumers: what's the difference, and when should you use each?

Synthetic respondents offer speed and cost efficiency for early-stage market research tasks like screening ideas and refining surveys, but they cannot replace real consumers for high-stakes validation, measuring demand, or capturing lived experiences. Therefore, the best practice is to use synthetic methods for initial exploration and real consumers to make final, consequential business decisions.

Why we built it

PickFu has been around since 2008. That’s 18 years of helping companies of all sizes make better decisions with fast, affordable feedback from real consumers – the kind of insight that used to be locked behind slow, expensive research studies. Speed has always been core to what we do.

That focus hasn’t changed, but the landscape has. Teams can now ask ChatGPT or Claude to generate ideas and content, and ask which option is strongest or what their customers would think. It’s instant and free, with no extra test or budget to set up.

The problem is that general-purpose AI tools can’t actually predict what real people would choose. Ask one to be a dog owner and it invents one. It has no record of what any real dog owner has ever picked, and no way to check if its answer is right. Without that grounding, you’re getting plausible-sounding fiction.

We still believe feedback from real people is the best way to validate your products, brand, and creative. There’s no true replacement for an opinion from an actual person.

But we understand the appeal of a faster answer. Building surveys, recruiting respondents, collecting answers, and analyzing results all take time and budget. If AI could reliably predict how people will react to a product, package, or ad, teams could move faster and test more ideas earlier.

So rather than dismiss synthetic research, we wanted to see if we could build a version grounded in something real.

That’s where PickFu has an advantage. We have something general AI doesn't: years of real choices from real people on past PickFu surveys. Not just which options they chose and rejected, but why they made those choices, in their own words.

We also have a verified consumer panel. That means we can check a synthetic answer against what real people said on the same survey. Most synthetic-only tools can't do that.

So we started with two questions: if we build synthetic respondents from all of that data, how close can they get to real people’s answers? And where do they fall short?

We also set a clear bar before releasing anything: grounded in real data, validated against real human responses, and upfront about where it works and where it doesn’t.

How it works

Think of each synthetic respondent as a kind of digital twin of the people in your target audience. Each one is built from:

• The profiles of real PickFu panelists (around 200 demographic and attitudinal traits)
• The choices people like them made on past PickFu surveys in that category, including what they rejected and why

So a result shows themes and patterns across a type of customer, not one person’s opinion.

When you run a test, here's what happens:

1. You pick an audience. There are over 300, across 25 product categories. For example: dog owners in pet food, or women who take supplements daily.
2. The agent shows you the plan (audience, question, options, sample size) and waits for your OK.
3. We draw 30-50 synthetic respondents at random and ask each one separately: based on this profile and these past choices, which option would you pick, and why?
4. We count the votes. A result is dozens of separate predictions, not one AI's opinion.

There are two ways to run it:

• Comparison. A question and 2-10 options, text or images. You get the options in the stronger half and weaker half, and the themes behind the split.
• Single concept. One idea and one open question. You get a summary, the main themes, where different groups disagreed, and follow-up questions worth asking.

What we learned building it

A few things surprised us. Each one changed the product.

More history helped with text, but hurt with images. Each synthetic respondent draws on past survey answers from real panelists. Early on, we only gave each one 10 recent answers, which meant just 18% had enough relevant history to work with. When we raised that to 50 answers, it jumped to 96%, and synthetic respondents got noticeably better at predicting text choices like names and taglines. But they got worse at predicting image choices. That’s because they don’t see past images, only written descriptions of them, and more descriptions added more noise than signal. So today, they use up to 50 past answers for text tests and 10 for image tests. We wouldn’t have guessed that – our testing found it.

Synthetic respondents agree with each other more than real people do. Real people are messy and often split their votes. Synthetic respondents are more consistent, so they tend to pile onto the same answer. For example, if real people would split 60/40 between two options, synthetic respondents might split 96/4. They usually favor the same option real people would, but they make it look like a landslide when it’s actually much closer. That’s why results show which options landed in the stronger and weaker half, instead of a winner with an exact percentage.

Made-up personas didn't hold up. Our first version of the single-concept mode used broad personas, like “the value-conscious buyer.” They started from real data, but AI filled in the rest of the details. Their answers matched real people less often, and each new persona took a lot of manual work to build. Now both modes use synthetic respondents built from real panelists’ data instead.

Running the same test twice won’t give identical results. Each run picks a new random set of synthetic respondents, and AI models don’t give the exact same answer every time. Which options land in the stronger and weaker half, and the themes behind them, should stay about the same. The exact split moves between runs, which is one reason we don’t show one.

Why you can trust it (and how much)

You shouldn't trust it the way you'd trust 200 real people. But here's what you can trust it for:

• It's based on what people actually did. Synthetic respondents predict from recorded choices, not a guess about what a demographic might think.
• It's a vote, not one opinion. Dozens of separate predictions are counted, so one odd answer doesn't decide the result.
• We check our work. We benchmark it against hundreds of past PickFu surveys where we already know what real people chose. It picks the same side as real people most of the time on text comparisons, and less often on images. We keep testing and adjusting the models.
• Synthetic and human results never mix. Synthetic only runs when you ask for it, results are labeled synthetic, and they never get pooled with your human survey results.

That's good enough to cut weak options before you spend money on them. It's not good enough to settle a decision you're betting on.

When to use synthetic, and when to use real people

A simple rule: synthetic decides what to test further. Real people decide what to do.

The synthetic + human workflow

AI is producing more product descriptions, ad variations, and creative assets than ever before. But the end consumer of all that content is still a human. When anyone can generate endless variations, the real advantage is knowing which one actually resonates with your audience – and that insight has to come from real people. Human feedback matters more now, not less.

So our approach is augmentation, not replacement. We built Synthetic Audiences to work as one step in a sequence alongside human feedback:

1. Generate your options
2. Screen them with a synthetic audience
3. Refine the ones that survive
4. Confirm the decision with real people

After every synthetic result, the agent offers a ready-to-run survey with real consumers on your stronger options. You can skip it, but for any decision with real money behind it, we'd recommend running it. Real respondents give you written reasons, actual margins, and results you can show to someone else.

Synthetic gives you speed early on, when you’re sorting through lots of options. Real people give you confidence at the end, when you’re ready to commit.

Here’s how one beta tester described it:

“In two days, I ran over a hundred synthetic tests for two non-fiction books. I rebuilt a back cover on the first round’s findings and tested it again the same afternoon, so research became part of drafting rather than a verdict at the end. The value is in the reasons more than the rankings. And it gave me a short, specific list of what to put to PickFu’s human panel before my proposal went out.”

— Paul O’Dwyer, Consultant

Try it

To use Synthetic Audiences, open the Synthetic Audiences page in PickFu and start from one of the example prompts. You can also ask the PickFu Agent in the web app, Slack, or through the PickFu MCP in tools like Claude to run a synthetic test. It’s free while it’s in beta.

A few current limits:

• US audiences only. Every synthetic respondent is built from US panelists.
• No custom audiences. Audiences are pre-built from real panelist groups.
• One audience per test. You can't combine audiences yet.

The top requests we're hearing so far are customizable audience options and markets outside the US. Keep the feedback coming! You can ask the Agent to submit feedback while you’re chatting with it, or reach out to me directly via email or LinkedIn.

Log into PickFu (or create your free account) and try your first synthetic test today.