Every week you make calls that carry real money. Which colourway to cut from the winter range. Whether the new bundle should lead with the routine or the saving. Which of two hooks gets the next five thousand dollars of Meta spend.
What’s in This Article
Most founders make those calls on gut feel, a quick poll of the team, and a glance at what a competitor did last month. Then they find out the real answer three months later, in the sales data, after the purchase order has already landed.
The numbers on guessing are ugly. Nielsen research found 85% of new consumer packaged goods fail within their first 12 months. And when researchers pull apart those failures, one pattern keeps surfacing: 72% of failed products ignored customer feedback during development. The people who could have told you the answer were sitting in your customer list the entire time. Nobody asked them.
A customer panel fixes that. It is a hand-picked group of roughly 25 of your best customers who have agreed, in advance, to answer your questions fast. Built properly, it turns “I reckon” into “I checked” on every meaningful product, pricing, and creative decision you make. This playbook shows you how to build one in 30 days, for close to zero dollars.
What a Customer Panel Is (and What It Is Not)
A customer panel is a standing research instrument: small, curated, and on call. You ask a sharp question on Monday, and by Wednesday you have 20-plus answers from people who actually spend money with you.
It is worth being precise here, because a panel is not three other things that look similar:
- It is not a brand community. A community, like the ones we covered in the brand community playbook, is a large open space built for connection and retention. A panel is small, closed, and built to answer questions. LSKD runs both: a 95,000-member private Facebook group for connection, plus focus groups held through the year in existing and new locations to pressure-test product decisions.
- It is not a survey program. Surveys go wide and shallow. Typical B2C email surveys pull response rates around 16%, and you wait a week for a usable sample. A panel goes narrow and deep, and because every member opted in, response rates of 60 to 75% inside 48 hours are normal.
- It is not a set of one-off interviews. Founder-led interviews are for discovery: an hour of open questions that surfaces problems you did not know existed. A panel is for judgement: fast, forced-choice answers to decisions you already know you have to make.
The payoff for adding this kind of process is well documented. The Product Development and Management Association found that companies running best-practice research processes saw new products fail at 24%, against 46% for everyone else. Same markets, same categories, half the failure rate. The difference is process, and a panel is the cheapest process upgrade available to a Shopify brand.
Part 1: Recruit the Right 25 (Curated, Never Volunteers)
The fastest way to ruin a panel is to post “want to join our feedback crew?” on Instagram. Volunteers skew toward whoever has the most spare time: your loudest superfans and your serial complainers. Both are useful voices. Neither represents the customer who quietly orders four times a year and tells nobody.
Curate instead. Pull the list from your data, then invite by email, personally.
- Start with your top 10% of customers by lifetime spend. These are the people from the top 10% customer strategy: proven buyers whose behaviour already tells you they get the brand. Give them about 60% of the seats.
- Add recent first-time buyers. Around 25% of seats go to customers with one or two orders in the past 90 days. They still remember what almost stopped them buying, which your loyalists forgot years ago.
- Seat a few lapsed customers. The final 15%: people who bought twice and went quiet. They will tell you things your fans never will.
Size matters less than founders think. Twenty to thirty members is the sweet spot: large enough that a 62/38 split on a forced choice means something, small enough that every member feels the founder knows their name. Below 15, one loud opinion drags the whole average. Above 40, it stops feeling personal and response rates sag.
Expect 30 to 50% of a warm VIP list to accept, so invite about 40 people to seat 25. The invitation itself does a lot of work: name the number of seats, name why they specifically were chosen, and spell out the deal. More on the deal next.
Part 2: Give the Panel a Home and a Fair Deal
Glossier is the reference case here. The beauty brand invited 100 of its top customers into a private Slack channel, where they exchange more than 1,100 messages a week. When the team noticed the most requested product in that channel was a cream blush, that is what they built next. Their Milky Jelly cleanser went further still: more than 400 customers described their dream face wash in the comments of a single blog post, and the product was engineered from those answers. It became one of the brand’s best sellers.
You do not need Slack or a content site to copy the mechanics. You need a home, and a deal.
The home can be dead simple. An email-only panel run from a Klaviyo segment works from day one. If your panellists turn out to be chatty, a private Facebook group or a group chat adds texture. Start with email and upgrade only if the panel asks for it.
The deal is the part most founders fumble. Here is the exchange that works:
- What they get. First look at every new release, insider-only early access on launch day, a real vote on what gets made, and the occasional store credit as a thank-you. Status and influence are the wage; product is the bonus.
- What you expect. Two to three questions a month, 60 seconds each, answered within 48 hours. Say this in the invitation so nobody is surprised later.
- What you never do. Pay cash per answer. The moment answers become invoices, panellists start telling you what they think you want to hear, and the incentive attracts people who want the money rather than the brand.
The free build: Klaviyo plus Google Forms. Total software cost: zero. Here is the five-step setup:
- 1. Create the segment. In Klaviyo, add a custom profile property called panel_member and set it to true when someone accepts. Build a segment where panel_member equals true. That segment is your panel.
- 2. Build a question template. One Google Form: a single forced-choice question, an optional “why did you pick that?” text box, nothing else. Duplicate it for every new question so each stays a 60-second job.
- 3. Send through Klaviyo. A campaign to the panel segment with the subject line “48-hour question: [topic]”. Keep the email to three sentences and one button.
- 4. Collect in Sheets. Link the form to a Google Sheet so every answer lands timestamped, with a running tally you can read at a glance.
- 5. Close the loop. Reply to the panel with the outcome within a week: “You picked the sage green. It goes live 14 March.” Closing the loop is the single biggest driver of next-question response rate.
Part 3: Build the Question Engine
A panel lives or dies on question quality. The good news: there is a short list of question types that earn their keep, and a shorter list of rules for asking them.
What to run through the panel:
- Product calls. Which colourway, which scent, which of two prototypes, which name. BlackMilk, the Brisbane label whose releases regularly sell out within minutes, grew its whole design process this way: its “Sharkie” fan base has been part of design conversations since the brand started at a kitchen table in 2009, and its leadership maintains there is nothing more powerful than sitting down for a whole weekend and listening to what customers want.
- Creative calls. Two ad hooks, two homepage headlines, two email subject lines. Test before the media budget is spent, not after.
- Offer framing. Should the bundle lead with the routine or the saving? Gift with purchase or a straight discount? Free shipping threshold at 75 or 100 dollars?
- Comprehension checks. Show the new size guide or the reworked product page block and ask “what would you expect to happen if you tapped this?”
How to ask:
- Forced choice beats open-ended. “Which would you pick?” gets you a decision. “What do you think?” gets you a paragraph of politeness.
- Never ask “would you buy this?” Everyone says yes to hypotheticals. Ask “which of these two would you actually order first?” and make them choose.
- One decision per question. The moment a question tests two things at once, the answers stop meaning anything.
- Always include the why box. The verbatims are the real prize: they become product page copy, ad hooks, and email subject lines written in the customer’s own words.
- Show, do not describe. A flat-lay photo of two colourways beats a paragraph describing them every time.
Your first three questions set the tone, so make them count. A good opening pulse: one live product decision (two options, real stakes), one creative pick (two hooks for the next campaign), and one open door (“what almost stopped you ordering from us the first time?”). That mix teaches the panel that their answers change real things.
Cadence: one monthly pulse of two or three bundled questions, plus ad-hoc 48-hour sprints when a live decision comes up. Hard cap of four asks a month. Scarcity keeps response rates above 60%; over-asking burns the asset inside a quarter.
Part 4: The 48-Hour Sprint (Wiring the Panel Into Real Decisions)
The panel only earns money when it changes decisions. The mechanism is the 48-hour sprint, and it deserves a permanent slot in your operating rhythm, the same way a weekly ad review does.
- 1. Trigger. Any reversible decision worth about 2,000 dollars or more: a colourway on a purchase order, the hook on a new campaign, a bundle price frame.
- 2. Frame it as A or B. Two options, one image each, one forced choice, one why box. Nothing else.
- 3. Send Monday morning, close Wednesday evening. Forty-eight hours, stated plainly in the email.
- 4. Tally and read the whys. The split gives you direction; the verbatims tell you the reason behind it, and half of them are usable copy.
- 5. Decide, then tell the panel what you chose. Even when you overrule them. Especially when you overrule them: explaining why builds more trust than agreeing ever does.
Two guardrails keep the sprint honest:
- The panel informs; the data decides. A 62/38 split is a strong signal, not a verdict. For high-stakes calls, confirm with an A/B test on real traffic before committing the purchase order. Your panel is 26 people; your store sees thousands.
- Watch the echo chamber. Panellists already love you, so they are brilliant on product and retention questions and weaker on cold-audience questions. For an ad hook aimed at people who have never heard of you, treat the panel as a first filter and let Meta’s auction be the final judge.
And know when to leave the panel out of it. Margin structure, legal questions, supplier choices, and “should we raise prices?” are founder decisions. Customers will always vote for cheaper. Frame pricing questions as forced choices between two framings, never as a referendum on the price itself.
Part 5: Measure the Panel Like a Channel (and Refresh It)
Treat the panel like any other channel: a handful of numbers, checked on the first Monday of the month.
- Response rate: 60% or better. The opt-in should keep you well clear of the 16% that typical B2C email surveys manage. If it slips for two months running, you are over-asking or under-thanking.
- Median response time: under 24 hours. Speed is the whole point. Slow answers usually mean the questions have become homework.
- Decisions informed per month: two or more. If the log is empty, the problem is not the panel. It is that you are not routing decisions through it.
- The panellist halo. Track repeat rate and average order value of panellists against a matched group of similar customers. Expect the panel to pull ahead, because involvement breeds investment. LSKD’s community numbers show where this discipline ends up at scale: around 450,000 active customers and a repeat rate near 60%, built on relentless listening.
Then refresh. Rotate roughly a quarter of the seats every three months: retire members who missed three asks in a row, graduate long-serving panellists into your VIP or ambassador tier, and seat fresh first-time buyers so the panel keeps sounding like this year’s customer rather than last year’s. Pair the panel with an always-on NPS survey and you have both instruments covered: the wide read on every customer, and the fast, deep read from 26 who care.
The Four Mistakes That Kill a Customer Panel
- Recruiting volunteers. You get the loudest 25, not the truest 25, and every answer skews toward superfan enthusiasm.
- Asking vague questions. “Any thoughts on the new range?” produces polite noise. If a question cannot be answered in 60 seconds with a choice and a sentence, it is an interview question, not a panel question.
- Never closing the loop. Panellists who never hear the outcome stop answering within two cycles. Participation is bought with acknowledgement, not store credit.
- Treating the vote as market truth. The panel de-risks decisions; it does not make them. Skipping the confirming A/B test because “the panel said so” reintroduces the exact guessing you built the panel to remove.
The Compound Effect: What 26 People Are Worth Over a Year
Run the numbers on a two million dollar a year Shopify brand and the panel stops looking like a nice-to-have.
- One avoided flop. The panel steers you off one dud colourway: 500 units at 18 dollars landed is 9,000 dollars of stock that never becomes a clearance problem, plus six months of cash flow you did not bury in a warehouse.
- Pre-tested creative. Hooks screened by the panel before launch nudge campaign click-through from 1.1% to 1.25% on 20,000 dollars a month of ad spend. Across a year, that is in the order of 25,000 dollars in media efficiency.
- A verbatim copy bank. Forty to sixty customer-worded phrases a year feeding product pages, emails, and ads. Brands that swap founder-speak for customer language on key pages routinely see conversion lifts worth another 15,000 to 30,000 dollars at this revenue level.
- The halo on the 26. Panellists buy more often and stay longer, and the best of them graduate into genuine ambassadors who sell for you.
Call it 50,000 to 80,000 dollars a year in avoided mistakes and captured lifts, for roughly 1,500 dollars in store credit and two founder hours a month. There is no other research spend on your P&L with that ratio.
Your 30-Day Panel Launch Checklist
Here is the takeaway version. Work through it top to bottom and you will have a working panel inside a month.
- Week 1: Pull your top 10% by lifetime spend, plus recent first-timers and a handful of lapsed customers.
- Week 1: Write the invitation: name the seat count, name why them, spell out the deal and the 48-hour expectation.
- Week 2: Send about 40 invitations, personally signed by the founder.
- Week 2: Set panel_member to true as acceptances land and build the Klaviyo segment.
- Week 2: Send the welcome email restating the deal and the first-look promise.
- Week 3: Run the first monthly pulse: three questions, at least one of them a live decision.
- Week 3: File the tally and verbatims, and start the copy bank spreadsheet.
- Week 4: Run your first 48-hour sprint on a real decision with money attached.
- Week 4: Close the loop on everything sent so far, including the outcomes.
- Ongoing: Four asks a month maximum, refresh a quarter of the seats each quarter, and log every decision the panel informed.
How a Panel Changes Your Ad Creative Economics
Here is the payoff most founders miss: a working panel does not just answer product questions — it de-risks your paid media. Industry-wide, only about 1 in 10 new ad creatives becomes a genuine winner, which means most of your testing budget buys losers. Every concept you screen through your panel first shifts those odds.
Run the maths on your own account. If you are spending $2,000 AUD a month testing creative and your win rate moves from 1-in-10 to 1-in-5 because the panel killed the weakest hooks before they ever spent a dollar, you have effectively doubled the output of the same budget. A 48-hour panel sprint costs you a handful of store credits; a failed ad concept costs $200-$400 in spend before the data tells you what 26 customers could have told you on Tuesday.
The workflow is simple: before a new concept enters testing, show the panel two or three hook options and ask which would make them stop scrolling — and why. Feed the language they use back into the ad copy itself. Customer phrasing consistently outperforms marketer phrasing, because it sounds like the person watching. Then let your creative testing framework confirm the winner with real spend data. Panel first, pixels second.
Inside eCommerce Circle, knowing exactly who your customers are and what they will buy next is one of the core pillars we work on with every member. If you want a second opinion on your research stack, let’s talk.
