Surveys
Introduction
Maya has spent 6 months developing her app for busy parents to plan meals. She has done her research, hired a designer, built core features and is now ready to launch her campaign. However, very few people are signing up to her app, and even fewer are coming back.
After speaking to potential customers Maya realised that most parents already have a meal planning routine in place. The real problem they have is finding affordable ingredients and keeping their grocery costs down.
Although the app might not have been a bad product to build, Maya hadn’t validated that her product was actually solving a problem people wanted to be solved.
By validating your Product Before Launch you’ll learn more about your potential customers. Understand if there is demand for your product, identify any weaknesses and make better decisions whilst keeping your costs relatively low.
Whether you’re validating a software product, physical product or new feature, the aim is to turn your assumptions into facts.
What Is Product Testing Before Launch?
Product Testing Before Launch is the process of validating your idea, concept, prototype or early version of your product. This helps businesses to understand if there is a demand for their product, if people find the solution helpful and if they’d use/pay for it.
It’s important to understand the difference between quality testing and product validation.
Quality testing ensures your product works, whereas product validation ensures you’re building something people actually want.
For example your mobile app might not have any technical issues but customers may not find it helpful. Or your new skincare product might pass initial quality checks but not be appealing to your target market.
A good product validation should answer the following questions:
Is there a problem? Are potential customers facing this issue?
Is it relevant? Does your product solve the problem?
Is it usable? Can people understand how to use it?
Is there demand? Will people try/use/buy your product?
This will help you to decide if you should continue, improve your idea or change your approach.
Why Should I Test My Product Before Launch?
Without validating your product you risk making costly mistakes. You could spend money developing, stocking, advertising and distributing your product only to realise the market aren’t interested.
By validating your Product Before Launch you can reduce these risks by;
Understand Customer Needs
You might have some assumptions about your customers. By speaking to customers and doing surveys you’ll gain more insight into what they struggle with, what they currently use and what they wish would be different.
Save on Unnecessary Costs
Testing your idea with a brief concept or prototype is usually cheaper than building a fully developed product. Getting feedback early means you can identify any unnecessary or confusing features.
Improve Product Positioning
Customers may describe the problem differently to how your team would. This can help with writing clear product descriptions, landing pages and marketing.
Identify Demand
By using a waitlist, product validation survey or pre-order campaign you can gain some useful insights into if there is demand. Whilst this isn’t a direct indicator of people purchasing it will give you more information than just getting excited friends.
Make Better Decisions
By testing your product your team will have a better understanding of what is working, what needs improving and who your target customers are.
7 Ways To Effectively Test Your Product Before Launch
There isn’t one way to test your product, and the best option will depend on how developed your idea is and what you want to learn.
1. Customer Interviews
Customer interviews are great to use when you’re still exploring the problem. Ask potential customers about their current situation.
If you’re looking to build a personal finance app ask potential users how they currently track their expenses, what they struggle with and when they last had issues managing their spending.
Some great questions to ask are;
How do you currently solve this issue?
When was the last time you faced this issue?
What is the most frustrating part of your current solution?
Have you paid for any tools/services to solve this?
What would you change about the options available?
Avoid questions that prompt people to agree with your idea. Just because they like your idea doesn’t mean they’ll change their behaviour or pay.
2. Product Validation Survey
A product validation survey will help you understand how common the problem is for a larger audience.
Once you’ve identified common issues through interviews you can run a survey to see if these issues are common.
Some great questions to include are;
How often do you face this issue?
What do you currently use to solve this?
How satisfied are you with your current solution?
What is missing from the options available?
How important is it to solve this issue?
Keep your survey short and to the point. A survey with around 8-10 well written questions can be a good starting point. It will depend on what you’re looking to learn.
Ensure you’re asking the right people. If you’re asking people who wouldn’t realistically use your product it can make the demand look higher or lower.
3. Concept Testing
Concept testing will help you Validate Your Idea before you invest in building the full product.
Write a short description, sketch, mock-up or concept to explain who your product is for, what it does and what problem it solves.
Ask potential customers;
How well do you understand the idea?
Does it solve a problem you face?
What features stand out as being useful?
What are you concerned about?
Would you consider trying/buying?
What price would you expect to pay?
If you’re looking to compare 2 product concepts ensure you present them fairly to avoid bias.
Remember just because they say they’d buy it doesn’t mean they will. This is an early indicator.
4. Landing Page & Waitlist Testing
A landing page can help gauge if people are taking action after learning about your product idea.
Create a simple landing page explaining the key benefits of your product. Include a call to action such as a waitlist, early access request or demo sign up.
Track;
How many relevant people visited.
Conversion rate of your landing page.
How many signed up to your waitlist.
Cost per signup if using paid advertising.
Where your traffic is coming from.
Any questions or concerns.
If you’re getting lots of traffic but very few signups there could be an issue with your audience, messaging, offer or concept.
A waitlist is a good indicator of demand but shouldn’t be treated as a guarantee everyone will become a customer.
5. Prototype & Usability Testing
A prototype is an early version of your product. This will allow people to experience the key features without a fully developed product.
This could be a click-through design for a mobile application or a sample/model for a physical product.
Give them realistic tasks to complete.
For example ask them to create a weekly meal plan for a grocery planning app.
Look at;
Did they complete the task?
Where were they stuck/confused?
What instructions didn’t they understand?
How long did it take them?
What did they expect the product to do next?
Don’t explain how everything works. The aim is to see if they can understand how to use your product without constant help.
6. Beta/Pilot Testing
A beta test allows a small amount of real users to try an early version of your product. This is particularly useful if you have a prototype and want to see how it performs in real situations.
Ask people who fit your target customer profile. Ensure they have an easy way to report any issues.
Look at;
Activation rate.
Repeat usage.
Feature usage.
Task completion.
Issues reported.
Retention.
For B2B products a pilot with a small amount of relevant companies can be incredibly useful to understand how it fits into their processes.
7. Product Market Fit Survey
A product Market Fit Survey is most useful once people have tried your product. This will help you understand if they feel your product is valuable enough that they’d miss it.
A popular question to ask is;
How would you feel if you couldn’t use this product anymore?
Options usually are;
Very disappointed.
Somewhat disappointed.
Not disappointed.
The Sean Ellis test looks at how many people select very disappointed. A score of around 40% or above is often used as an indicator you have product market fit.
Whilst this is a benchmark it doesn’t guarantee you’ll be commercially successful. Ensure you’re asking people who have experienced the key value of your product. Don’t ask everyone who has created an account.
Ask follow-up questions such as;
What is the main benefit you’re receiving from this product?
Who do you think would benefit the most from this?
What would you use instead?
This can help you identify your best customer segment and understand what makes your product valuable.
How To Test Your Product Before Launch
If you use multiple methods without a clear plan you’ll get lots of feedback but very little decisions. Follow these steps to ensure your product validation is focused.
1. Define The Problem
Write down the specific problem your product is solving. Don’t start by listing features. Focus on the customer need.
2. Define Target Audience
Write down who is most likely to face this problem. Think about their needs, behaviours, current solutions, budget and buying behaviours.
3. Write Down Riskiest Assumptions
Write down what needs to be true for your product to be successful. For example customers need to face the problem regularly, find the solution helpful and be willing to pay.
4. Choose The Right Testing Method
Use interviews to explore issues, surveys to identify general trends, concept testing and prototypes to see how people use it.
5. Decide On Success Criteria
Decide what You Need to see to move forwards. This could be a minimum level of importance, a target number of quality signups or task completion.
6. Get Feedback From Relevant People
Ensure you’re asking the right people. Don’t just ask friends, colleagues or supporters who may not be as critical of your idea.
7. Analyse & Decide
Look for any common trends. Decide whether to move forwards, improve your concept, change your target audience or stop.
How Many People Should I Test?
The right sample size will depend on what type of research you’re doing, who your audience is and what decision you’re trying to make.
As a general guideline;
Testing Method | Useful Starting Point |
|---|---|
Customer Interviews | 8-15 relevant people. |
Early Usability Testing | 5-8 users. |
Product Validation Survey | 50-100 responses. |
Concept Testing Survey | Larger sample may be required. |
Beta Testing | Manageable amount of relevant early users. |
These are just guidelines. A small number of people won’t give you reliable data on the whole market.
If you’re making a big decision consider how confident You Need to be, how large your audience is and what sample you’re using.
How To Analyse Your Product Testing
Getting feedback isn’t useful unless it leads to a decision. Start by comparing your results to your success criteria.
Look at quantitative and qualitative feedback.
Quantitative data could include survey scores, signups, conversion rates, task completion and repeat usage. Qualitative feedback could include comments from interviews, complaints, suggestions and customer experiences.
You could also use an ROI Calculator to see what return on investment you could get. This will depend on your assumptions and numbers.
Break your findings down into 3 categories;
Go – You have evidence to show there is a problem, a solution and demand. Move onto the next stage of development/launch.
Improve – People recognise there is a problem but You Need to improve your product, pricing, features or messaging.
Stop/Reconsider – The problem isn’t important to your target audience or the solution doesn’t provide a meaningful benefit.
If you’re running experiments with variations of your landing page/product message you can use an AB Test Calculator to understand the relationship between your conversion and your assumptions.
Don’t change your success criteria based on disappointing results. Often negative feedback will give you the information You Need.
Common Mistakes To Avoid
Even businesses who do research can draw the wrong conclusions if they don’t set up their testing correctly.
Wrong Audience – You may not get accurate demand from people outside of your target market. Ensure you’re asking the right people based on your customer profile.
Leading Questions – Questions such as "Wouldn't this feature make your work easier?" will encourage positive responses. Use neutral language.
Interest Doesn’t Equal Purchase Intent – People may say they find your idea interesting but not be willing to pay. Look for stronger indicators such as trial signups, deposits, pre-orders and repeat usage where appropriate.
Too Many Changes – If you’re changing the price, design, features and messaging it will be hard to know what influenced the results. Try to test key assumptions individually.
Ignoring Negative Feedback – Don’t ignore any criticism. Look for common issues and find out why they’re happening.
No Clear Success Criteria – Decide what a successful result looks like before starting your experiment. It’s easy to see almost any result as a success.
Only One Test – Customer needs and market can change. Continue to validate as your product develops and reaches new customers.
Tools To Help With Product Validation
You don’t need an expensive research setup to start validating your product idea. There are a few simple tools you can use to help gather feedback, track behaviours and organise your findings.
Online Surveys – Use surveys to understand what issues your customers are facing, what they prefer and what they consider when purchasing. Look at our survey templates to get started and tailor your questions to what you’re looking to learn.
Analytics – Track behaviours, signups and engagement. Having a clear analytics setup will help you track responses and identify trends.
Customer Feedback – Gather feedback on how satisfied people are with your product. Use an NPS Calculator or CSAT Calculator to track feedback over time.
Integrations – Ensure you can easily gather feedback by integrating with other tools. Look at our relevant Integrations.
Simple product validation surveys can be created using tools like Ovoform. Choose a tool based on what questions You Need to ask, how you want to analyse the responses and your budget.
Final Thoughts
A successful product launch starts long Before Launch day. It starts when you begin to understand the people you want to serve.
By validating your Product Before Launch you’ll gain insights into if there is a problem worth solving, if your solution makes sense and what You Need to change. You don’t need a fully developed product or lots of money to start validating. A few thoughtful interviews, targeted product validation survey or simple prototype test can give you some key insights.
Start with your biggest assumption and choose an appropriate method to test it. Be willing to act on what you learn. Sometimes the best outcome will be to improve your idea and sometimes it’ll be to change direction and not waste any more money.
The aim of validating isn’t to prove your original idea is perfect. It’s to give your product a better chance of solving a real problem.
