How We Used a Private Beta to Get Product Market Fit

Using a private beta, an early prototype or explainer video is a great way to pre-sell and validate your product idea. Here's how we did it for Fieldboom.

Product Market Fit

This is the second chapter in our new series called Scaling Up, which will show you the exact, step-by-step process you need to take an idea, validate it, turn it into a business, and generate sales, customers, and repeat orders. Make sure you subscribe to get notified when additional chapters are added.

In Chapter 1, we detailed the exact process we used to validate initial demand for Fieldboom  (including a process for pre-selling to customers).

In short, we described how we “scratched our own itch” and built a product that solved our pain points.

But it wasn’t enough to solve our own problem. We made sure others had that problem by…

  1. Asking our network
  2. Getting initial waitlist signups and surveying them
  3. Offering a demo and seeing if people would pay for lifetime access

As a result, 100 people paid us for lifetime access before we even had a product built.

That’s real demand.

Those 100 customers became the start of our private beta.

In a private beta, you offer a select group of customers the chance to try and give feedback on early versions of your product.

In this article, we’ll detail how we interacted with those private beta customers to ensure product-market fit.

We’ll tackle topics like…

  • Expectations: When can customers expect an actual product?
  • Feedback: Does your initial product “scratch the itch?”
  • Feature Requests: What if customers ask for features you didn’t want to build?

If you get these details wrong with your private beta customers, you’ll have an angry mob ready to disparage your business in their professional circles.

Get this right, and they will guide you closer and closer to product-market fit.

(Note: Ready to build your customer base? Fieldboom is a simple survey tool to help you with lead capture. Try it free for 15 days.)

1. Setting Expectations: When Will They Get Something for Their Money?

Rule #1 for pre-selling and private beta signups?

Set expectations as soon as you can. Be open and honest about the timeline you’re working with.

Don’t assume your customers know how long you’ll take to develop your product, because they don’t.

Instead, tell them how long it will be.

This was essential for us: most SaaS customers expect something the moment they pay.

But when we pre-sold Fieldboom, we couldn’t hand over access to our software: it didn’t exist.

Yet, they still gave us $100 (some gave us $200 — we experimented with the price point). And we had to make sure they knew before their payment was even done processing that they’d have to wait.

That’s a critical moment in your relationship with the customer. You have to communicate expectations as fast and as honestly as possible.

But any entrepreneur knows that development times can vary… a lot. It’s easy for things to take way longer than expected.

The solution?

Make the best estimate you can, and add a 20% buffer (minimum).

Some business owners prefer to double their estimates, and that’s okay.

If you deliver early, no one’s going to be upset with you.

If you deliver late, you have to explain why.

So when we accepted pre-payments from customers in early 2017, we promised that we would launch in March 2018.

We secretly hoped to launch by February 2018. 

We didn’t hit February, but it wasn’t a problem: our customers were still pleased with our “on time” launch in March.  

Takeaways:

  1. Set expectations around your delivery timeline.
  2. Set a delivery timeline that’s at least 20% longer than you think you need.

2. Establish a Communications Cadence

After setting initial expectations, you might be tempted to proceed without updating customers along the way.

Yes, building a product is hard, but you can’t crawl into a cave and keep your customers guessing. Their continued engagement is necessary if you want to gather feedback later on.

Besides, customers who are interested enough to pre-pay for your product want to know what’s happening.

So tell them.

Send them screenshots. Tell them how development’s going. If you hit a roadblock and are working around it, let them know.

Here’s a screenshot of one of our update emails in January 2018:

Product Market Fit: Fieldboom Beta Update in January 2018

The key is to convey real progress.

If you’re working on a feature that will take one month to develop, then show them what you’ve done two weeks into development.

For example, building reporting and analytics took us weeks, but every week we were making small progress so we were able to tell our beta users about that feature as it progressed.

Pick a communication schedule — once a month, one a week, or whatever works for you — and stick with it.

We sent updates once every week until launch.

Takeaway: Send consistent updates detailing tangible progress to your customers.

3. Gather Feedback on Early Progress

Fair warning: This is the hard, unglamorous, in-the-trenches step that most entrepreneurs will either skip or short change.

But it’s arguably the most important.

10, 50, or 100 paying beta customers is awesome.

But it doesn’t mean you have product-market fit. And that means you don’t yet have a sustainable business.

Customer feedback — old fashioned, qualitative, extremely detailed feedback — is what entrepreneur after entrepreneur has said will take you from initial demand to product-market fit.

So…

Once you have something tangible — a 3D design, a wireframe, the first module of a course — it’s time to get those private beta customers on the phone.

Yes: On. The. Phone.

For Fieldboom, we conducted around 200 customer calls to get feedback on initial designs. That’s not a typo, that’s just us putting in the hard work.

We asked questions like…

  • How many different tools are you cobbling together to perform this task now?
  • Which of our competitors’ products are you using to do this task?
  • What do you like the most about their product? The least?
  • How much time does this take you to do this every week?
  • Is this your #1 problem, or your #10 problem?
  • Will it be just you using the product, or other people who work with you too?

But because they were phone calls (not static surveys), those questions were just the beginning.

Through conversation, customers explained what they needed. These discussions yielded use cases, pain points, and necessary features.

We’d use The 5 Whys Framework to dig deeper and get to the root cause of problems.

Quantitatively, because of those conversations, we could put numbers to what our customers wanted to accomplish using Fieldboom (the ‘jobs to be done’):

  • 60% wanted leads from their website
  • 35% wanted feedback from their customers
  • 5% wanted feedback from their employees.

Collecting feedback from our private beta users helped us prioritize the most important features of our software.

Rather than putting out as many features as possible, we could focus on a few key features (and address the rest later).

Takeaways:

  1. As soon as you have something tangible, share it with your customers for feedback.
  2. Use customer interviews to set development priorities.
  3. Get on the phone, talk to as many customers as possible. Don’t shortcut this.

4. Measure (and Re-Measure) Product-Market Fit

Once you’ve iterated to a first version of your product, you can finally start testing product-market fit.

This is the well known iterative loop of build, get feedback, iterate, etc. 

Ideally, early customer conversations help you choose the right features to develop.

Once you’ve created them, you can see how well they help your customers do what they want to do.

Get that prototype or beta version in their hands, then collect feedback.

As you make improvements based on that feedback, quantify your success using one or both of the following method:

What Would They Think If You Disappeared?

In a survey we (repeatedly) sent out to Fieldboom’s beta testers, we asked how disappointed they would be if Fieldboom were unavailable.

(Here’s an example of the survey we sent out to our customers).

Product Market Fit: An example e-mail survey that was sent out to customers

Ideally, 40% or more of your private beta customers will say that they would be “disappointed” or “very disappointed” if they could no longer use your product.

That 40% is the threshold you need to reach for good product-market fit.

Here’s what well-known growth hacker Sean Ellis wrote about this question on his blog:

I ask existing users of a product how they would feel if they could no longer use the product. In my experience, achieving product/market fit requires at least 40% of users saying they would be “very disappointed” without your product. Admittedly this threshold is a bit arbitrary, but I defined it after comparing results across nearly 100 startups. Those that struggle for traction are always under 40%, while most that gain strong traction exceed 40%.

It took us four months of iterating to hit that number.

Don’t be afraid to circle back with the same survey after making improvements.

What’s Your Net Promoter Score?

Product Market Fit: An example of the detractors, passives, and promoters for recognizing a great idea vs. a not-so-good idea.

Another measure of progress is your Net Promoter Score (NPS).

On a scale from 0 to 10, customers assign a number to how likely they are to recommend your product to others.

Anyone who chooses 6 or less is a detractor: likely to go out of their way to steer colleagues away from your product.

Anyone who chooses a 7 or 8 is passive.

Anyone who chooses a 9 or 10 is a promoter: someone who will actively promote your product to friends and family.

Your NPS is your percentage of promoters minus the percentage of detractors.

Have a score of 60 or more?

You’re good to go. If you don’t have a way to survey your early users to measure your NPS score, Fieldboom can help.

Takeaways:

  1. Aim for 40% or more of your private beta members to say they would be disappointed if they couldn’t continue using your product.
  2. Start measuring your Net Promoter Score and aim for 60 or higher.

Know When to Say No

If you spend enough time talking to customers about what they want, you’ll get a lot of requests you either can’t (or shouldn’t) grant.

As great as it is to speak with customers, they don’t know your business the way you do.

Some of their ideas may be unrealistic. Maybe that feature would take you in a direction you don’t want to go.

In that case, you can…

  1. Add the feature anyway, even though it’s complicating your product
  2. Decide not to add the feature, but let the customer wait indefinitely
  3. Tell them ‘no’ (along with an explanation of why, when appropriate).

At Fieldboom, we prefer option #3.

Be polite about it. But don’t leave them hanging.

Basecamp is a great example of a successful company that said ‘no’ to its customers — over and over.

Do it in the right way, and most customers will be understanding.

It helps to take customer feedback for what it is: data that helps you make the best decisions you can for your company.

And sometimes, it takes saying ‘No’ to better serve your customers.

(Note: If you don’t have a way to survey your early users to measure your NPS score, Fieldboom can help. Copy this example survey we used to get started quickly.)

Want To Sky Rocket Your Sales? 🚀

Our new conversion quizzes will help you generate more customers from your web site by offering automated, personalized product recommendations. It's not uncommon to increase sales by 25%-100% using a conversion quiz.

Show Me How

Olivia is a freelance copywriter who loves science, cats, and swing dancing. If she can tell a story with her writing, it's a good day.