Hiten Shah: Stay Data-Informed to Build Something Customers Want

This article comes excerpted from Hiten Shah’s soon-to-be-released book: 5 Habits to Building Better Products Faster.

Over the past 10 years, Hiten has built and grown 8 different web-products, including Crazy Egg, KISSmetrics, and now Quick Sprout. Each of these companies used content marketing to drive the lion’s share of growth.

Sign up for Hiten’s SaaS Weekly newsletter to get a copy of the book delivered straight to your inbox, along with fresh business insights each Monday morning.

Companies like Amazon and Uber are known for using big data to serve their customers better. Because of those success stories, everyone in tech wants their company to be data-driven.

But if you’re building something new from scratch or entering uncharted territory, you actually need to be data-informed rather than data-driven.

Andrew Chen sums up the difference perfectly: “Being data-informed means that you acknowledge the fact that you only have a small subset of the information that you need to build a successful product.”

The difference between data-driven and data-informed seems small. But it was enough to cost Intel billions of dollars. In 2005, when Steve Jobs asked Intel CEO Paul Otellini to build a new processor for the first iPhone, the veteran exec said no. His gut told him smartphones would be huge, but the existing data didn’t back it up, so he passed.

The problem was that Otellini used data to back up what was already working for Intel in 2005. But he didn’t realize what that data could tell him about a product that didn’t exist yet. Big companies like Intel use forecasts and market research to steadily grow sales for tried-and-true products year after year. But those models can’t account for a game changer like the iPhone.

When you’re building something new, data can’t make your decisions for you. Data needs to inform your decision making. I know it firsthand, since this approach is what allowed me to build KISSinsights—the company that grew into the Qualaroo you know and love.

How We Built KISSinsights with a Data-Informed Hypothesis

The data-informed approach starts with a simple hypothesis for how a new product can solve a problem in the market. It’s an educated guess backed up by data, but not dictated by it. That’s how we started KISSinsights.

It began when we were holding customer development interviews for our analytics product, KISSmetrics. Product people were telling us the same thing again and again: “It’s too hard to figure out what people are thinking when they visit my website.”

This was around the time that analytics companies were popping up left and right, all trying to solve that exact problem: helping companies understand their customers. It had nothing to do with what KISSmetrics was trying to solve, but hearing it so frequently told us it was important. As our research lead Cindy Alvarez says, “One person might be a nutcase. Ten people are not all nutcases.”

The qualitative data from those interviews informed the hypothesis that turned into KISSinsights by telling us what pain point we could solve. Despite the tools available to them, product managers were unable to do fast, effective, ongoing customer research. Now, we needed to figure out what the root cause of that pain was, and what kind of product could solve it.

We traced that root cause to the user survey tools available at the time. They were terrible. They forced users to go to a separate page, interrupting whatever they were doing on your site. Then, they gave every user the exact same survey, regardless of how long they’d been a customer or been on the site. That meant the survey questions usually weren’t relevant, so users rarely filled them out. Overall, those tools made it incredibly difficult to get user feedback.

That gave us our hypothesis for the product. If you can send users a survey question that’s super relevant to what they’re already doing on your site, and it’s easy to complete without interrupting their workflow, then they’ll actually answer it.

The result was a pop-up survey you could target to users who had been on a specific page of your site for a pre-determined amount of time. It took less than five minutes for a developer to set up and less than thirty seconds for a user to complete. Instead of the usual 1-2% response rate most survey tools got, KISSinsights got a response rate anywhere between 10% and 40%.

The big takeaway of the story: 20 customer development interviews gave us enough qualitative data to launch KISSinsights. We didn’t have enough data to know whether it would be successful, because there’d never been a product like it before. But we had enough to know there was a problem we could try to solve. If we were totally data-driven, we wouldn’t be able to justify building this new product. But since we were data-informed, a hunch backed up by the data we had was enough to act on.

Within a year, KISSinsights was acquired by Sean Ellis. It’s now called Qualaroo.

How to Let Data Inform Your Next Product

Being data-informed isn’t always about gathering more data. It’s about taking the data you’ve already got and looking at it from another angle. Here are three tips for using data to build a new kind of product:

• Summarize the data in one sentence. It’s easy to get bogged down in numbers. This makes you look beyond that and focus on actionable takeaways. For us, that sentence was, “Today’s survey tools make it impossible for product managers to get useful, frequent customer feedback.”
• Always be testing. Don’t rely on data from just one source. Sure, we started with qualitative data from those customer development interviews, but that only got us to launch. Past that point, we used other methods like A/B testing and usability research to stress-test our hypotheses as the product grew.
• Deploy data across teams. Being data-informed isn’t just for product people. You need to teach everyone the power of data and make it part of your company’s DNA. Even engineers need a measurable, data-informed approach to building software. That’s why at Sailthru, chief data scientist Jeremy Hammerbacher spent a third of his first two months teaching engineers statistical learning.

The data-informed approach isn’t just about building one innovative product. It’s about equipping your team to expand the number of possible solutions to every problem they face—not limit them.

Nothing to Fear

In product development, going with your gut is usually seen as risky while going with the data is the safe move. But an over-reliance on data can be a crutch. And in cases like Otellini’s at Intel, that crutch can cause you to miss out on something huge.

Going with your gut doesn’t have to be dangerous. Your instincts are informed by all the experiences you’ve built up over the years. If those instincts are informed by a little data—even qualitative data from a few customer development interviews—they can be more valuable than anything you’ll get from a bunch of charts and numbers.