Restoring Trust to Data and Accelerating Product Growth with Amplitude

It’s incredibly frustrating to hear product managers declare, “I don’t trust this data, so I’ll just go with what I think.” But we can’t always blame them for their distrust. Every little dent in an organization’s data armor degrades a team’s ability to become data-informed. And if the existing data is confusing or hard to access, people can’t use it to help a product grow.

At Whisk, we know this better than anyone. It can be difficult to restore trust in data, and even though we had a strong desire for instrumentation and experimentation, there wasn’t always a way to press forward with what we learned. We didn’t have a way to support our growth approaches with legitimized data and actionable insights.

If your existing data is confusing or hard to access, people can’t use it to help a product grow.

Over the past six months, however, Whisk has found a clear path toward better data management. As a result, we’ve been able to fix the past, restore trust in our data, and grow our platform.

Spearheading product growth at Whisk

Whisk is an all-in-one app for sharing and monetizing food content. Users can save recipes, meal plan, and create shopping lists to save time preparing home-cooked meals. Samsung acquired us in 2019, and we play a major role in their IoT strategy.

We’re working on developing the entire food platform for Samsung, building out all the APIs that power food-related experiences across the company’s major divisions. Today, dozens of Samsung appliances feature Whisk technology, from refrigerators to connected oven ranges and stovetops.

Coming to Whisk was an incredible opportunity for me. I initially began work on the B2B2C strategy, but we soon shifted more of our focus to the consumer experience. In the transition, I became Head of Product, tasked with taking a more product-oriented lens to growth. I wanted to expand our existing culture of experimentation and enhance our data-informed decision-making.

But first, I needed to get our data situation under control.

Understanding the data problem

Data is at the heart of what we do at Whisk. Outside of product analytics, we’ve also built this incredible food ontology which in and of itself is a data product. The team has a solid culture of analytics instrumentation. What we didn’t have was organization.

I don’t have a technical background, but I’ve come to understand two things as I worked alongside data scientists. First, the vast majority of data work is making sure data is observable and well-engineered. Second, data must be highly available and accurate to inform stakeholders.

When we shifted our focus to the consumer experience back in 2021, I booted up our first growth team. It didn’t take long to see that gaps in our tech stack created unnecessary obstacles to product growth.

• We had too much instrumentation. With hundreds of events instrumented differently across apps, it was hard to maintain good data quality and make sense of it all. Our teams weren’t focused on the key events.
• We had a big gap in our experimentation. It was becoming harder to maintain a laser focus on our main KPIs and metrics. We used multiple experimentation platforms with different capabilities, defaulting to retention as a primary metric. It was not ideal for the high-velocity growth culture I wanted to build.

We weren’t looking for a new platform at the time. We were already Mixpanel customers, but the platform started to deprecate their A/B testing functionality, which would’ve taken us in the opposite direction from where I wanted to go.

That’s when Amplitude came to mind.

Whisk’s migration to Amplitude

I’d long been a fan of John Cutler, a product evangelist at Amplitude. John offered to lead a product workshop for anyone in his network, and I reached out to him about being over-instrumented and its impact on Whisk’s data culture. During the workshop, we brought together teammates from QA, engineering, and design to examine ways to identify key events in our life cycle. It was a great experience, and at the end, John mentioned that there is value in having your analytics and experimentation side by side and that Amplitude can do both.

We planned to integrate additional experimentation SaaS onto Mixpanel, but the more I learned about Amplitude, the more impressed I became with its all-in-one toolset.

For experimentation, Amplitude guards against PMs peeking into results too early and drawing inconclusive results. Even though it wasn’t the most built-out experimentation platform on the market, we liked Amplitude’s vision. We also saw the benefit of having all of our data and experimentation tooling in a single, highly observable system. We could track people that had been in long-running experiments. We could manage experiment independence quite easily. We could stay focused on accurate data. So we leaped.

Improved product innovation

We put Amplitude to work right away, handling much of our experimentation. We’ve been running ROI pilots on various monetization opportunities within the core user base, including an A/B test designed to measure the engagement impact of running sponsored ads on our home feed.

By placing more ads on our home feed, we expected a decrease in scroll depth (which correlates with retention). We compared the current Whisk home feed with one that included a targeted ad placement and discovered a startling outcome: no difference.

But this is only just the beginning. We’re now running multiple tests at once and PMs are thinking more intelligently about how to track changes and measure engagement effectively.

Amplitude helps us understand events in context, which has led to an increase in content creation and retention.

Amplitude helps us understand events in context, which has led to an increase in content creation and retention. For example, we developed a feature called “Made It,” which prompts users to add notes, tags, and pictures once they have prepared a recipe. After dozens of iterations that yielded lackluster results, we zoomed out in Amplitude and examined where different user actions were correlated. It became instantly clear that some users wanted more connections. They didn’t see the value of posting for the sake of it; they wanted someone to post for.

Using this insight, we helped people find and connect with others who value and share their insights in-app. We’re looking to use this knowledge in new ways, and are currently evaluating monetization strategies for a “creator fund” based on engagement driven by user content. The more we learn about how content drives engagement, the more we can offer.

We’re also answering a host of strategic and product-specific questions within Amplitude. Currently, we’re working on uncovering the viral inflection point of recipes, tracking whether social rewards are effective at driving a habit loop, and determining the magic number of followers that create an active user. We also better understand which and how frequently engagement events happen during certain days. The product team distributes these insights to stakeholders and other members of the Samsung team. Our data is widely cited in internal research about cooking behaviors of consumers in the US and UK, driving product innovation and marketing strategies.

Robust features help us capitalize on a new market

Whisk’s primary strategy includes building an ongoing engagement engine and gaining greater visibility into user types. A big priority is understanding and capitalizing on a market that hasn’t been won yet: the vertical social food community.

Before using Amplitude, we had a one-size-fits-all approach to key engagement actions with a high likelihood of retention. Today, it’s easy to do a clustering and correlation analysis of different segments within Amplitude, identifying different personas and creating more customized and tailored journeys. And it’s all done in a single click.

Whisk still maintains a startup culture despite its rapid growth, so everyone up and down the organization logs into Amplitude to get their jobs done. This democratization of data means more people are able to gather insights, and these insights are typically more impactful than previous insights from Mixpanel. Amplitude’s usage across Whisk includes data analysts, PMs, marketing and engineering teams, and executives. Even the partnerships team gets involved from time to time. Amplitude charts and dashboards are publicly available to the whole team, saving a lot of time.

We regularly use Compass, which shows how new user events correlate with retention; Personas, which groups users into clusters based on the similarities of their event behavior; Segmentation, which shows what users are doing in the product; and Funnel Analysis, which offers insight into how successfully users are navigating a defined event path.

Another essential feature for us is data planning. Before our Amplitude adoption, we’d thought about using Iteratively to improve analytics consistency across platforms. Fortunately for us, Amplitude acquired Iteratively around the time of our evaluation. Iteratively’s data planning and developer experience features are now a part of Amplitude Data, which we’re leveraging as we build out the team that’s creating stellar processes around release instrumentation. With Amplitude, transformations to fix any messy historical data are much easier to set up and manage.

We’ve migrated all our event documentation into Amplitude Data and are slowly moving through hundreds of events. For the first time, we’re getting a much better sense of where there’s inconsistent instrumentation and where there are missing properties. We can see where we had gaps in the past and correct those errors on the fly. So even old reports with these malformed events can offer valuable insights.

All of these newfound user insights are exciting for our team because it’s showing us that a vertical social food community can be more engaging than one completely driven by algorithms. The fact that we can see users actually cooking, then responding and saving more from people they follow shows that our core hypothesis is working.

A clear path to world-class infrastructure

Experimentation within Amplitude takes the guesswork out of measuring the impact of product changes, helping us restore trust in data. Amplitude allowed us to build the data-informed culture that we already had the pieces to do.

A data-informed culture can lead to quick, effective, and sustainable product growth.

As we move toward improved testing and tracking, Whisk is in a position to create a world-class data and experimentation infrastructure—and become an exemplar to the rest of the Samsung community. Our culture is more data-informed than ever before, which has led to the quick, effective, and sustainable growth of Whisk.