Prioritize Outcomes Over Outputs

Editor’s note: this article was originally published on the Iteratively blog on May 3, 2019.

Lighthouses provide a beacon to guide ships at sea; without them, disasters happen. While most people think of lighthouses as simple mechanisms, they are sophisticated engineering feats that require a high degree of trust.

Similarly, your product analytics provides you a beacon to monitor your customer experience and the data that influences business decisions. If you’re not measuring the right metrics or don’t have a shared understanding of the goals for your business, then chances are you’re sailing fast towards disaster.

Teams that operate with a “feature factory” mentality, shipping feature after feature without taking the time to understand the levers of their business and how their work is laddering up to their overall goals, often create product bloat and subsequently increase customer churn.

While most organizations want to make data-backed decisions, they’re often less clear when it comes down to what to track and what makes good metrics.

As a result, we often see teams looking at vanity metrics or celebrating success theater, and this is because they don’t practice good data literacy or have proper instrumentation in place.

Setting the right metrics

The right metrics will help you:

• Evaluate the health of your business
• Measure the effectiveness of feature releases
• Define your roadmap and strategy

If teams want the insights, they need to be willing to put in the work required to document their goals and metrics. This is the only way you’ll know what data you should be capturing in the first place and ensure that you’re not reading tea leaves.

Encourage your team to focus on outcomes over outputs by defining clear hypotheses that are tied to metrics. Using the template, “We believe X will result in Y because Z” helps teams capture hypotheses consistently and makes it easier to prioritize based off of the expected outcome. It’s helpful to visualize the relationship between your goals, metrics, and hypotheses.

It’s easy enough to follow up afterwards to validate the hypothesis. Did shipping a free plan increase the number of evaluations by 45%, and if not, why? This exercise encourages teams to obsess over the learnings because shipping is no longer good enough; they need to follow through and measure the impact of what they’ve shipped.

Tips for better metrics

1. Keep it simple

Metrics should be easily understood by everyone in the organization. Great metrics should also inspire action; teams should know how to react when they change. Is the “sign up” event triggered after the customer clicks submit, when the email is verified, or when the account is provisioned in the database?

2. Show ratios and compare over time

Avoid looking at absolutes and instead, compare ratios or rates. There is a big difference in the amount of information conveyed when looking at “6000 sign ups this week” versus “+15% in sign ups WoW.”

3. Create realistic forecasts and tie metrics to OKRs

Is +15% good or bad? How do you know if you’re on track to hitting your goals? Forecasts should align with your OKRs and help your team prioritize their work. When you’ve hit your desired result, move on to the next most crucial thing.

4. Make metrics accessible

Your metrics should be accessible to everyone in the team. Otherwise, how do you expect them to change behavior? Metrics set a common goal for the team to rally around.

5. Assign an owner

Metrics should have a single owner who is empowered to drive results. This helps ensure accountability and prevents teams from duplicating efforts.

Starting small

Setting the right metrics is an iterative process, and it will take some time to adopt with your team. The best thing to do is start small and continue to improve over time.

The number one advantage for SaaS companies is creating a data-informed culture that is obsessed with learning. To foster that culture learning you need to invest in data literacy and empower your team to take ownership of metrics. What are some ways that you promote data literacy and build a culture of learning within your organization?

If you’re actively working on improving your product analytics, tell us more. Join the Amplitude community to share your ideas and learn from others.