How To Set Retention Goals Like A Scientist

In 2012 NASA sent the Curiosity rover on a mission to Gale Crater, Mars. Curiosity was designed to pursue four specific experimental goals, to help fulfill NASA’s long-term goal of colonizing Mars:

1. Figure out whether there had ever been life on Mars.
2. Better understand the climate of Mars.
3. Better understand the geology of Mars.
4. Prepare for human exploration.

They’re weighty goals, indeed, but not particularly actionable. That’s why NASA scientists make a distinction between goals and technical, actionable objectives that will actually get them there. The corresponding objectives NASA picked for Curiosity looked like this:

1. Search for geological clues to past environmental conditions on Mars.
2. Inspect the terrain for signs of erosion, sedimentation, volcanism, and cratering.
3. Find samples of rocks and soil with evidence of mineral deposits.
4. Calibrate instruments to Mars’s surface to give our orbital measurements more accuracy.

Integrating goals and objectives is the essence of the scientific process. For an experiment to be a success, there needs to be agreement and consistency: goals must have objectives which when completed indicate success. This is as true for the Rover as it is for your app’s retention problem.

Break your retention problem down into goals and objectives, and you’ll have a repeatable process for cutting churn. You’ll be able to focus on the right factors, adjust when things aren’t working, and experiment your way to an app with amazing growth potential.

Separate Your Goals And Objectives

A goal, put simply, is something you hope to accomplish with your experiment: an ambition. An objective represents how you’re actually going to achieve a goal. It’s the “how” to the goal’s “what.”

These distinct categories are important, but they’re also never permanent—goals and objectives can both change at a moment’s notice. Every time you make a discovery, in fact, is a moment of reflection: what’s been learned? Was the technique we used successful, or is there another that would get us better results? That re-assessment is critical to keeping the process productive.

Follow the truth wherever it leads

Y Combinator co-founder Paul Graham, who has directly compared startup founders to research scientists, advises every company he meets to focus on crafting a scientific approach to retention.

Growth is the only essential thing for a startup, he tells founders, but also a “bewilderingly multifarious problem”—one with way too many variables and factors to attack head-on. To Graham, a former programmer, the solution is to look at it as an optimization problem: once you know the goal and objective you’re working with, pick a factor. Optimize for it, assess the results, learn from the process, and start again:

• If you hit your actionable objectives but don’t complete your larger objective, and your goal isn’t affected, then you had the wrong actions in mind.
• If you hit your actionable objectives, accomplish your higher objective, and your goal is not affected, then you had the wrong higher objectives.
• If you hit your actionable objectives, accomplish your higher objective, and your goal is positively affected, then you’ve run a good experiment.

Testing different combinations of goals and objectives is fundamentally how startups grow. Data-driven companies like Facebook, LinkedIn, and Airbnb didn’t stumble into success—they found the levers that gave them a repeatable growth process, and they didn’t second guess it when they did: the proof was in the data.

“You’ll generally do best to follow [this] constraint wherever it leads rather than being influenced by some initial vision,” Graham says, “just as a scientist is better off following the truth wherever it leads rather than being influenced by what he wishes were the case.” But if growth is a multifarious problem, setting up a framework of goals and objectives improving your retention isn’t exactly child’s play. Let’s look at how one of the SaaS world’s foremost experts in growth organizes his team’s retention efforts.

How To Create A Retention Plan

Brian Balfour, VP of Growth at marketing giant HubSpot, likens the churn-fighting process to building a machine. If you’re able to set up the right processes and incentives, then growth will basically run itself.

To do this, Balfour uses a goal-setting system invented at Intel and popularized at Google. It’s called OKRs—Objectives and Key Results. Here’s how Balfour might incorporate the OKR system to set up goals at HubSpot:

1. Draw up a statement of your goal.
2. Set a timeframe (30 to 90 days).
3. Assign three key results, each of which needs to be an objective you can measure:
   1. Improve retention by 50%
   2. Improve retention by 2x
   3. Improve retention by 10x
4. Brainstorm the actionable objectives you’ll use to hit your key results.

The key results, Balfour says, need to be separated out in terms of the likelihood that they’ll succeed. You should have a 90% chance of achieving the first one, and a 50% and 10% chance of achieving the second and third, respectively.

This provides an essential psychological boost. If you’re spending a week A/B testing different combinations of button colors, then hitting one easy objective connected to a hyper-important goal is a way to feel good about your work.

Having “reach” objectives, on the other hand, encourages everyone to go above and beyond. It’s a carrot, and when you succeed in hitting it, it’s a really powerful one.

Of course, since you could imagine any number of different actionable objectives, you will have a lot of different options for how you choose to pursue your first retention experiment. We’ll look now at the ways retention goals and objectives are often formulated, not to be prescriptive, but to give you a sense of how your own framework should emerge.

Three Fundamental Stages of Churn

Dividing retention up into three timespans—long, medium, and short—is widely accepted as a go-to tactic for turning churn into a solvable problem.

To understand why, let’s look at a a graph from mobile intelligence service Quettra, who analyzed retention rates for thousands of apps downloaded from the Google Play Store. Here’s what the average app’s retention rate looks like

By looking at this graph, it’s clear that we commonly divide retention into the long, mid, and short-term because there are three fundamental stages of churn.

On average, apps are down to 30% of their original users—after one day. After two weeks, they’re down to 10% of their original users. But by three months later later, the curve has flattened: less than 5% of those original users churn out between Day 14 and Day 90.

You can understand this visually as well. Divide the graph up by Day 0 to Day 1, Day 1 to Day 14, and Day 14 to 90, and you’ll see three significantly different rates of change. That’s really solid proof that those periods of churn are due to different factors:

• Short-term: Downloading an app is a frictionless transaction these days. It’s also one that requires zero commitment, and that’s what most people give it. These people opened your app, may have tinkered around, may have not, but they didn’t open it the next day. Either way, it just didn’t do it for them.
• Mid-term: These people downloaded your app and saw something in it, at least at first. For one reason or another, they got value out of using your app. It wasn’t something they wanted to delete right away, but eventually, they did.
• Long-term: Your app solved some kind of problem for these users. You might refer to this as a good problem to have, depending on where you’re at, because long-term churn does rely on your app actually providing value. But you still have a problem, because you want your best users to stick around.

What Your Users Do Before They Churn

When you separate your users out by short, mid, and long-term retention, what you’ll see is distinct differences in how they use your app. Now you need to craft objectives, with that in mind, that will target each group of users differently.

The point is to move users from one stage of retention to the next—you want to get more short-term users into the mid-term, more mid-term users into the long-term, and for your long-term users to stay engaged over time.

• Activation: A considerable percentage of mobile apps—nearly 30% and growing—are only used once; about 80-90% of apps eventually get deleted. This kind of user behavior would be a good sign that you’re not getting users to activation moments, or Aha! moments, fast enough. It means that your app is extremely liable to get used once and deleted as users poke around, meandering without purpose. Apps with huge short-term churn numbers, it follows, need to focus on bringing their users to their Aha! moments faster. That means more people getting that core value and sticking around into the mid-term, where you’ll have greater opportunities to connect with them.

• Patterns: At Price Intelligently‘s SaaSFest 2015, HubSpot Product Manager Dan Wolchonok described a key symptom of mid-term churn: active users who come and perform one action per week. He calls it drive-by usage, and you work to reverse it by encouraging patterns of behavior rather than one-off acts. You’ve shown that you can get these users to take interest in your app. Now you just have to get them to see it as a habit, not an amusement, and if you can sustain that, you’ll start seeing more and more users sticking around in the long-term.
• Re-habituation: Those who used your app for months and then churned got value out of it. Your app was a habit for them. Then something happened, and the habit was broken. Those users are gone, but that doesn’t mean their habits are dead. Some gym rats go weak in the knees walking past a Burger King, reminded of old times—your users, similarly, might just need a nudge to get back to using your app. Do this right, and you can turn your app into something unforgettable and ever-present.

How You’re Going To Take Action

With all your ideas about how your short, mid, and long-term churners behave on the one hand, and how you want them to behave on the other, you need to come up with one more set of actionable objectives. These are the actual techniques that you’re going to be using on a daily basis, the granular objectives around which your growth team is going to revolve.

• Conversion: As your users advance from opening your app to becoming activated, they proceed through a series of steps, and in between each step you can calculate a conversion rate. Say 41,496 people sign in on a particular day— 24,205 will view an article they like, 8,162 will join the related community, 6,517 will post a comment on the article—and for those lucky people the Aha! moment will kick in. To fix your activation problem, you need to look at each one of these conversion rates and identify where you have opportunities to activate more users. Fix this early leak, and you’ll see more users move from short to mid-term retention. That’s what’s really going to start bending your retention curve up and helping you find real growth.
  

• Stickiness: To build patterns of usage, you have to first understand what features people are actually using in your app. Here, we see that activities like adding a song to a playlist (performed 1,500k times), searching (900k), and favoriting (500k) are largely insignificant compared to playing songs (2,650k). Finding an action like this that your users perform far and away the most is a good way to understand where you have your core value. Drive more usage toward that feature is how you’ll build a habit around your app that will take users from mid-term to long-term users.

• Re-activation: To keep people engaged over the long-term, we need to keep finding reasons to bring users back to the app. For example, 80% of users who received a push notification returned to the app, even in the later stages of users’ lifecycle in week 4 and beyond. This is a great sign—if we can find relevant, on-point ways to notify users, we’ll constantly be able to re-habitutate them to the app, improve our long-term retention, and meet our overall retention goals.

Over And Over And Over Again

Chamath Paliphitya, lead of the first Growth team at Facebook, gets a lot of questions about how Facebook grew at such an astronomical rate. As you can imagine.

He said: “We did three really obvious brain dead things and the reality was we lacked enough self-awareness and ego to frankly just continue to do these very simple things over and over and over again.”

Measure, try, test. Over and over and over again.

Here’s the “big secret.” Every company doing huge things got where it’s at not by growth hacking or clever tricks, but by developing reliable processes for getting better, persevering through the doubt, and doubling down on successful experiments. That’s what it takes to build a great company and reach your loftiest goals.

Comments

REGINA RINGO4423: You also can esign the form and fax it.