How to Make Event Tracking Part of Your Release Process

Editor’s note: this article was originally published on the Iteratively blog on March 15, 2021.

When building new features or products, it’s very common to leave analytics to the last minute—or even to forget about them entirely. This scenario may look familiar to you:

• PM works on a release
• Release happens
• CEO asks the PM how it’s performing
• PM: Let me ask the data team
• Data team: You never brought us in, there is no data on this feature
• PM goes back to the CEO with no answers
• Data team and PM are distraught

Situations like this can potentially happen quite often, and it’s really important to remember that no one party is to blame for this. A lot of it may well come down to culture.

Pointing at “culture” as a key part of the problem might seem easy, because culture is hard to define. But, it’s very often that an organization’s values and goals aren’t always fully reflected in the way team members behave. For example:

Your organization maintains that it makes data-driven decisions to provide the best service for your users. Everyone understands that a good foundation for this is a solid data strategy–otherwise you won’t be producing reliable insights from which to make those decisions.

But, in practice, conversations about your data and insight strategy (or even putting one together), don’t seem to happen. The task gets pushed aside and forgotten about, and reliable analytics rarely materialize.

This happens because of a gap between your organization’s values, and the actual everyday culture—it’s very easy to slip into this gap. Oftentimes teams will focus more on getting insights out of the data rather than building good practice around actually capturing the data. Maintaining good data culture is hard!

Building such a culture is more than just hype and celebration theater. In this post we’ll give you some practical advice on how starting out with simple, enforceable processes will help you maintain your intended data culture. One that focuses on capturing high quality data, and turning that into useful, actionable insights which lead to good decision-making.

Integrate analytics into your software development life cycle

When an engineering team gets to work building parts of a product, they will write code and do the usual things with it: branch, commit, test, review, merge. This is to ensure everyone’s on the same page with the build and any mistakes can be easily rectified.

There’s no reason not to treat analytics in the same way. It’s likely that you already have some kind of tracking plan (if not, we have a guide on how to get started with this), so a great way to start implementing it is to break it down into Jira tickets, just like with any other sub-task. An amazing tracking plan won’t matter if it’s not getting implemented. You will continue to miss vital insights unless you consider that:

• You need buy-in from relevant stakeholders and leadership teams that analytics tracking is just as important as the feature you’re building
• Tasks that implement the tracking plan should be prioritized alongside all other tasks for the build
• If there’s no tracking, you’re not ready to release the build

We all know that just because it’s on a series of Jira tickets, it doesn’t mean it’s going to happen. This is where culture shift really comes in. Ensure the tracking plan becomes a part of your software development life cycle every time by celebrating the success of a feature and not just the fact that the feature shipped. After all, if your company produces digital products, then shipping features is the entire point. Try and avoid celebration theater—celebrate when you see a feature performing well.

The only true way to understand how a feature is performing is by gathering analytics—which you will of course be doing, if your tracking plan was implemented right from the first build.

A note on QA in the context of analytics: you may be thinking that while implementing a tracking plan is straightforward enough with the right tools and culture, there is still no obvious, elegant solution to verify that it’s actually working. This is why Amplitude integrates with CI and lets you add analytics coverage to your existing test with our unit testing plugin.

Establish a repeatable process for analytics tracking

Another reason why the git process works so well is that everyone follows it consistently, and thus it’s naturally embedded within your company culture. You can build processes around analytics tracking that can become part of everyday workflows just as easily.‍

The biggest enemy of introducing a new process is a lack of buy-in. You can’t just say, “this is how we’re doing our analytics now” and expect everyone to jump on board. We’ve always maintained that analytics tracking is collaborative; when you’re putting together your tracking plan, all relevant teams should have a hand in shaping it.

That means involving all key stakeholders when coming up with new processes: the product team, the data/analyst team, and the engineering team. The unique expertise of those teams will help you decide:

• What your business goals are
• The metrics you will use to determine whether these goals are achieved
• What naming convention you will use for events, and other such taxonomies. (e.g. is it ‘songPlayed’ or ‘song_played’? More detail about this in our piece about best practices)

Agreeing on these processes together is a great first step in getting organization-wide buy-in, and making it a part of your culture. Once you’ve got a tracking plan, it’s important to identify who takes ownership over it—putting it down to “everyone” doesn’t work. You need that one /person to take responsibility and drive it forward.

You’re not adding these processes on top of others, you’re integrating them. If you want to build repeatable processes like these into your organizational culture, make them as easy as possible for teams to adopt them into their workflows. It’s unlikely that team members will want to disrupt their well-established workflows to accommodate new processes. Instead, see how those processes can seamlessly fit in with existing ones. For example, Amplitude makes this really straightforward with our command-line interface—this ensures developers can instrument your tracking plan easily, and accurately without having to leave their preferred environment.

Align your tracking goals with your business goals

If you’re building agile products (e.g. using the build, measure, learn framework), you will most definitely be using data to make decisions. However: when deciding what direction to go in next, don’t start with data—start with a question.

Firstly, what are you trying to achieve? Are you trying to get a new feature together, or run an experiment? Maybe you have a set of specific goals this quarter. Whatever it is, try not to think about what data might be able to do for you. Instead, build your culture towards asking the right questions, and seeing if you have the data to answer those questions. So, think about things like:

• The success metrics for your outlined goals or experiments
• The events you need to track in order to be privy to these metrics
• What actions you’ve already taken based on existing insight—did they work?

If you find you cannot answer these questions with the data you are collecting, it means you need to tweak your tracking plan. More data isn’t always the answer – but accurate data most definitely is.

Part of building a good data culture is helping the team understand that the way you use your data is your differentiator, not the data itself. Start encouraging natural curiosity, and celebrating the impact of making decisions based on internal insights.

A good data and analytics culture is an ongoing process

You can’t build a culture overnight. Enable your desired culture to grow by demonstrating the value of new processes and celebrating the resulting wins. Try to foster attitudes around using data to sense-check hunches and ideas, rather than collecting data because it’s “nice to have.”

Keeping event tracking top of mind across teams doesn’t need to be complicated at first. You probably don’t need to start with more than ten questions. Pin these down, reiterate these among teams, and work from there. There’s no need to optimize for every eventuality right from the beginning.

The advice outlined in this blog post is just a starting point. Once you get into a good rhythm with this, you’ll notice that the processes you’ve established are second nature among the teams; just like with writing code, tracking analytics will become a more standardized, auditable practice.

Using Amplitude makes this process extremely easy: your tracking plan exists as a dynamic document which seamlessly integrates into your team’s workflow. If you’re interested in trying out Amplitude for your company, create an account today or book a demo with our team to learn more.