Claim Your
Product Coins
Do you know this excited feeling you get when you start working on a new feature? When you start going through all the things you’ll need to consider and plan ahead? There’s everything from the customer to the design and experience, dependencies, technology, the market, and the list goes on. As a Product Manager, you may have faced questions like:
I could go on for hours. I’m not sure how you feel, but I’ve had moments in my career where I wasn’t able to answer questions like that at all! In my previous job at a large-scale tech company, I learned a lot about how to define KPIs and metrics while building products for more than one million customers.
Recently, I’ve been reading many questions in Product Communities about KPIs and tracking like::
A couple of my mentees asked me similar questions. That’s why I’d like to share some basic knowledge about KPIs and metrics that will help you find the answers you need.
When I started getting deeper into the topic of measuring and quantifying projects and products, I got lost in gazillions of definitions and interpretations. Maybe that’s something you can relate to. I love collecting information from different sources, picking the best pieces, phrasing them, and simplifying them in an understandable way. When it comes to KPIs and metrics I need to admit that the best definition I’ve ever found was from Richard Hatheway, which doesn’t need any rephrasing in my opinion:
KPIs are a quantifiable or measurable value that reflects a business goal or objective (strategic) and how successful the business is in accomplishing that goal or objective. A metric is also a quantifiable or measurable value, but it reflects how successful the activities taking place are (tactical) to support the accomplishment of the KPI.
In short: KPIs are strategic and metrics are tactical. Metrics help to accomplish the KPI. Clear enough? 🙃
At first, that seemed pretty clear to me. That changed though, when I started experiencing the almost endless world of KPIs (and metrics). At some point, I had too many KPIs in my brain and things started going in circles. I got out of it when I started focusing on less instead of more. That’s why I want to share the 7 most important KPIs that helped me to do a significantly better job as both a Product Manager and a Leader.
There are many books and blogs out there educating people on the diversity of KPIs. One of my personal favorites is the book “Lean Analytics”. It’s a great read for Product Managers as well as Executives to help them gain a basic understanding of how to measure companies and products. The following KPIs and definitions contain theoretical basics as well as practical examples.
You can distinguish between quantitative (represented by numbers) and qualitative data (represented by e.g. text that can’t be easily quantified). I’ll also focus on the quantitative aspect of KPIs & metrics which will contain the notion of qualitative aspects. I’ll mainly call them “KPIs”. These principles can also be applied to metrics.
Note: I’ll write more about qualitative data and research in a separate article.
Until 2017 I hadn't heard of the term “North Star metric” or “One Metric That Matters” at all. The idea behind this (further) north star metric is to represent a clear measurable indicator for a company or a product. If you need to decide on one KPI that represents the core value of your product or feature, it should be the North Star metric. That doesn’t mean that other KPIs are less relevant.
Let’s look at Google’s north star metric:
The lower the time, the better. Google’s goal is to help people to find answers or search terms they’re looking for as fast as possible.
For YouTube, it’s the opposite:
The longer a user spends time on YouTube the better! YouTube makes money with the advertisements in its videos. The more and longer you watch their videos, the better it is for YouTube.
The North Star metric isn’t only important for measuring the value and success of a product. I’ve learned that it gives the whole organization a clear direction. The Product and engineering teams at Google focus on optimizing for better search results and fast loading times. Everyone in the company has a clear understanding of the value their product delivers. Obviously, these days, Google has more products and KPIs. The search engine is just one example. Defining a North Star metric for your company, products, or even team helps a lot in terms of alignment and strategic direction.
I believe every company should have a North Star metric to measure the business’s value & success. Or to be more specific: Two “north stars”...
The idea of leading vs. lagging KPIs blew my mind when I heard about it the second time! The first time I read about it and forgot. The second time an Agile Coach reached out to me after a meeting I had with my Engineering team discussing KPIs. He told me that my KPIs are defined nicely but are missing leading indicators. From that moment on, I knew I needed to change the way I define KPIs…
What are “leading” and “lagging” indicators?
Here’s an example:
Let’s imagine you’re building a landing page to sell a product of your choice. The goal of this landing page is to increase your sales. What would be the KPIs (or metrics) to track?
The “number of sales” is an excellent lagging indicator and lets you know how successful you were.
How can you ensure that your landing page is on track with your goal?
This could be the “number of visitors”, “the number of visitors leaving the page in less than, e.g. 1 minute” or other KPIs based on how you measure things. These are leading indicators. I also like calling them “health KPIs” because they tell me if things are going in the right direction (or not).
After understanding this concept I started always thinking in two ways when I worked on projects or user stories:
How can we measure the success (lagging) and how can we check if we’re on track (leading).
By the way: What’s your north star metric? 🤔
I like to call the Vanity KPIs the “Ego KPIs” because they make you feel “gooood”. Do you know the basic Google Analytics tracking data you get? To me, most of them are (lagging) Vanity KPIs. It makes me feel good knowing that the time spent on my landing page is on average 8 minutes! It makes me feel good knowing that 1.372 people clicked the “read more button.” Why did no one buy anything? This kind of KPI can hardly answer such questions. If you truly want to become “data-driven” it’s important to “read between the lines”. Especially, when you’re presented with these kinds of numbers.
That brings us to the opposite type of KPIs: The “Actionable KPIs.” I like calling them the real “feel-good KPIs.” Understanding data and knowing what to do is what really makes me feel good.
Actionable KPIs have, in most cases, a lead character. Instead of measuring the number of clicks on a button, you track something like: Time spent from opening the page to clicking the button. Alternatively, you can analyze “time spent from opening a page until the first interaction” instead of “time spent on the page.”
Common Vanity KPI traps:
Number of:
I like going the “extra mile” when I define KPIs and metrics instead of taking shortcuts and falling into these common traps.
Instead of tracking “downloads” you can track the downloads generated via the Facebook ad from the 1st of Jan to the 31st of Jan. If you generate downloads from other sources you can define similar metrics and dive deeper into each of them based on the results. It takes more brainwork and effort, but, you’ll love the results.
Note: Most of the examples mentioned above are metrics rather than KPIs.
Distinguishing between correlations and causalities is about understanding the root cause. That means when you define or review KPIs you want to be able to derive the right conclusions. We’re living in a fast-changing and complex world with a lot of dependencies. It’s important to know that either correlations or causalities are in most cases >1.
Let’s look at a simplified example:
During summer, the consumption of ice cream increases drastically compared to winter. At the same time, many doctors report increased cases of sunburns in the summer period. Does the likelihood of getting sunburned increase when you eat ice cream? Maybe? Maybe not! What we know is, there is a correlation.
What about causalities? A reason why people get more sunburnt in summer is that the sun is much stronger. People who visit the beach spend too much time in the sun and get sunburned (whether they do or don’t have sunscreen). Also, some construction workers work outside the whole year. They spend too much time in the sun as well and get sunburnt. We can go on with this example. You see, there is more than one causality in this case. Spending too much time in the sun causes sunburn. That’s the only way you can get sunburnt. The ways in which people get sunburnt are broad and ice cream is not the root cause.
When I define or read KPIs I always think about correlations and causalities. If I want to prove or disprove a hypothesis it’s important to me to look at this kind of indicator.
Note: If you’re standing too long in the queue to buy ice cream then it’s causality 🔥🍦
There are obviously many more KPIs and metrics that can be used that I haven’t mentioned in this post. If you want to learn more feel free to listen to our podcast episode with Ben Yoskovitz the co-author of the book Lean Analytics.
Organizations, no matter their size, produce and consume information and data all the time. Whether you set up a new department, a new team, build a new product, or optimize an existing one:
Good data help us to understand the past, make better decisions in the present, and forecast the future.
I like defining KPIs and metrics top-down. That means starting with the big picture. If you understand the “why’s” it’s easier to derive your KPIs from that. Whenever I define KPIs I focus on outcomes, not on output.
That’s why I always start with the North Star. If it’s already defined you can continue with your main product/feature or process KPIs on a leading and lagging level. While you define these it’s important to keep in mind the vanity vs. actionable dimension. After I’ve created my first draft I ask myself how I want to interpret the data later on. Do they help me and my team? Can I identify success and understand why my product is successful (correlation vs. causality)?
Starting from top to bottom and going back from the bottom to the top takes some time and brainpower. However, at the end of the day, the whole organization will benefit from it.
It’s important to be fast when it comes to shipping things these days. The Lean Startup movement, for example, has shown many ways to build, measure, and learn in a fast way. There’s one thing I must emphasize:
Don’t ship a feature before you have the metrics in place.
“If you can’t measure it, you can’t improve it” - Peter Drucker
