Measuring Product Performance

The two most common reasons why Product Managers measure product performance are:

• to determine the success and opportunities of a product you own
• to determine the success and opportunities of a competitor's product before entering a new market.

Assessing Your Product

When it comes to products you own, it is essential to look at your product, company and business model as a whole. All Product Managers are working on developing new products and features to ensure the success of the products and gain revenue and market share. But ask yourself:

• What does success look like as a whole for the product and the business?
• What are the company's goals?
• What are the company's mission and vision?

Companies are always looking at different personas to try to profile different users and customers of their product(s). Often they're referenced as demographics. Maybe they dominate a certain age demographic that is particularly profitable. But personas can also describe certain tiers of users and the way they are intended to interact with your product. Use as much of the data and analytics you can obtain from measuring those who use your product. Look back at past product documents and figure out whether that persona was initially targeted or was it a surprise. Figure out which personas are profitable and which are not.

Always look at your return on investment as well - what is our return on investment and projected return on investment in the future? How does this product impact return on investment of other products or features owned by the same company? These are all significant areas to look for data points.

Assess what opportunities remain in the product. Again, talk to users, and see what they like and don't like. The users will talk about their problems, but whether you can fix them is another story. Still, you'll want to know all the remaining possible opportunities when familiarising the product.

And finally, market size and penetration; how big is this market, is it growing or shrinking over time, and is it worth continued investment, or should we pivot or abandon the product? All of these decisions require a lot of metrics to help us make the best decisions possible.

Assessing the Competitor's Product

When assessing a competitor's product, a lot of research into the market is involved. When carrying out this research, you may want to look for answers to specific questions to get a feel for your competition and the market. For example, you could ask yourself:

• What does success look like at the competitor's company?
• What goals do they have, and what are they trying to achieve at a high level?
• What are the company's mission and vision?
• What other products do they have?

These questions can help us understand why they built this product and who they're trying to target.

When looking at personas, they may dominate a certain age demographic, leaving an opportunity for you for other ages. Or, maybe there's a more significant reason why they target that demographic, possibly because that's where most of the money is spent today. Remember, you won't have direct access to the product or the data and analytics that come with it, but you can look at things like keyword and traffic analysis to determine which search phrases they're targeting and their estimated traffic.

When analysing personas, try and look for features the competitor does not offer. Maybe you could come in and make that pain point a benefit in your product and quickly gain market share. When researching, you could talk to existing users of competitors in that space and get to know their pros and cons.

But always analyse the market and find answers to questions such as:

• What's working and what's not?
• How big is the market size today? Is it growing or shrinking?
• Does your competitor dominate the whole market today or a tiny portion of it?

These data points will help you make the best decision possible as the Product Manager.

Product Metrics

When using metrics to measure your product's performance, there are two types to measure:

Baseline metrics - a single number that's considered acceptable and you shouldn't see fall below it. For example, if we determine our baseline traffic per month is 4,000 customers, we don't want to see anything less than 4000 per month going forward.

Threshold target metrics - this can be a range that's considered acceptable by the business, similar to baseline, but gives you the important range that you're shooting for.

What can we measure when defining these metrics? Here are a few key ones:

Revenue - the top‑line income figure of how much money we're bringing in. Whilst this is an important number to look at, you'll want to dig into how much money you're spending to drive that revenue.

Churn - this is being used more and more, especially as we see more SaaS (Software‑as‑a‑Service companies being created. Churn essentially means cancellations. It's not uncommon for some companies to experience a gain in customers each year, as well as churn. For example, Netflix may see 5% of customer churn, but year over year, 20% more customers sign up, leaving them with 15% more customers year over year.

NPS (Net Promoter Score) - NPS is another popular metric and is a way of understanding how your users or customers feel about your product. Often in data, you'll have many metrics about what they do in your product, but it's important also not to forget how they're feeling and thinking about your product.

Cost per acquisition - how much does it cost your business to find a new paying customer? You may have to pay for ads on Google, Facebook, Instagram, LinkedIn, etc. From all of those services, you can assess your conversion rate and then look at how much it costs to run those ads to gain new customers. It's important to know how much you're spending on marketing to understand if you're turning a profit from the users who do pay.

Lifetime value - How often do those paying users come back and buy more? Of all your paying customers, do 99% buy one thing and then never return? That could be an opportunity to try to add new marketing to previous customers, or it may give you a better understanding of how much you can spend on ads before losing money, assuming each user will only buy one time.

Orders - an essential but straightforward metric. How many orders overall have you received given a time frame, for example, orders this month or this year?

Visits - how many users visit your website or application? Visits are important, especially when looking at funnels of users. It can also indicate how many of your current customers actually use the product they have signed up for. When looking at visits to a site or application, you can also track other indicators such as sessions (the time between a visitor's first arrival at a page on the site and the time they stop using the site) and bounce rate (single-page sessions divided by all sessions, or the percentage of all sessions on your site in which users viewed only a single page and triggered only a single request).

Key Performance Indicators (KPI)

A key performance indicator is a metric that demonstrates how effectively a company or product is achieving key business objectives. Of all the metrics you can track, which ones are truly the most important to you and your business when trying to understand the current and future success of the product? These can be tough to choose because, as we looked at, there are so many metrics to choose from, and it is unique to each product.

There are three essential areas when setting KPIs.

1. Set a target - every single KPI should have a target you're hoping to achieve for a certain period.
2. Data source - explicitly define where the data is coming from, so there can be no question of bias or accuracy.
3. Reporting frequency - How often will this data be tracked and shared among stakeholders?

Remember to keep the KPIs measurable and time-bound.

KPIs could come in various forms. They could be more quantitative, like measuring churn rate by understanding how many customers are leaving our company to move to our competitors in a given period. They could also be qualitative metrics too, especially for an early-stage company. For example, measuring the response to new feature rollouts and ensuring a specified satisfaction level is received. This can be achieved through a survey to gauge if the product fits the market. KPIs could be internal too (for example, measuring employee productivity) to make internal decisions such as making staff more efficient or calculating when new team members are required as the business grows.

Objectives and Key Results (OKR)

Objectives and Key Results, or OKRs, for short. These are business goals paired with measurable results that help determine the objective's success over time.

For example, let's say that the company has the following objectives:

• Raise awareness of the company's brand
• Increase monthly active users across all products
• Every product should create a consistently positive user experience

The three objectives the company is working towards can help the Product Managers make product decisions and even pick KPIs. Taking one of those objectives as an example - 'the product should create a consistently positive user experience'. We can set the following key results:

1. Reduce customer churn by 5% in Q2. So we want to get 5%, measurable, and Q2.
2. Increase the net promoter score (NPS) from 6 to 8 by Q4.

These sorts of key results will really help narrow in on the metrics and product decisions.

Remember to always keep up with your business objectives and link these performance indicators back to your business's objectives, mission and vision.