Insights from the talk by Cássio Scozzafave, Product & Experimentation Specialist at Ab-InBev
During Product Camp 2024, Brazil’s largest product event, Minders took the stage at the Content Aquarium with exclusive talks organized to promote practical and inspiring discussions. Cássio Scozzafave, Product and Experimentation Specialist at Ab-InBev, brought to light one of the most discussed topics in the world of technology and business: how to identify and use metrics to make strategic decisions. The talk “Essential Metrics: Strategic Decisions Driven by KPIs” revealed practical approaches to transforming data into real results.
If you missed the chance to attend, we’ve gathered the main takeaways for you to apply in your daily work.
1. The Golden Rule of Metrics
Cássio began his presentation with a fundamental premise: “If a metric doesn’t drive a change in behavior, it’s a bad metric.”
This concept, widely advocated in the book Lean Analytics, served as the foundation of the talk. He explained that good metrics are those that:
- Help understand the context of a situation.
- Facilitate strategic decision-making.
- Have a direct impact on business actions and results.
Metrics that are only used to «fill dashboards» or impress in meetings, like likes or number of downloads, are classified as vanity metrics. While they may seem important at first glance, they rarely move the needle on real outcomes.
2. The Importance of Narrative
According to Cássio, a metric alone doesn’t tell the full story. Context is essential. He illustrated this point with an example from Zé Delivery, where he works directly with experimentation:
«If I say that Zé Delivery made 5 million orders last month, is that good or bad? Without context, it’s impossible to know.”
By building a narrative that includes historical growth and market conditions, a standalone metric comes to life and starts to guide concrete decisions.
He also brought up the concept of a well-constructed narrative to align teams and stakeholders. For this, it’s essential to answer:
- What is the company’s current context?
- What are the product or service’s goals?
- How does this metric reflect the strategy and challenges faced?
3. Relevant Metrics: How to Identify Them?
Cássio presented a practical checklist to evaluate if a metric is truly useful. According to him, good metrics are:
- Understandable: Can everyone on the team interpret it?
- Reliable: Is the data source solid, and can the numbers be validated?
- Comparable: Are there benchmarks or historical data that help measure progress?
- Relevant: Do they connect to the company’s strategic goals?
He emphasized that the relevance of a metric depends on the context. For example, while NPS may be essential for measuring satisfaction in B2C companies, in B2B contexts with fewer customers, this metric may be inappropriate.
4. Metrics Applied in Daily Work
During the talk, Cássio shared a notable example of how he used metrics at iFood. He identified that the contact rate (support call rate) was high during promotions, generating operational costs. After optimizing communication with restaurants, he managed to reduce this rate by half. However, he was surprised by feedback from the strategic team: reducing costs wasn’t the priority at that moment, but increasing the user base and orders was.
This experience reinforced the need to align metrics with the product and company’s strategic goals.
5. Visual Tools: The Enhanced Opportunity Tree
Inspired by Teresa Torres’ model, Cássio presented an improved version of the opportunity tree. He highlighted how integrating relevant metrics into each branch helps identify clear hypotheses and validate solutions more efficiently.
The tree consists of:
- Main Objective: What the company wants to achieve (e.g., increase revenue).
- Supporting Indicators: Metrics that support the objective (e.g., new customers, average price).
- Hypotheses: Specific actions to move the indicators.
6. An Invitation to Reflect
Cássio closed his talk with a provocation: “What are you doing to improve the product culture at your company?” He emphasized the importance of involving all stakeholders in the process, educating teams, and creating shared responsibility around decisions driven by metrics.
Cássio Scozzafave
