Misunderstanding Measurement Motivations
Or, why we measure and how it can go wrong.
Business is obsessed with measurement. Graphs must go up and to the right! But what if the problem isn’t what we’re measuring, but why we are measuring it?
In this post, I'll build a taxonomy of why we measure, then explore what happens when we measure for one reason but interpret the results for another.
Exploring - Measuring to Learn
When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be. (Lord Kelvin)
The purest form of measurement comes from science. This category focuses on creating, organizing, and predicting knowledge about systems.
We measure for scientific discovery and explanation to reveal fundamental principles and causal relationships in natural and social worlds. We use systematic classification to create ordered systems that structure knowledge and define relationships. Through social and environmental monitoring, we track the state and trends of populations, societies, and environments over time. And with forecasting and prediction, we use data and models to estimate future events, behaviours, or outcomes.
Optimizing - Measuring to Improve
Measure twice, cut once! (traditional carpentry quote)
These measurements inform choices, guide actions, and manage outcomes.
We conduct individual assessment and diagnosis to evaluate specific traits or conditions that inform decisions like hiring, treatment, or educational placement. We measure for process improvement and quality control to optimize efficiency and quality in systems, from manufacturing to service delivery. Through resource allocation and planning, we guide strategic distribution of finite resources for maximum impact. And with risk management and safety metrics, we identify and mitigate potential threats to protect assets and well-being.
Standardizing - Measuring to Align
A standard is a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose.
These measurements create shared frameworks necessary for cooperation.
We establish standards and interoperability by creating common references that enable seamless interaction between systems and organizations. We measure to ensure accountability and compliance, verifying adherence to laws, regulations, and ethical standards. Through measurement, we facilitate exchange and trade by providing objective bases for fair exchange of goods and services. And for legal and evidentiary determination, we provide objective evidence for resolving disputes and making judgments.
Misaligned Measurement
The real problems emerge when we confuse these functions. Here's how misaligned measurement creates dysfunction:
Exploring ↔ Optimizing
When we confuse discovery with action, measurement loses its way.
Treating KPIs as universal truths rather than optimization tools leads to perverse incentives such as measuring code by lines written (or catching rats). Conversely, rigid classification systems meant for exploration can stifle innovation when treated as action items. Fixed backlog categories ('must-have' vs 'nice-to-have') bury transformative features that don't fit neatly into predetermined boxes.
When discovery metrics are mistaken for performance metrics, experimentation becomes risky, and learning slows down.
Exploring ↔ Standardizing
Discovery metrics become dangerous when shared whilst standards can blind us to new insights.
A/B test results shared widely accidentally turn into performance reviews, stifling the experimentation they were meant to encourage. Monitoring metrics like NPS stop revealing insights and start shaping narratives; the story becomes the goal, not understanding. In the other direction, interoperability metrics can entrench existing tech. What was meant to enable becomes a constraint on discovery.
When exploratory measures become institutionalized, they stop revealing the unknown and start enforcing the expected.
Optimizing ↔ Standardizing
When optimization metrics become institutional standards, or compliance becomes the optimization target, behaviours change
Resource allocation metrics create their own gravity: what gets counted gets funded, so easily measured front-end tweaks overshadow critical difficult to estimate architectural work. Individual assessment metrics like velocity or bugs per developer foster competition over collaboration. Meanwhile, evidentiary requirements for legal purposes (SOC2?) surround product architecture, making teams build defensively to satisfy audit trails rather than iterating boldly. Risk management metrics meant to guide decisions suppress innovation when they become compliance checkboxes.
Optimizing isn’t standardizing. Confusing the two locks teams into the lowest common denominator solutions.
Putting It All Together
Measurement works when motivations are clear and fails when its purpose gets misread. When exploring metrics become control levers, optimizing metrics become gospel, or standardizing metrics kill user experience, we've lost the plot and created dysfunction.
Measure with intention. Interpret with nuance.
Define upfront whether you're exploring, optimizing, or standardizing. Keep those lanes separate to reduce gaming, resistance, and narrative distortion.
When friction arises, ask:
Am I measuring to explore, to optimize, or to standardize?
Who is the audience, and what will they do with these numbers?
What behaviour might this metric incentivize, even unintentionally?
Measurement shapes culture. It signals what matters. Using it wisely means choosing why you measure before deciding what to measure.