Years ago, I heard a stupid philosophy joke: a student taking an intro philosophy class asks the professor “but how do we know?”; then, when the professor gives an answer, the student asks “okay, but how do we know that?” This keeps on going until eventually the professor gets fed up and says “by looking”.
I gather this is absolutely not how philosophers actually think about epistemology, but that final line has always stuck with me anyhow. I’ve repeatedly had suggestions to address issues at work—code quality is bad, our system design has fundamental issues, our management processes sap people’s autonomy and morale—met with the same refrain: “but how will we know?” How will we know that drawing better interface boundaries will improve our system design? How will we know that our codebase is better? How will we know that code quality or system design matters?
How will we know?
What I wanted to say each time: “by looking!”
I see an overwhelming desire to reduce complex questions to simple metrics. Not just using quantitative metrics as an input to a nuanced decision-making process but rather reducing decisions to “number goes up” or “number goes down”.
Some emphasis on metrics is a reasonable reaction to baseless shoot-from-the-hip decisions, but it’s morphed into a thought-terminating cliche and an instinctive political maneuver.
Tech companies are famously metrics-driven, but the obsession with metrics predates tech by decades if not centuries. I saw the same observation in Moral Mazes about US corporations in the 80s: “making your numbers” (which could well mean “juicing” your numbers!) was already a cornerstone of internal corporate politics, and internal politics was a cornerstone of corporate management.
Partly, people reach for metrics because they want to avoid politics: if you have a simple number to point to, nobody can disagree about whether the number went up or went down! No conflict, no consensus-building. This doesn’t work. All you’ve done is push the politics—the conflict, the consensus-building—into choosing which metrics to track and how to measure them. We might not have a conservation law for internal politics, but it’s close. Worse yet, this pushes the organization further from reality and forces a decision earlier in the process, when we collectively have less information to make the decision!
Simple metrics are also a great way to avoid making a decision. It wasn’t my fault, I just did what the numbers told me! Trying to avoid blame is just natural.
At the same time, people also do not want to rely on others’ judgement. “By looking” works if you know how to look—but what if you don’t? What if you do not have the experience or expertise to evaluate something yourself? Then you have to trust some other experts’ judgement. If the experts disagree, you have to figure out some way to reach consensus on a question you can’t answer yourself. And, sure, that is difficult! But perhaps “we figured out a way to remove expertise from key decisions to avoid difficult discussions” is not an outcome to aim for.
Some decisions really can be metrics-driven. Others should only be informed by quantitative measures. And some important decisions can’t be meaningfully quantified at all. Data can help you refine a design, but it can’t help you get to a fundamentally strong design starting from a blank page. At some stage you simply can’t avoid human judgement and taste.
So how can we build team cultures that can handle all three kinds of decisions?
At the very least, it means that sometimes “by looking” should be the answer!
