The Problems of Product "Differentiation"

29 Jan 2021

It seems obvious that there are advantages to standing out from the crowd—to differentiate your product or service from the others in your space. But successful differentiation doesn’t mean that people will choose your product. In fact, asking and acting on the issue of differentiation has at least two important problems when seen through the lens of Jobs to be Done (JTBD).

The Pauling Effect

22 Jan 2021

A friend of mine who is very bright makes his living as a data scientist that specializes in fantasy sports. Recently, he said, “I’m steeped in data and modeling. And I looked at the studies and some of the models on SARS-CoV-2 and hydroxychloroquine [HCQ] and found some of them persuasive,” to the point that he wanted to acquire HCQ and ignore advice to wear a mask.

Why should business care about Simpson’s Paradox?

9 Sep 2020

Chances are you’ve never heard of Simpson’s paradox, but it matters. In sum, Simpson’s paradox—sometimes referred to as the “reversal paradox”—says that how we choose to analyze a data set may cause us to draw opposite conclusions. For instance, in the 1980s, there was a study about the most effective way to treat kidney stones, using existing outcome data. When looking at the outcomes for all patients, procedure A showed better results. However, when scientists cut the data by stone size, they found that for all stone sizes, procedure B was better. To make matters worse, there is no boundary to Simpson’s paradox. If researchers had cut the data by gender or height or BMI or whatever, they could have seen reversal after reversal after reversal.

Anomalies: Improving Good Theory

29 Aug 2020

There is another important consequence of the power of anomaly: all good theories must be susceptible to anomalies, otherwise there is no progress. Correlation is like a sliding scale of confidence–it’s akin to an analog signal that we receive with varying degrees of fidelity. Consequently, we can seldom state definitively if some event is an anomaly or merely “noise.” On the other hand, a causal model is more like a digital signal—it gives us discrete “yes” or “no” indicators for anomalies. As we mention in the first chapter, the causal approach to manufacturing created clear go/no-go signals that either kept work moving or initiated an investigation into a defect. Without the clear binary signals, sub-par work would inevitably sneak through.

How an Anomaly Focus Eliminates Bias

The current widespread focus on the problems of replication in scientific studies—especially those of the social and biomedical sciences—has been to invest in ever more sophisticated analytical tools and techniques: more data, more sophisticated tools of analysis, faster computing, etc. All of these efforts have missed an alternative solution: causality. Most research in biomedical and social science begins with data analysis. But data analysis alone is not capable of yielding a genuine causal model. Essentially, all analysis is subject to the problem of induction—roughly, correlation is not causation.

False Negatives

26 Mar 2020

A few days ago I wrote a post on how easy it is to have “false negatives”—negative for virus, but the person is sick. I made a table to show how easy it is to overestimate the efficacy of a test.