Extreme Clarity pt2: Percentages and Percentage Points

When is 10% not 10%?

My current employer, like most large companies, is pretty data-driven. At scale, metrics are often the only way to rationalise about what’s happening in the community and track product progress.

Which is why it’s so important to ensure that we’re being extremely clear when describing metric movements. Getting this wrong might mean over- or under-estimating your team’s work, or some customer trend — leading to poor decision making, incorrect prioritisation, or misattributing credit.

In my experience, there’s no more common mistake than confusing the difference between percentages and percentage points when describing metric movement.

Getting this wrong can result in the reader leaving your communication with a VERY different understanding of the situation than you intended.

Example time!

Consider this line from a product team's recent weekly status report:

We increased click-through rate by 10%!

But digging in, this was a misleading statement. The problem is that click-through rate is a metric that’s already expressed as a percentage. Their baseline was 13% and their new CTR was 23%.

This isn’t a 10% increase. It's a 10 percentage point increase. A 10% increase would mean CTR had increased from 13% to 14.3% (13*1.1).

Now, you can also express that as a percentage increase, but it’s not "10%", it’s 100*(23-13)/13 = 77%!

10% vs 77%: That’s quite a difference - and if two readers have two different understandings of the same thing, and those understandings differ by such a wide margin, you can easily see how this might lead one to make very different decisions. Over time, this may compound to drastically different outcomes.

So in this case, it was wrong to use the symbol "%" - the correct articulation should have been:

We increased click-through rate by 10 percentage points.

But this also is somewhat problematic: it's missing context.

Moving something from 1% to 11% is likely a big deal (10x!). Moving something from 80% to 90% is perhaps less impressive.

To deliver extreme clarity, you really need to say BOTH (or include the from & to for context).

So ideally, this should have been articulated as:

We increased click-through rate by 10pp (13% to 23%).

or

We increased click-through rate by 10pp (+77%).

Gives both clarity and context, and leaves no room for ambiguity in the mind of the reader (note that of these two, I personally think the former is the clearer/better articulation).

---

Another Example

Consider the following line from another recent product team’s status report:

Over the half we reduced prevalence by 24%, 3.5% of which can be attributed to our classifier improvements.

3.5% of 24% is just 0.84 percentage points - and implies 96.5% (100-3.5) of the impact was driven by efforts others than classifier improvements - doesn’t look great for the classifier work!

But upon deeper digging, what they actually meant was 3.5 percentage points of the reduction was attributable to the classifier improvements. This means 14% of the impact was driven by the classifier work. That’s 4x more of a contribution than my reading of the original statement!

Now, another problem here is that I’m now suspicious of if there really was a 24% reduction in prevalence. The only thing making me thing that might be right is that a 24 percentage point reduction in prevalence is unlikely given that VPV prevalence, for example, is usually expressed in basis points (i.e. 1/100th of a percentage point).

Ideally, this would have been written in a way that’s extremely clear:

Over the half, we reduced prevalence by 24% (43 bips to 32.6 bips), 3.5 percentage points (14%) of which is attributable to our classifier improvements.

Again, SO MUCH CLEARER - and no room for ambiguity in the mind of the reader. Everyone can understand, unambiguously the contribution and the context of this work.

So, when referring to a change in a metric that’s already expressed as a percentage

• Make sure you’re not using “%” when you should be using "percentage points", "p.p." or "points".

• Ideally include both the % and pp movement — or, even better, the before & after value of the underlying metric.

As ever, XKCD got there first: