Unmasked Negative

Unsurprisingly, my test came out negative.

Before getting the result, I had called the Ipsos MORI helpline to see if they could give me more information about the test and its accuracy. I must not have been the first one to enquire about accuracy, because the helpful operator had a prompt answer: ‘If you’re positive, you definitely have the virus; if you’re negative, you most probably don’t have it, but you can’t be certain’. He was not as clued-up about the test manufacturer, but he came back to me after checking with his supervisor: ‘I believe it is called Wuxi ‘.

So apparently I have taken a maximum Specificity Smoking Gun test: a positive result would have been conclusive proof of infection, irrespective of the Base Rate. But I came out negative — as I almost surely expected, given that, without symptoms, I could safely assume my prior probability of infection to be as low as the ONS Base Rate estimate. In the meantime, this had gone up to 0.09% (with a 95% confidence interval of 0.04% — 0.19%), or 1 in 1100 — curiously almost identical to the assumption in my original virus story:

(Strangely enough, given the media penchant for alarming but meaningless statistics, such ‘50% increase’ in the infection rate from a week earlier remained unreported).

However small my priors, seeing them further reduced to near zero in the light of a negative test result was a good feeling. Me being me, however, I called the helpline a second time after the results and asked the same questions. Lo and behold, I got… different answers. This time the operator — a different person — while reassuring me that the test was ‘quite accurate’, would not commit to giving ‘percentages’. And the reported manufacturer was different — ‘either Wondfo or Orientgene ‘.

Oh well. None of the three Chinese manufacturers report any accuracy information on their websites. But as long as their tests are ‘quite accurate’ — i.e. somewhat confirmative — a negative result from a low Base Rate gives me, and people around me, virtual certainty that I am not infected.

But what if the result had turned out to be positive? In that case, whether the first operator was right would have mattered a great deal. A positive result from a maximum Specificity test means certain infection. But with a low Base Rate of infection even a small deviation from 100% Specificity means that a positive result is very likely to be a False Positive.

Say for example that, as in the Table below, Specificity is not 100% but 95% — still very high. And say that Sensitivity is 70%. With the current ONS Base Rate of 0.09%, 9 out of 10,000 people have the virus. Of these, 6 will test positive and 3 will test negative. Whereas of the 9,991 people who do not have the virus, 500 will test positive and 9,491 will test negative. It follows that PP, the probability of infection given a positive test result, is as low as 6/506=1.25% (allow for rounding). Whereas NP, the probability of infection after a negative test result, is 3/9,494=0.03%.

In other words, of the 506 people who test positive, only 6 are True Positives — 1 out of 80 — and 500 are False Positives. Whereas of the 9,494 people who test negative, 9,491 are True Negatives and only 3–1 out of 3,516 — are False Negatives.

You can play with the blue numbers on this spreadsheet. You will see that even with a 99% Specificity PP remains small at less than 6% — 1 out of 17. Whereas NP is still approximately one third of the Base Rate — 1 out of 3,664.

Only with maximum 100% Specificity will PP jump all the way to 100% — no False Positives — whereas NP is even smaller at 1 out of 3,701.

You can also see that results are not very sensitive to Base Rate variations. 0.09% is the average infection rate in England, but the ONS estimates that it is currently higher (56% higher!) in London, at 0.14% (with a 95% confidence interval of 0.04% — 0.32%):

Plug 0.14% or even 0.32% in the BR cell and you will see that the resulting increases in PP and NP are small. That is why, although I was pleased with the negative result, it was what I almost surely expected — just as I would almost surely expect to draw a white ball from an urn containing 1100 white balls and 1 black ball — or even 313 white balls, if I plug the upper bound of the London confidence interval. After the test, my urn contains many more white balls, but there were plenty before.

Obviously, all the numbers above rest on the ONS Base Rate estimate, which is the right prior assumption in the absence of symptoms. Raise BR to, say, 50% — which would be a reasonable assumption if I had sufficiently specific symptoms — and the numbers are entirely different: PP is 93% and, crucially, NP is 24% — a 1 in 4 chance of a False Negative.

This raises the question: what is the accuracy of the tests used in the ONS study? The answer is in Paragraph 10 of their methodology guide: “we think the sensitivity of the test that the pilot study uses is plausibly between 85% and 95% (with around 95% probability) and the specificity of the test above 95%”. There is no information about the test manufacturers but, assuming they are the same or similar to the ones used by Ipsos MORI, then the first operator was wrong: the test I took is not a Smoking Gun. Based on BR=0.09%, a test with, say, 90% Sensitivity and 97% Specificity further reduces NP to 0.01% — 1 out of 10,769 — which pleases me even more. But PP is not 100%: it is 2.6%.

Think about it: 10,000 people are tested and 308 unlucky ones come out positive. But most of them — all but 8 — are False Positives. The ONS can account for test inaccuracy and cut the 3.08% positive rate down to arrive at the 0.09% Base Rate. But what do they tell the positives? What are they counted as? The same is true for Ipsos MORI and whoever is testing asymptomatic people in a low Base Rate population. How many of the reported cases we hear about every day are False Positives wrongly counted as True Positives?

Anyway, I am a happy negative. Yes, I might still be the 1 in 10,000 unlucky False Negative (or 3 in 10,000 if BR=0.32%). And let’s add to it the chance that, despite dutifully following precise instructions, I might have bungled my self-test — a tricky affair: I was wary about the nose poking, but nudging my tonsils and the nasty gagging reflex that came with it was worse.

But overall it’s a tiny risk, much smaller than other risks I happily live with every day.

Obviously, not being infected today does not mean that I cannot get infected tomorrow. So I will continue my social distancing and hand washing. But I will again run the risk I took in questioning the rationale of blanket lockdowns. Call me a Palm Beach crackpot — what’s wrong with the place? — but now that I know I am not an asymptomatic carrier merrily going about infecting other people, I won’t wear a mask if I don’t have to.

Leaving effectiveness aside, there is no point bearing the cost of reducing a risk that is small enough already.

Originally published at https://www.massimofuggetta.com on July 4, 2020.