How do medics diagnose diseases? They might get some help from the RAF
Running tests to diagnose diseases should be easy right? Let’s say you take a blood test – if the result is above a certain threshold, you’ve got the disease. Easy. Well, unfortunately it’s often more difficult than that, read on to find out why.
Blood tests, like any thing else in a population, have a range of normal values. Imagine they’re a bit like height. If you plot height of people on a graph, it would look like a “Bell-shaped curve” with a lot of people with heights around the average, and fewer people very tall or very short. This is called a “normal distribution”.
It’s the same with blood tests. Blood tests have a “normal range”, with most people having values around the average, and fewer at the extremes of higher and lower values. Let’s say we’re looking at a disease like hyperthyroidism (an overactive thyroid). You could work out a cutoff value (the dotted line in the picture below), which anyone with hyperthyroidism has a blood test above, and anyone without hyperthyroidism has a value below.
That would be great, but unfortunately it’s really really rare for that to be the case in real life. Often there’s an overlap between “normal values” and the range people with a disease have, like in the picture below. The problem with this is that if you use the same cutoff, there’s going to be some people who don’t have the disease with a value above the cutoff (called False Positives) and some people with the disease who have a value below the cutoff (called False Negatives).
So how can medics get around this? Well, to understand some of the techniques they use, let’s go back in history to the Second World War.
During the Second World War RAF command had a problem. They had a new radar system which scanned the skies looking for enemy aircraft. The problem with this is that occasionally it would pick up things that weren’t enemy aircraft and they didn’t want to scramble their Spitfires in response to a flock of birds or something. This situation is pretty similar to healthcare. There’s a set number of resources (whether that’s Spitfires or hospital beds), and they want to make sure they get the best combination of not missing something they should respond to (either not missing enemy aircraft or missing people with a disease) without using up their limited resources for things they didn’t need to respond to (whether that’s scrambling Spitfires to flocks of birds, or overinvestigating people who are actually healthy because they wrongly think they might have a disease).
So the RAF developed a clever system called “Receiver Operating Characteristic”. This moves the cutoff up and down, and sees how the system responds to this, in order to work out what the best cutoff will be.
Let’s look at how that works. In the graphs below, the cutoff moves up and down. The first one shows a cutoff somewhere in the middle. Most of the time the system works well, but there’s an overlap in the middle where there are errors. In about half of the errors, Spitfires are scrambled to respond to flocks of birds by mistake (shown in the graph as Big Bird off of Sesame Street). In the other half of errors, Spitfires are not scrambled when there actually are enemy aircraft. If they shift the cutoff value up, they miss more enemy aircraft but are scrambled less often, if they shift the cutoff down they miss less enemy aircraft but are scrambled more often when they don’t need to be.
Medical researchers use exactly this technique to work out the best cutoff value for diseases. If it’s too low they get more “False Positives”, if it’s too high they get more “False Negatives”. Any system has some compromises to it, and the cutoff value selected will be based on things like how severe the disease is, and whether you can get additional information from other things that might suggest whether you have the disease or not. This is why it’s really rare for medics to use just one test to work out if someone has a disease. Often it’s a combination of getting an accurate story from the patient, and doing a combination of other tests, whether they’re blood tests or imaging tests (like ultrasound, X-rays, CT or MRI).
And it that wasn’t complicated enough, you need to get your head around how some of these diagnostic tests work in real life, especially in rare diseases. To better understand it, let’s use an example. Let’s say we’re testing for inherited disease in babies. We’ve got an amazing test: if a baby is positive for the disease, the test is positive 100% of the time. Woohoo. And let’s say if a baby doesn’t have the disease, the test is negative 95% of the time. This sounds awesome, right? Let’s say the disease occurs in 1 in 1000 babies. It’s a reasonably rare disease, but the test sounds great. So we’ll test 1000 babies with it.
After the testing is done, a Mum brings her baby in to clinic because they have a positive test. You’re the medic in clinic, so how likely is it that this baby, with a positive test, actually has the disease?
I’ll give you a bit of time to think about it. Imagine the Countdown Clock ticking…
The chance of this baby actually having the disease is LESS THAN 2%. Wait, what? How come, the test sounded fantastic. Let’s look at the numbers. We’ve tested 1000 babies. The disease occurs in 1 in 1000 babies, so on average one baby out of the 1000 we tested will have the disease. This one baby will test positive, as our test is right every time a baby has the disease. Out of our 1000 babies, 999 won’t have the disease. 5% of them will test positive as we said that if a baby doesn’t have the disease, the test is negative 95% of the time. So there will be 51 babies with a positive test – one who actually has the disease, and 50 (5% of 1000) who don’t actually have the disease.
This is called the “False Positive Paradox”, and that’s why medics have to be careful we don’t always assume a positive test in a fairly rare disease is accurate.
So in this article we’ve seen how the interpretation of diagnostic tests is often quite complicated. This is why it’s rare for medics to rely on just one test, but use these tests as part of a whole workup, with different tests all contributing to the picture.
Blood tests, like any thing else in a population, have a range of normal values. Imagine they’re a bit like height. If you plot height of people on a graph, it would look like a “Bell-shaped curve” with a lot of people with heights around the average, and fewer people very tall or very short. This is called a “normal distribution”.
It’s the same with blood tests. Blood tests have a “normal range”, with most people having values around the average, and fewer at the extremes of higher and lower values. Let’s say we’re looking at a disease like hyperthyroidism (an overactive thyroid). You could work out a cutoff value (the dotted line in the picture below), which anyone with hyperthyroidism has a blood test above, and anyone without hyperthyroidism has a value below.
That would be great, but unfortunately it’s really really rare for that to be the case in real life. Often there’s an overlap between “normal values” and the range people with a disease have, like in the picture below. The problem with this is that if you use the same cutoff, there’s going to be some people who don’t have the disease with a value above the cutoff (called False Positives) and some people with the disease who have a value below the cutoff (called False Negatives).
So how can medics get around this? Well, to understand some of the techniques they use, let’s go back in history to the Second World War.
During the Second World War RAF command had a problem. They had a new radar system which scanned the skies looking for enemy aircraft. The problem with this is that occasionally it would pick up things that weren’t enemy aircraft and they didn’t want to scramble their Spitfires in response to a flock of birds or something. This situation is pretty similar to healthcare. There’s a set number of resources (whether that’s Spitfires or hospital beds), and they want to make sure they get the best combination of not missing something they should respond to (either not missing enemy aircraft or missing people with a disease) without using up their limited resources for things they didn’t need to respond to (whether that’s scrambling Spitfires to flocks of birds, or overinvestigating people who are actually healthy because they wrongly think they might have a disease).
So the RAF developed a clever system called “Receiver Operating Characteristic”. This moves the cutoff up and down, and sees how the system responds to this, in order to work out what the best cutoff will be.
Let’s look at how that works. In the graphs below, the cutoff moves up and down. The first one shows a cutoff somewhere in the middle. Most of the time the system works well, but there’s an overlap in the middle where there are errors. In about half of the errors, Spitfires are scrambled to respond to flocks of birds by mistake (shown in the graph as Big Bird off of Sesame Street). In the other half of errors, Spitfires are not scrambled when there actually are enemy aircraft. If they shift the cutoff value up, they miss more enemy aircraft but are scrambled less often, if they shift the cutoff down they miss less enemy aircraft but are scrambled more often when they don’t need to be.
Medical researchers use exactly this technique to work out the best cutoff value for diseases. If it’s too low they get more “False Positives”, if it’s too high they get more “False Negatives”. Any system has some compromises to it, and the cutoff value selected will be based on things like how severe the disease is, and whether you can get additional information from other things that might suggest whether you have the disease or not. This is why it’s really rare for medics to use just one test to work out if someone has a disease. Often it’s a combination of getting an accurate story from the patient, and doing a combination of other tests, whether they’re blood tests or imaging tests (like ultrasound, X-rays, CT or MRI).
And it that wasn’t complicated enough, you need to get your head around how some of these diagnostic tests work in real life, especially in rare diseases. To better understand it, let’s use an example. Let’s say we’re testing for inherited disease in babies. We’ve got an amazing test: if a baby is positive for the disease, the test is positive 100% of the time. Woohoo. And let’s say if a baby doesn’t have the disease, the test is negative 95% of the time. This sounds awesome, right? Let’s say the disease occurs in 1 in 1000 babies. It’s a reasonably rare disease, but the test sounds great. So we’ll test 1000 babies with it.
After the testing is done, a Mum brings her baby in to clinic because they have a positive test. You’re the medic in clinic, so how likely is it that this baby, with a positive test, actually has the disease?
I’ll give you a bit of time to think about it. Imagine the Countdown Clock ticking…
The chance of this baby actually having the disease is LESS THAN 2%. Wait, what? How come, the test sounded fantastic. Let’s look at the numbers. We’ve tested 1000 babies. The disease occurs in 1 in 1000 babies, so on average one baby out of the 1000 we tested will have the disease. This one baby will test positive, as our test is right every time a baby has the disease. Out of our 1000 babies, 999 won’t have the disease. 5% of them will test positive as we said that if a baby doesn’t have the disease, the test is negative 95% of the time. So there will be 51 babies with a positive test – one who actually has the disease, and 50 (5% of 1000) who don’t actually have the disease.
This is called the “False Positive Paradox”, and that’s why medics have to be careful we don’t always assume a positive test in a fairly rare disease is accurate.
So in this article we’ve seen how the interpretation of diagnostic tests is often quite complicated. This is why it’s rare for medics to rely on just one test, but use these tests as part of a whole workup, with different tests all contributing to the picture.
Comments
Post a Comment