Should People Avoid Whole-Body Screening Info?

The most controversial part of last week’s article on the Midjourney ultrasound scanner was medical experts’ recommendation against whole-body screening (including existing whole-body screening technology using MRI).

Isn’t this crazy? Whole-body screening can save lives by detecting serious diseases like cancer. The experts counterargue that it finds so many false positives - minor zit-like imperfections that would never have caused problems, but which cost patients time, money, anxiety, and side effect burden to investigate - that it ends up net negative. But isn’t this just a problem of setting thresholds correctly? Can’t you commit to only investigating the most obviously bad things, then ignore the rest?

This seemed like an interesting problem to investigate in more depth, so I’ve tried to get numbers. These are rough estimates loosely based on parameters extracted from unsatisfactory studies1 - please don’t take them seriously as exact values, just as right-order-of-magnitude estimates. We’ll focus on whole-body MRIs, since this is a well-studied existing technology, then speculate later on how the results might generalize to whole-body ultrasound.

For every 1,000 seemingly-healthy people who get whole-body screening MRIs:

• 680 look fine and no follow-up is needed.

• 300 have mildly concerning findings. They’re told to follow-up with specialists, get further tests, or come back for more imaging later.

• 20 have extremely concerning findings and get immediate biopsies (surgeries to collect tissue samples from the area).

• Of those 20 people who got biopsies, 10 turn out to really have some serious disease. This is usually cancer, and for simplicity we’ll focus entirely on cancer going forward.

• Of those 10 cancer patients, 4 end up living longer and healthier lives because their cancer was detected early. The other six either have such slow-growing cancers that they would never have noticed before dying of something else, or such deadly cancers that detecting them early doesn’t help, or would have been found by standard screening so soon afterward that the extra screening didn’t buy meaningfully more time.

• Meanwhile, the 300 people who followed up with specialists and got extra tests will spend some number of years seeing more doctors and getting more tests and waiting and seeing, and eventually for 4 of them this will result in detecting some dangerous condition in a way that causes them to live longer and healthier lives.

So in total, 8 of the 1,000 people have benefited. The average benefit among those 8 people is 4 quality-adjusted life-years, so the total benefit from the scans is ~32 QALYs.

What’s the total cost?

• The original 1,000 scans cost about $2K each, and each takes about 3 hours of patient time (one hour for the scan itself, but we add two hours for driving, waiting, arranging the appointment, etc).

• The 300 patients who follow up each cost about another $2K in extra scans, tests, and doctor visits, and let’s say this takes them each ten hours (probably over the course of several years; some of what they’re doing is waiting and seeing whether the anomaly gets bigger vs. goes away). Rare and minor side effects of the testing might contribute -0.005 QALYS per person.

• Additionally, these people are anxious. 29% of them report “moderate” or “severe” distress. It’s easy to trivialize this and round it off to meaningless, but it’s doesn’t feel trivial when it’s happening to you or a loved one. In my day job as a psychiatrist, I sometimes encounter patients having mental breakdowns (the polite term is “adjustment disorder”) over potential cancers that they’ll have to wait six months to retest. The literature quantifies this as an average cost of 0.01 QALYs per uncertain test result.

• The 20 biopsies each cost about $5K and take about 10 hours. Biopsies are pretty minor as surgeries go, but there’s always some risk of post-surgical complication, and in the end these probably cost 0.04 QALYs each. Add the extra 0.01 for anxiety, and it’s a total of 0.05.

• We’ll very optimistically assume that there’s no extra cost to being diagnosed with a cancer that doesn’t require extra early treatment. The doctor says “You have cancer, but it’s a very slow cancer that will take thirty years to harm you, and you’re eighty years old, so this doesn’t matter. Just be chill.” This doesn’t always work in real life, but we’re trying to imagine a best-case scenario.

• We’ll very optimistically assume that after enough tests, the smartest doctors can distinguish cancers that should be treated and cancers that shouldn’t be treated with 100% accuracy, or at least that the number of cancers caught according to the above statistics is at some threshold where falsely identifying a cancer as treatable is rare.

• We’ll assume that the extra cost of treating the treatable cancers is zero. Why assume this? Because if we didn’t detect the cancer early, we would have detected it later, and still had to treat it, and the cost of treating small early cancers is probably no worse - and likely better - than the cost of treating large late cancers (even though treating the latter is less likely to work). So zero is actually close to an upper bound!

So total costs are $2.7 million, 6,200 hours of patient time, and 5 QALYs.

32 QALYS ←→ $2.7 million + 6,200 hours + 5 QALYS
27 QALYS ←→ $2.7 million + 6,200 hours
25 QALYs ←→ $2.7 million
$108,000 per QALY

So, converting everything to the same units, whole-body screening costs $108,000 to save one quality-adjusted life year. Usually in health economics, $100,000 per year of healthy life saved is considered the bar for a good cost-effective intervention (though other sources say $150,000). So this is somewhere around the bar, not unambiguously good or bad.

Could we improve this number by refusing to follow up on ambiguous results?

I think these numbers assume that we’re already being pretty rational in ordering followup. We’re following up on the 300 cases with minor findings because that genuinely discovers 4 extra cancers, in a way that’s more cost-effective than the original round of screening. If doctors irrationally followed up on truly minor results, then that would make the cost-benefit ratio worse.

What if you’re personally rich and $2,000 means nothing to you?

In this situation, wouldn’t the benefits obviously outweigh the costs? The majority of the costs in the analysis above were financial, but this hypothetical rich person can ignore them.

This should be surprising, since every statement I’ve seen about the costs of whole-body screening downplays the financial angle and says that disadvantage is in anxiety, follow-up, and side effects from additional testing. Let’s double-check it.

This individual is buying a 1/125 chance of gaining 4 QALYs (=0.03 QALYs in expectation) at a cost of $0 (money is meaningless to him!), 3 hours of his time, plus a 1/3 chance of -0.015 QALYs of anxiety and testing side effects and 10 additional hours of medical appointments, plus a 1/50 chance of -0.05 QALYs from a biopsy plus another 10 hours of medical appointments. Now the calculus is 0.03 QALYs benefits vs. 0.006 QALYs + 6.2 hours costs = 0.024 QALYs benefit vs. 6.2 hours costs.

0.024 quality-adjusted life-years = 210 quality-adjusted life-hours. Even if you assume that half of those are spent sleeping or doing meaningless work, sacrificing 6.2 good hours to gain 105 good hours is a great trade.

It seems like the rich person who doesn’t care about money should do this!

Wait, can we really compare life-hours spent in medical treatment to life-hours gained from avoiding fatal diseases?

When I think about this harder, there’s something suspicious here.

Suppose there was a medical procedure which had a 5% chance of catching a disease which, if undetected, would have taken four years off your lifespan. On average, getting this scan would add ~900 good hours to your life. So a rational person should be willing to spend 899 hours scanning for this procedure (since in expectation they would benefit.)

But 899 hours of scans would look like going to your doctor’s office for one hour every weekday for three years! And in real life, only ~half of people bother to go to regular checkups, which takes one hour per year and probably have effects nearly this big. If any real person went to their doctor for an hour daily for years to decrease their risk of cancer by 5%, we would call them an insane hypochondriac! And there are dozens of things the average person could do for an hour a day to vastly increase their health (like exercise, or cook nutritious food), and most people don’t bother.

But maybe this is irrational? Sure, most people would hate going to the doctor every day, but if people are willing - even excited - to get whole-body MRIs, then it seems like we should let them, since this is more rational than the irrational normal human behavior we’re comparing it to. Granting that people have weird cognitive biases, it seems counterproductive to communicate the efficacy of whole body MRI by translating it into a domain where they have a bias, then telling them to avoid the whole body MRI because their bias makes them avoid the other thing.

So maybe the correct guidance is something like “The cost-benefit calculation for this whole-body MRI is worse than for many other good things which you don’t do. You’re not doing those other good things because you’re an idiot. However, that same idiocy has led you to believe that you’re somehow getting a leg up on the medical cartel’s evil conspiracy by getting a whole-body MRI, and that’s made you actively delighted about improving your health in this one specific case. So even though by your usual idiotic values you shouldn’t, in reality you should, so go ahead.”

What problems did our previous spherical cow analysis leave out?

The cost-benefit analysis above looks pretty good, at least for certain types of rich people. Why might we be reluctant to trust it?

First, it uses many statistics that I called “order-of-magnitude estimates”, and any of them could be off by an order of magnitude.

Second, it assumes that cancer is the only disease. There’s probably some extra benefit from finding non-cancer diseases (and maybe some extra cost from finding non-cancer false positives).

Third, it assumes that doctors do a good job. A few years ago, my aunt had some concerning symptoms and got a scan. The scan showed she probably had cancer. The doctor inexplicably ignored it for several months and she died. RIP. The doctors in studies usually do a better job than random other doctors, because they’re somehow linked to the sorts of prestigious and thoughtful institutions that conduct research, plus they know they’re being watched. But if you get a doctor more like my aunt’s, you’ll pay the same costs in money, time, and side effect burden, but get zero benefit.

Fourth, it assumes that the average person’s response to being told that they have a slow-growing cancer which will won’t hurt them is “Thank you for this information; as a perfectly rational agent, I will now proceed to never worry about this again.” But in fact, many people become extremely distressed, nocebo-effect themselves into psychosomatic symptoms, and maybe doctor-shop until they find someone willing to give them counterproductive chemotherapy. If even a few patients do this, it quickly erodes our life-years-gained calculation.

Fifth, it assumes that when all the most accurate tests are done and all the best doctors have been consulted, it’s at least possible to distinguish the cancers that should be treated from the slow growing ones that won’t hurt them. This isn’t always true, and sometimes people will be completely rational and still make the wrong choice. We can get this number arbitrarily low by setting stricter and stricter thresholds for which cancers we treat, but the stricter we go, the lower we drive number of true positives caught, until eventually our 32 QALYs gained number becomes a large overestimate. This decreases the benefits some amount, but I don’t have a good sense of how big an effect it is.

Sixth, it assumes that the population you care about is comparable to the study population which I got these numbers from. The median age in these studies is early 50s. If you’re younger than that, you’re less likely to have real disease, so your benefits are lower; if you’re older than that, the opposite is true. But also, these studies focus on people who chose to get full-body MRIs. These people might be sicker than you (maybe they chose to get them because something was bothering them, even if it wasn’t serious enough that their doctor ordered normal screening). Or they might be less sick than you (because they’re rich, or because they care enough about their health to get unusual tests, which probably means they care about their health in other ways like eating healthy and exercising). The second effect probably predominates over the first, so for a fixed age population-wide screening might work better than these studies suggest.

Seventh, it assumes current levels of technology and medical wisdom. If we implemented a national population screening program, it might encourage people to get better screening technology, and it would produce data that doctors could use to more wisely decide which findings to pursue vs. ignore. On the other hand, if we implemented a national screening program, then the quality of the marginal doctor working on whole-body screening might decrease (since we would need to devote far more doctors to this field).

Eighth, it assumes that I accurately calculated the cost of false positives in money, time, and side effects from additional testing. I got these numbers from AIs that estimated them from studies, but aside from the known dangers of both AIs and studies, they collapse a large amount of variation into one headline statistic, and they feel low to me.

Ninth, this is assuming you only get screened once. But most people who go for whole-body screening go every year. The cost-benefit calculation for multiple screens is worse than for one screen, because the first screen detects all the problems you’ve accumulated over your whole life, and the second only detects the new problems you’ve accumulated over the past year (but the second still costs just as much time and money as the first). On the other hand, this also decreases side effect burden from the second (fewer new anomalies to provide false positives) and takes away some side effect burden from the first (instead of following up with a specialist, you might be able to follow up by getting your regularly-scheduled next yearly scan.)

Tenth, there are probably dozens of unknown unknowns of approximately this magnitude.

Depending on which of these factors end up mattering in real life, they could push the cost-benefit calculation from where it is now (around the edge of usefulness) to anywhere from fantastically useful to horribly counterproductive. In the past when doctors have come up with clever cost-benefit analyses like the one in this post, these sorts of known unknowns and unknown unknowns have predominated, and the real effects have been much different from the expected effects - usually much worse. So doctors try to err on the side of being conservative and not doing anything until preliminary studies show that it helps.

But if the calculations above are right, and only ~8 people out of 1,000 benefit, then it would be hard for high-quality studies to measure the effect. You’d probably need hundreds of thousands of participants to find a signal. Instead, we only have meta-analyses of low-quality studies measuring secondary endpoints (eg cancers detected) like this one. But secondary endpoints are notorious for failing to generalize to the things we really care about (like lives saved).

So since the cost-benefit analysis is merely on the edge of being worthwhile, and there are no good studies showing that the assumptions in the cost-benefit analysis are remotely true, and in the absence of good studies doctors err on the side of caution, they currently recommend against whole-body MRI screening. But for the specific scenario of a rich person who doesn’t care about money, who is willing to accept rational over intuitive models of the value of time, and who feels confident that they can avoid excessive anxiety over false positives, there’s a case - not a fully evidence-backed one, but still a case - that they might be mildly net positive.

What about Midjourney’s proposed full-body ultrasound scanner?

Hard to say, because it’s untested new technology and they don’t report many firm numbers about how good it is. But trying to speculate from what we know:

First, the potential benefit is lower. Unless the Midjourney team managed to solve other currently-unsolved problems with ultrasound along the way, it would only be able to detect about half the number of cancers that an MRI could. You could even make a case that this flips its existence to net negative - it would depend on the open consumer-behavior-question of whether whole-body ultrasound would substitute for whole-body MRI (i.e. people who would previously have gotten the MRI would get the ultrasound instead) or add to it (i.e. people who previously would have gotten neither would get the ultrasound). If the former, total number of cancers detected would go down!

Second, the scans might take less time. I don’t think this shifts the numerical cost-benefit analysis much, because the majority of the time for both procedures is driving to the testing center, waiting in the waiting room, etc. If people really hate being in the scary MRI machine for an hour, it could increase the number of people who got the test, but that wouldn’t change the per person cost-benefit ratio.

Third, it might cost less. Midjourney claims that it will, but this hinges on optimistic assumptions (they are so successful that they can manufacture them at massive scale). Since our previous calculation found that financial cost was the biggest negative of whole-body MRIs, this could potentially be decisive and flip the sign to positive even by standard cost-effectiveness numbers.

Fourth, Midjourney is leaning on the hope that people will get this scan yearly or even multiple times per year; this is where some of the benefits of scale come from. But this decreases the efficacy per scan for the reasons mentioned above. And if you don’t do this, you lose some of the scale and the price probably goes up.

Fifth, it might have a different false positive rate. I don’t know if this would be higher or lower. Some people say it would definitely be lower, because surely the combination of massive data + AI could solve everything. I don’t agree. First, these screens will start out with less data than MRI, because they’re a new technology and MRIs have already been done tens of millions of times. Second, ultrasound starts out weaker than MRI and they have to overcome this limitation. Third, even if a billion results would give you enough training data to make this better than MRI, you have to convince a billion people to do a thing which is currently worse than the alternatives to get there. Fourth, at some point you brush up against fundamental limits of the technology. Ultrasound can determine the size, shape, density, and location of things in the body. But sometimes harmless things have the same size, shape, density, and location as dangerous things, and even after you’ve wrung as much information as possible out of those parameters, you’re still not sure. Mr. X is 5’8, overweight, and lives in Chicago. Is he a terrorist? Would it help if I told you his height to five decimal places?

(“But if you get multiple scans at different times, you can see how fast it’s growing!” Okay, Mr. X gained three pounds since last year, ready to answer now?)

So I don’t think a mature version of this technology shows signs of obviously being better or worse than existing whole-body screening MRIs. If you’re excited, it should be for economic, regulatory, or cultural reasons. Maybe they can scale up faster! Maybe they can deploy Silicon Valley levels of capital and ambition against other barely-related unsolved imaging problems! Maybe they can short-circuit the existing medical authorities in ways that existing MRI companies have somehow failed to do, and successfully deliver the message that rich people who don’t care about money or time and who are immune to anxiety and won’t make irrational decisions might benefit!

Any of these would be a more grounded hope than “of course more data will always be good and worth it”.

I’m a rich person who doesn’t care about time or money, and is immune to anxiety, and won’t make irrational decisions. Should I get a whole-body screening MRI?

Seems plausible, but I worry that the set of people who think they fall into this category is bigger than the set of people who really do.

Allow me to go into arrogant condescending doctor mode for a second: I see a lot of people saying they’re sure that they would never be so foolish as to get a procedure that most experts think is unnecessary - or if they did, they’d only do it after extensively reading the research, or at least after discussing the evidence with their favorite AI. They say this in the process of explaining why they’re definitely going to use the Midjourney scanner, a procedure that most experts think is unnecessary. Most of them didn’t read any radiology papers or ask any AIs before deciding to join Team Whole-Body Ultrasound.

Your problem is that your threat model is an uneducated ninety-year old woman collapsing on her fainting couch and hysterically saying “Dearie me! A disease? I must get all possible medical treatment forthwith!” This is maybe a small part of the threat model - although I know some of these women and they’re often cannier than you expect.

But another part of the threat model is some very smart rich person whose boutique concierge doctor (paid per procedure!) will convince him that he’s scoring a point against the evil medical cartel by getting this exciting new contrarian high-tech surgery. Or that your favorite integrative longevity wellness influencer says on his podcast that he got the procedure and his biomarkers went up, and if it makes a biomarker go up then it’s got to be good! Or that your executive personalized medicine screening panel tells you that as a twice-exceptional neurodivergent person with the MTHFR mutation, you can’t rely on generic recommendations made for the average patient, but they will help you sculpt your own bespoke treatment plan including all the latest breakthroughs that ordinary doctors are too low-IQ and uncreative to know about. One of them is by a buzzy AI startup! It uses rays! If a procedure uses rays, and you still don’t get it, doesn’t that kind of make you an anti-tech decel? Seems problematic!

If you think you’re the exception, prenuvo.com, I bet they’d love to have you.