How We Can Err if We Rely on Randomized Controlled Trials

This transcript has been edited for clarity.

Eric J. Topol, MD: Hello. This is Eric Topol with my co-host Abraham Verghese for Medscape Medicine and the Machine. We have Professor Trisha Greenhalgh from the United Kingdom visiting with us. I've been following Trisha and her work for many years, but this is the first time I have met her, at least through Zoom. So it's particularly exciting. Welcome, Trisha.

Trisha Greenhalgh, OBE, MD: It's great to be here. Thanks for inviting me to be on the show.

Topol: For those who don't know Trisha, she's one of the leading primary care academicians in the world. Her education was at the University of Cambridge and the University of Oxford. She has had posts at University College London and Queen Mary University of London. Now she is running a program on healthcare delivery and where it intersects with social science at the University of Oxford. She oversees advanced degrees in translational health.

It's stunning how many contributions you made during the pandemic — some of the most extraordinary papers, whether on airborne transmission or long COVID, and more. We're happy to have you with us today to get your views about, not just the pandemic, but also about primary care. How to Read a Paper: The Basis of Evidence-Based Medicine and Healthcare is one of your most famous works.

We're 3 years into this mess, and we haven't done all that well in managing this virus. What are your initial thoughts?

Greenhalgh: That is the $64,000 question, as they say, isn't it? I was pretty impressed with the Lancet COVID-19 Commission report, which came out in September 2022, where they identified a number of failures. It made quite depressing reading. They described the pandemic responses as a global failure.

A number of things are worth picking up on, but one of my favorite topics is the failure to recognize that the virus was airborne. And I would say it's more than that. It wasn't just a failure to recognize that it was airborne. It was prematurely assuming that it was not airborne.

People didn't keep an open mind and say it might be airborne. In general, they assumed it was not airborne. They assumed it was droplet-transmitted, and we started off down this path of ritualistic handwashing. Now, handwashing is not a bad idea. I'm a parent. I taught my kids to wash their hands. Of course, it's a good idea to wash your hands. That's not what we're talking about. We're talking about closing our minds to the possibility that this virus could travel in the air, and that got us into a lot of hot water.

Abraham Verghese, MD: You had a lovely paper that described why it was that science couldn't get its voice heard properly and the political and social forces that subsumed all this. I'm in awe of the way you've been able to dissect these various components. It was the first time I'd read a paper that helped make sense of all these dynamics. I hope this will serve us well in the next epidemic. Can you talk about that and why voices like yours and ours were not heard early on?

Greenhalgh: We did publish a paper in Wellcome Open Research with Mustafa Ozbilgin and Damien Contandriopoulos, an international group. Mustafa is a critical social scientist, and Damien is also a social scientist. We wanted to apply theories of scientific thinking.

For example, some sociologists study scientists. They study how scientists get together, how they work. They do anthropology. They come and watch scientists in their labs. People study systematic reviewers as if they're a kind of strange tribe.

Our work was specifically located in the social science of science. I can give you the background to that because, funnily enough, I was attending a World Health Organization (WHO) committee meeting by Zoom, which is what everyone was doing in the pandemic. This was in about September or October of 2020.

I wasn't a member of that committee. I was there representing a charity that had asked if I could step in for them. So there I was, observing the machinations of this committee. And I was absolutely struck at how they refused to contemplate the possibility of airborne spread. They refused to look at the evidence. I remember putting links into the chat and getting replies from members of this committee saying, "Oh, we don't look at that evidence. It's low quality." I thought, Oh gosh, well, I'm only here as a guest, so I can't report on that.

I can't write a paper about that, but I can certainly write a paper taking things that were in the public domain and unpacking what it was about this particular WHO committee that made them so sure that the droplet theory of transmission was correct, that the evidence base in favor of that was strong, and that the whole of aerosol science was both poor quality and irrelevant.

So Damien, Mustafa, and I applied Bourdieu's theory of scientific capital, which is a wonderful theory. Bourdieu was a person who talked about capital in general. When you're bringing up your kids, you teach them things. And you don't just teach them mathematics and spelling and such. You also teach them how to behave. Perhaps, you take them to concerts and teach them how to be nice kids, that type of thing.

Bourdieu's theory of capital was that certain members of the educated classes will teach their kids all sorts of things that will build their capital. He also had this idea that scientists build capital in particular disciplinary traditions. And guess what? Infection prevention and control scientists build their capital partly by doing randomized controlled trials (RCTs) of handwashing, for example. That's a very specific example. But scientific capital within infection prevention and control does not include the kinds of things that aerosol scientists do, which is, for example, measure how gases travel in spaces. That's just not part of their scientific capital.

The other aspect of scientific capital is accolades and prizes and medals and publications in journals that are held to be prestigious. You started off this podcast by reading out a few bits from my bio. And the idea is that you present me as someone who's got scientific capital in whatever topic we're talking about.

Now, one of the things about that infection prevention and control committee at WHO was that the scientific capital (particularly of the leading members of that committee) had already been built. They had prizes for this, that, and the other. They published the definitive studies of handwashing and that kind of thing.

In our paper, we unpacked all that both for the WHO and for public health people. We used the United Kingdom and Canada and I think Japan. We compared them. One of the interesting things about the Japanese response was that they kept an open mind. The prime minister himself and other senior members of government published — on a website on the 9th of March 2020 — that "we don't know how the virus is transmitted, but we have a bit of a hunch that it might be airborne. So, while we're continuing to investigate that, perhaps you should not meet with too many people in indoor spaces, and perhaps you'd like to wear masks." They shared that scientific uncertainty.

The other thing they did was to get advice from senior aerosol scientists who had quite prestigious appointments in Japan. It was quite a cool thing to be an aerosol scientist in Japan. It wasn't cool to be an aerosol scientist in the United Kingdom or Canada or the United States.

And indeed, WHO now famously ignored a letter from 238 aerosol scientists who wrote to them in July 2020 saying, "Look, we think this is important. We believe lots of people are dying unnecessarily. We'd like to help." That letter was not quite ignored because the committee did change its website. But they changed the website in the opposite direction. They changed the website to say, "We still think there's no evidence of airborne spread," which, in retrospect, was very, very sad.

The point is that I was applying my training as a social scientist (I took some time out to study social sciences in the middle of studying medicine). So we were applying that and recently applied that same set of ideas in a paper that is less social science for The British Medical Journal (BMJ). This was published just a couple of months ago as part of The BMJ's series of papers to inform the thinking of the official inquiry; whether or not the official inquiry reads those papers is up to them. The BMJ hopes they will.

Topol: Japan had best pandemic performance in the world as far as industrialized nations go. I'm sure that had something to do with the openness and proper communication. Also, as you well know, they were distributing masks to their residents very early on.

The paper you published in The Lancet on the ten scientific reasons supporting airborne transmission is one of the most-cited papers of the whole pandemic, which says a lot because you know how many papers were published. How can physicians deny the use of masks with a respiratory virus? How can scientists and physicians say that masks don't help when you have a virus that is respiratory? I still don't get this.

Greenhalgh: It's counter to common sense, isn't it? I qualified as a doctor in 1983. And by the early 1990s — my last child was born in '92 and I'd just gone back to work — there was this new thing that hadn't been around when I went on maternity leave, called evidence-based medicine (EBM). At the time, I was working at University College London and like almost everybody then, I thought, Oh, look, this is the new thing. We jumped on it.

Within a couple of years, I was running training courses in EBM in London. At the time, Dave Sackett from the EBM community was running these courses in Oxford and we were liaising. I was very much part of that exciting early EBM movement.

I'm not anti-EBM. I'm still absolutely passionately sold on the idea that RCTs are a great idea for some questions. I'm a cancer survivor. I owe my life to an RCT. The treatment that cured me 7 years ago of a rather nasty cancer was tested in an RCT. So, I'm not opposed.

But over the last 30 years or so, I have begun to realize that while RCTs are a brilliant idea for testing drugs, they are a catastrophically bad idea for testing certain other things, masks being one of them. There are a lot of philosophical reasons, but there are also commonsense reasons. The RCT is an artificial thing. You randomize people to A or B, and you have to keep everything controlled.

The thing about masking in a community in the real world is you cannot control everything. In fact, you can control very little. If I ask you to give your consent to wear a mask and then get tested for COVID, that's great because you have given informed consent. But what about everybody else in the community?

You can't recruit everybody. It's completely impossible. The only possible thing you can answer in an RCT of masking is, does person A wearing a mask prevent person A from developing COVID? You can't find out whether person A wearing a mask prevents person B from getting COVID because you never consented person B to be part of your trial. It's really that simple.

The other reason these trials of masking are a purely thought experiment is that, guess what, we're in a dynamic system. I'm not mathematical, but if we say that the doubling time is 9 days and we reduce COVID transmission by 10% every 9 days, it all adds up exponentially.

After a few weeks, we've prevented 40,000 cases. But how can we capture that in the RCT design? We simply can't. What we can do is model it or we can observe what happens in the real world.

We can observe that in these schools, there were mask mandates, and they were enforced. Look at how many kids got COVID. You can do those things. It's not bad science to look at real-world case studies.

It's not bad science to say, on the 8th of March in Amsterdam, there was a famous performance of the Saint John Passion and 130 people got COVID, and four of them died. The conductor nearly died. Oh, they were singing. Maybe that was a factor. Maybe someone who was singing was also transmitting a virus. It's not bad science to speculate that may have happened just because it wasn't a controlled experiment. It was a real-world experiment, and we should be looking at those.

The question of how doctors ignored all the evidence around airborne spread can be traced back to EBM. Without throwing the baby out with the bathwater, it was hardwired into us to say, "We need an RCT for everything." We didn't get an RCT for masks. We didn't get an RCT for airborne spread. So people say, "Oh, there's no evidence."

It's not quite that simple; I'm caricaturing it slightly. But broadly speaking, that's what happened. I'm from the same department as the Oxford Center for Evidence-Based Medicine and I can confirm that there wasn't a lot of looking at the full range of evidence in favor of aerosol transmission. There just wasn't. I was working with that group initially and they didn't want to look at it. But that's what we did. Very early on, one of my students came to me and said, "We've got this question that's come in from public." And I said, "That looks to me like a question about how fluids travel in spaces. I can't answer that. You're going to have to find somebody." So I Googled it all and found Lydia Bourouiba, who was a world expert on sneezes and how sneezes travel.

She would film people and their sneezes. I thought, Gosh, she's published in The New England Journal of Medicine about sneezing and coughing. I dropped her an email and asked if she'd like to work with us. I suppose that was my initial introduction. We met the rest of the aerosol scientists, who are just great people, on social media.

Verghese: I trained during the same era as you and watched EBM evolve with an arrogance around that viewpoint. It also dismissed anything that wasn't quantitative. It dismissed qualitative studies of the kind that I do. I was excited to see one of your papers talking about that imbalance of how qualitative science and narrative is dismissed.

Greenhalgh: Yet, we always think of qualitative sciences as interviewing a few patients or something like that. I believe that can be done well or badly just as mathematical modeling can be done well or badly, and RCTs can be done well or badly. But aspects of basic science are also effectively qualitative. My doctorate was in insulin kinetics. I had to look at what happened to a protein.

It was all very qualitative, but there were quantitative aspects. There was also a lot of thinking, a lot of talking about mechanisms and theory. I'll take this opportunity to plug a paper of mine — obviously it's written by not just me but with co-authors as well — it's this paper published recently in BMJ Evidence-Based Medicine called "Adapt or Die."

We threw down the gauntlet to EBM: You have to adapt or die. I thought the editor would probably change that title, but he didn't.

We're saying, "Yeah, look, an RCT is fantastic but we need to know the mechanism by which something works." We went back to Sir Austin Bradford Hill, who led the UK's first ever RCT of streptomycin for tuberculosis and formulated the Bradford Hill criteria for causation.

On the one hand, Bradford Hill was this blue-eyed boy of the EBM movement because he'd done the first RCT, but on the other hand, he'd actually said, "We need to know the mechanism by which it works." He said, "You can have this probabilistic evidence from your randomized trials but you also need to know how the phenomenon happens."

We were drawing on work by Mike Kelly and Jeff Aronson. They are people from the EBM movement. I know both of them. Jeff taught me when I was a student, so he isn't a young man now because I was at medical school 40 years ago.

But I think they coined the expression, EBM-plus. It may not have been them, but they certainly used it before I did. EBM-plus means all your EBM stuff plus mechanistic evidence, evidence of causality. And one of the things about the evidence of causality is that you have to stack it up.

There's this kind of evidence and that kind of evidence. For example, look at the evidence of transmission of SARS-Cov-2. There were those concerts, funerals, the super-spreader events, the kinds of events where lots of people tended to get infected; places where people were indoors singing or chanting.

But that alone wasn't enough evidence. There was also evidence of isolating the virus. The first time they isolated the virus, I believe, was from the air inside someone's car because it's a tiny space and presumably, people have been breathing away inside. And there was loads and loads of other kinds of evidence, where they connected ferret cages and the air from one cage went into another cage and then all the ferrets in the second cage got COVID, that kind of thing. So you're stacking up lots of different kinds of evidence. Taken together, that evidence builds the case that COVID is transmitted through the air.

The other thing is that there was no disconfirming evidence. Nothing disproved it. People had set up all sorts of studies to disprove this hypothesis and none of those had produced disconfirming evidence. I mean, there were certainly failed studies where people looked for virus in the air and didn't find it. But aerosol scientists would come along and say, "You haven't done that right. That's just not the way you would find virus in the air even if there was virus there." So that kind of critique, the peer review, was very important.

Certainly, the "Adapt or Die" paper presented the EBM community with this kind of ultimatum: You're going to have to accept mechanistic evidence because if you don't, you aren't going to be of any help in the next pandemic, waiting for your RCTs. That had a mixed response, I have to say. Even the editor was nervous.

The editor tweeted about our paper. He got slammed by all these EBM diehards on Twitter. So he deleted his tweet. He probably regretted publishing the paper.

We spent a long time responding to reviewer two, who I believe didn't understand the paper, and then reviewer two was invited to publish a rebuttal in BMJ Evidence-Based Medicine, if you want to read the counter-argument to what we presented. But personally, I don't think he actually takes apart our argument. He just restates that EBM is what it is. And in a way, if that's the best EBM can do to argue against the need for mechanistic evidence, then we rest our case.

Topol: I want to switch to another big topic, which is related. If you get a COVID infection, you can get long COVID. I bring that up because you're a standout proponent of the humanistic qualities of medicine. As you mentioned, and as we know, your program is integrating social science with the traditional interdisciplinary aspects of biomedicine.

Now, just like the dissing of masks, some people out there still don't believe there's such a thing as long COVID. It's incredible. I want to get your views about this because I don't understand it. By now, there have been so many reports, so many millions of people around the world are suffering significant disability. Yet, people continue to say, "They're making it up. They're just depressed or whatever." Can you address that?

Greenhalgh: There are two ways into this, aren't there? First, there is no single biomarker that I as a doctor can measure and say whether you do or do not have long COVID. Long COVID isn't as unique of a condition in that.

But if you go into hospital with chest pain and say, "I have chest pain," they'll check your high-sensitivity troponin level. And they'll tell you whether or not you're having a heart attack. We are quite used to being able to do a test that will discriminate reasonably well people who have the condition from people who don't have the condition. We can't do that with long COVID. The diagnosis is made on the basis of the history, the pattern, and how things unfold. For example, it often fluctuates. Those are fairly soft indicators. I believe that is why some doctors in particular are quite skeptical.

Second, doctors probably don't read all the obscure literature, where all kinds of metabolic abnormalities have been quite well documented. Those metabolic abnormalities don't tend to get measured in patients who are attending long COVID clinics or even more in primary care practices. People don't tend to get many tests.

But another thing needs to be factored in and that is the political risk, the ideological risk we take if we accept that long COVID is a real disease, if we accept that it is common, if we accept that in some cases it is persistent. It could be lifelong. We do not know. It's only been around for 3 years. We don't know whether people who are not better yet will ever get better.

If we accept that that is the case, it basically means that society is going to get sicker and sicker and sicker and more and more people of working age are going to be unable to work, or we're going to have to implement major changes in our way of life to prevent COVID. That means expensive investments by governments and businesses and public buildings in ventilation and filtration systems. It also means that if you get on a bus or a train or a plane, you're going to have to wear a mask. Someone might make you wear a mask.

Now, we've been around and around this argument as to whether that is a reasonable and acceptable thing. In some settings — the libertarians, for one — it's not accepted, particularly in schools. Why should my little kid have to wear a mask? This is interfering with their learning and with being a normal child and all the rest of it.

But if that kid catches COVID, brings it home and gives it to the dad or the granddad, and they develop long COVID, we have to contemplate that this is going to have a major impact on society, which isn't going to go away. It's not like the 1919 pandemic flu that was with us for a season and then died out. COVID may not die out.

I believe people are not ready to get their heads around that. So in the absence of a biomarker and given that most of the evidence is buried in journals that most people don't read, I can see why this is happening. I really can. People change their minds when a family member develops long COVID. That is when they realize that maybe it's not all in the mind.

It's an interesting and tragic phenomenon, but if you go back to 2003 and follow up with people who had SARS-1, some had long SARS. They had the same kind of thing. It didn't manifest in an identical clinical way but it was very comparable.

Verghese: It's also comparable to chronic fatigue syndrome (CFS) or myalgic encephalopathy (ME) in the sense that it's hard to define. You have a lot of unbelievers and people who are skeptical, and then you have patients who become rabid fanatics and are angry if you dismiss any of their symptoms. The whole field of long COVID has become quite charged.

Greenhalgh: It is very charged. I'm not an expert on CFS/ME. There are certainly parallels but there are also some differences. People with a specialist interest in CFS/ME would know more than I do.

But certainly, one of the things that is similar is what the patients call gaslighting — the idea that doctors say to them, "There's nothing going on here. Sort yourself out, man up," or whatever. That's deeply insulting for patients. We're doing a study called "Locomotion." We have patients on our advisory group with long COVID and we have long conversations with them about the kind of mind-body link because we're not saying that there's no mental component. I once had fatigue after a viral illness. It only lasted a month but that was long enough. I remember feeling pretty miserable and depressed about it. I was not truly depressed. But I was miserable. Was that directly due to the virus? Or was it a mental reaction to the fact that I couldn't do the normal things that I wanted to do? I still don't know the answer to that. Once my chronic fatigue got better, my mental health improved quite quickly. The idea that we would split it into bodily symptoms and mental symptoms — it doesn't work like that because we treat the whole person.

Topol: No question about that. Before we wrap up, I want to get your views on another matter that brings us together, which is communication. From our discussion, I can see that you're one of the most clear-eyed, persuasive physician researchers I've ever met.

You're also active on Twitter, which is in the midst of what appears to be a self-destruction act by Elon Musk. It has served as an important way to exchange information and whatnot to not only the medical community but to the public as well. What are we going to do now? What has been your experience with that platform and what's your plan?

Greenhalgh: Great question. Twitter is the town square. My work on COVID and on a lot of other scientific topics has been greatly informed and enriched by a dialogue that is directly with the lay public. I tweet about my scientific work. Other scientists follow me. Medical journals follow me. Politicians follow me. And citizens and patients follow me.

Everybody chips in. You have to be quite brave and quite humble to share your ideas with the lay public. It's just like when you've got a kid who's asking something like, "Why does the sun rise?" And you think, Oh, I thought I knew that, as you try to explain it.

So people who are asking the naive questions can often make you realize that you don't understand your science as well as you thought you did. And it helps; I've always found it useful. People say, why do you waste your time on Twitter? I think Twitter saves me time. Anyway, Twitter isn't dead yet.

I put out a Tweet this morning all about an argument I was having with a reviewer because the reviewer said, "You don't need any theory. Why are you going on about theory? What is your evidence that theory helps?" I put out a Tweet asking, "Can people help me?" And there's this wonderful response thread now with about 50 suggestions linking to articles that have demonstrated the usefulness of theory. That's a nice thread. I've already sent it to my students. So Twitter isn't dead yet.

For all sorts of reasons, it looks like the moderation of Twitter is being compromised. Far fewer people will be going in and checking whether things are correct, whether they're reasonable. I've certainly had some shocking, slightly pornographic images cross my timeline in the last few days, which I haven't seen for quite a long time. I wonder if that will get worse. I'm staying at the moment. Someone said, "Stick around and fight like a Ukrainian."

I have also joined Mastodon but I'm not very active there at the moment. The thing about Mastodon is that I can link with fellow scientists, but I can't get that immediate dialogue with the public. Also, I have about 2000 followers on Mastodon and about 175,000 followers on Twitter. So I get more response on Twitter. And, it's easier to use. There's been a lot of work on that platform that's made it easy to use, easy to tailor, easy to curate. You can get your threads, all that kind of thing.

I believe that what will probably happen is that it will go into a dip. It may be sold. Someone may get a bit bored with this new toy. And then it'll revive again. I actually think it's going to pick up, maybe because I'm an optimist.

Verghese: What a pleasure it's been to listen to you. One of the dangers of medicine is that we get more and more specialized. We put blinders on. Your career and your publications are a wonderful reminder that it's art and science. We've got to train our trainees to keep that in mind and be open to all the sciences other than medical sciences to inform our practice.

Topol: Trisha, you're just too reasonable.

Greenhalgh: It's common sense, actually. I believe it's worth picking that up because one of the things I'm getting into is the philosophy of pragmatism, which is about common sense. It's the philosophy of common sense and being confident to keep in touch with your commonsense self. I would admit to that without a doubt.

Topol: You're a model for that. We've learned a lot from you and about you through the pandemic. Your impact across medicine has been profound.

Thank you for joining us. I know the Medscape audience will thoroughly enjoy this. I hope we'll be able to reconvene with you in happier times, when the pandemic is largely behind us and we get into other topics like your book, How to Read a Paper, and all these other things you've been doing over the years.

Follow Medscape on Facebook, Twitter, Instagram, and YouTube

Medscape © 2022 WebMD, LLC

Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.

Cite this: Eric J. Topol, Abraham Verghese, Trish Greenhalgh. How We Can Err if We Rely on Randomized Controlled Trials - Medscape - Nov 23, 2022.