Do Lockdowns Work?

As the delta variant of the coronavirus continues to surge across the U.S. the question of should we lockdown again is on a lot of people’s minds. But, shouldn’t we stop and look at the data to see if lockdowns work?

In a new paper, our very own Anthony Fowler has done just that. And what the data say about the efficacy of state imposed shelter in place orders may surprise you.

Listen on Apple Podcasts or wherever you enjoy podcasts.

Transcript

Anthony Fowler: 

I am Anthony Fowler.

Will Howell: 

I'm Will Howell.

Wiola Dziuda: 

And I'm Wiola Dziuda, and this is Not Another Politics Podcast. So this school year just started. For my child, I'm a little bit nervous how this year is going to go. Are we going to have Delta surge? The kids are still unvaccinated. Are we going to have school closures? Are we going to have shelter in place orders as we did a year and a half ago?

Anthony Fowler: 

Can I just ask a clarifying question? When you said you were nervous, Wiola, are you nervous about COVID being severe? Are you nervous about the health implications of the pandemic, or are you nervous that we might overreact to the pandemic and harm the economy and burden ourselves in other ways, or maybe it's both? Maybe it's both at the same time. I'm just curious.

Wiola Dziuda: 

I think I'm nervous about my child getting sick, though I fully recognize that that's slightly irrational parental nervousness because the data seems to suggest that so far the kids are not extremely affected, though who knows what kind of other variants are going to come, Mu and Kappa? But I'm also nervous about the disruptions. I'm nervous about what if there is an outbreak at school. Are we going to have a school closure for two weeks? Or what if there's an outbreak at the university. Are we going to be told to go back to Zoom teaching?

Will Howell: 

Yeah. And there's uncertainty everywhere because we don't know exactly what works, do we, right? We think we know some things. Masks are a good idea, right?

Wiola Dziuda: 

Vaccination seems to be a good idea.

Will Howell: 

Vaccinations seem like a very good idea. But then there are these other things that we can do that effectively tell us to stay home or to limit our mobility. Well, what do we know about that? This is something, Anthony, you have written about, that you've explored. Tell us about this paper that you've written.

Anthony Fowler: 

Sure. Yeah. I wrote a paper. We published it recently with our colleague, Chris Berry, who we've had on the show before. So Chris and I wrote this paper with some of our recent MPP students, Tammy Glazer, Sam Handel-Meyer, and Alec MacMillen.

The paper actually came out of a class. Chris and I were supposed to teach in Barcelona in the spring of 2020. We have a study abroad program there. We were very excited to go to Barcelona for the spring, and like everything else, that was canceled. At the very last minute, we thought, "Okay, well, we have to teach some other class," and so we thought, "Well, let's do something relevant. Let's get a bunch of students together, and let's do something. We don't know exactly what it's going to look like, but let's do something that's related to the pandemic." We're obviously not biologists or epidemiologists, but we are public policy scholars, and so we are well-equipped to study policy interventions and government responses to the pandemic. So that was the topic, was let's collect a bunch of data, we'll analyze it together as a class, and then this paper came out of that class on the effects of various government responses to the pandemic.

Wiola Dziuda: 

That sounds really terrific, although being in Barcelona not during COVID probably would have been even better. But, hey, we wouldn't have your paper. So in this paper, you look at a particular policy, the one that we were exposed first to. I think this was the first line of defense for various governments, which is shelter in place.

Anthony Fowler: 

Yes. I think most of us, before the COVID pandemic, didn't think of this as a likely possibility, that our governments would be ordering us to stay at home, and so this was a pretty extreme kind of thing to do. So it makes sense to see whether or not it was effective or not. The very top-line results, we studied the effect of these shelter in place orders on COVID cases, COVID deaths. We also have some data on mobility, how much people are actually moving around as measured by their GPS data from their cell phones, and we also looked at unemployment.

It looks like there was essentially no detectable effect of shelter in place orders on COVID cases and deaths. There is a small effect on mobility that dissipates pretty quickly. There is some meaningful effect on unemployment as well, where it looks like these shelter in places orders were increasing unemployment. It looks like there's a little bit of a lag there, but once these orders have been in place for a couple weeks, it looks like unemployment really does suffer in these places. So at least the very top-line results don't make it seem like shelter in places were doing the first sort of thing they were supposed to do, which was stop the spread of the disease.

Wiola Dziuda: 

I personally am puzzled by this result, in the sense that my prior thought was, "Of course, they must work. Of course, people would respond to them." I responded to them. Even though I think shelter in place took effect, I don't know, what, March 24th, I probably haven't left my house after March 24th for four months. So, Anthony, can you tell us a little bit more about how you got the finding that you got, exactly how you looked at the data to conclude that indeed there was very little effect on deaths and cases?

Maybe a place to start is to think about the data themselves that you're using. So where do you get the data on health and COVID deaths and unemployment and mobility? What are you looking at?

Anthony Fowler: 

Sure. So for the COVID cases and deaths, lots of people were using this data. There was this Johns-Hopkins Coronavirus Resource Center that was compiling the data that was submitted to them from state and local governments, from hospitals, from everywhere they could get it, and there was this one resource that was pulling all that data together. That's where we got all the data on COVID cases and deaths. It's worth pausing for just a moment that this was all happening in real time and probably the data was not perfectly reliable. There probably were in fact mistakes. Actually, in the process of our students working on their projects and us working on this paper, we did find a bunch of little errors. Some of them might not have been consequential, but some of them might've been very consequential. We made our best efforts to correct the obvious errors.

That's one challenge of doing this kind of research in real time, is there's a rush to say something. I think that there are a lot of studies out there where they're just trying to publish the first paper on COVID and are using data that might be very preliminary, might have mistakes in it, and so forth. We tried very hard to fix some of those mistakes, but there are some cases where there are some kind of inexplicable data points, where the number of cumulative cases is just zero in a state on a day when it doesn't make sense because you know they had cases before and things like that. Anyway, so there are those kinds of mistakes that we did our best to fix and address in some way.

The other data sets, though, like I said, we have some data on... So we have data on unemployment that comes directly from the Department of Labor. We just have that at state level, not at the county level, but there are these monthly unemployment reports at the state level. The data from shelter in place policies was also somewhat difficult because these things are... Every county is doing something differently. There were various efforts by researchers to try to compile these things and figure out what was going on. There was a really nice repository by research [inaudible 00:06:28] in Washington, where they're pulling together all of these state and local policies, so we tried to utilize some data from there. There was some people at Johns-Hopkins that had found some county-level data, the National Association of County... So there are these different resources available. We're trying to compile them all so that we can have a good measure of in each county, what's going on, in every state, what's going on, what share of residents are covered by different orders, things like that.

Then, yeah, there's this mobility data. We didn't do anything innovative in the mobility space. We used data that other people had been using. There's a group called Unacast that have put together all of this data on mobility that measures in every county on every day how much are people moving around as measured by the GPS data that's coming from their cellphones and how does that compare with previous years before the pandemic. So you're kind of accounting for seasonal trends and you're seeing does it look like people are moving around more this year than they were last year, et cetera.

Will Howell: 

So can we linger a bit on the data involving COVID infections and COVID deaths? Because that's where you have these null results. This strikes me as just a really tricky thing in that you've already underscored the general administrative challenges associated with building a data set in real time or drawing from a data set that's being built in real time. Even if you abstract away from those very immediate local concerns, there are other kinds of worries that one might have when we think about the effect of a disease on the health of a population. So think, for instance, about somebody who turns on the news and sees that all these people are dying of COVID and then doesn't go into the hospital, get checked for something that's not COVID-related but that is potentially life-threatening, and decides to stay home and therefore doesn't get treatment and then dies. That's somebody who, one might argue, is obviously affected by the disease or by the pandemic but who wouldn't be counted in this particular instance.

Or people who actually contract the disease but think, "The last place I want to be is a hospital right now because hospitals are the places where people go to die and I don't want to die," and then they don't get any treatment either, and they, too... That's somebody who is actually immediately affected but isn't receiving treatment and who may not have ever been diagnosed but who wouldn't be counted in the data that were available to you and to us as a country. Do I have this right?

Anthony Fowler: 

Yeah. There's lots of things to talk about there. Suppose you just care about the direct... What does it mean for COVID to cause your death? You tell the story where it could be a very indirect thing, but there's a medically diagnosable sense in which COVID causes some deaths and doesn't cause other deaths. So even if you're just trying to measure the direct cases and the direct deaths that were caused directly by someone having the COVID disease, even that's not so straightforward because the number of cases that we detect is very dependent on how much testing we're doing. Especially early in the pandemic, when there's not a lot of testing going on, you would worry that... For example, a big inferential concern for our paper would be what if states start ramping up testing around the same time that they implement shelter in place orders, and so it'll look like the shelter in place orders are causing more cases when, in fact, they might be preventing them.

So that is a problem, and we do, in one analysis, try to account for known changes in testing regime. In cases where we know that a state started implementing more testing on this date, we actually account for that, and we'll do our analysis within testing regimes. For our paper, I think the data on cases is probably not nearly as reliable as the data on deaths because the data on cases is so sensitive to how much testing is going on. But the other thing to say is that we are, of course, not capturing all of the adverse effects of COVID by any stretch of the imagination.

Anthony Fowler: 

There are all kinds of bad things that are happening because of COVID that aren't measured in cases and deaths. Like you said, of course, there's the economic impacts, there's the social impacts, and there are even other health impacts, like, for example, someone who says... Probably, if you're having heart attack, you're still going to go to the hospital, even in the middle of a pandemic. But if you just have long-term diabetes and you need regular check-ups, you might push those back a little bit, and then that might have some long-term health consequence because of the pandemic. That's something that we're not measuring directly. There certainly are people who have tried to measure excess deaths. What were the total cumulative health impacts of the COVID pandemic? It looks like they were in fact pretty severe.

Will Howell: 

Yes, because we can tell lots and lots of stories about the indirect effects on mortality, right? There are the indirect effects in wellbeing, friendship patterns, divorce rates, all sorts of things. But just on mortality, we can tell stories about people not going to the hospital, not getting their meds, not getting the kind of exercise they used to get for fear of contracting a disease should they leave their home, that kind of thing. If you wanted to capture these, it seems to me the best shot we have at it is just looking at the total volume of excess deaths. That, too, is fraught with challenges, but you just see how much did dying increase in a particular location during a particular time of the year relative to the past. But you're honing in on documented COVID deaths using the trackers that we were all paying attention to when we were trying to get a handle on how big of a deal this pandemic was.

Anthony Fowler: 

Exactly. That's right. So the basic methodology is a difference in differences design. We've talked about those before on the show. So just think about the state-level analysis for a second. Our outcome is number of cases or number of deaths in a state on a particular day. We're going to run a regression. We're going to count for time trends, so we're going to have day-fixed effects that count for the fact that the number of cases and so on is changing over time nationally for all of the states in kind of predictable ways. We're also going to have state-fixed effects and account for the fact that New York just has more cases than Iowa and so on. Then we're going to have our treatment variable of interest, which is what share of the state is covered by a shelter in place order on that particular day.

Now, one of the concerns you have is, of course, states are implementing shelter in place orders precisely when they expect cases to be increasing. So I think if you just run that plain vanilla diff in diff design, you would worry that you would get a positive bias or you would underestimate the helpful effects of shelter in place orders. To further try to account for that possibility is we also just control for what the outcome was, the either cases or deaths per capita. We control for those levels in the days running up to that particular day. So we might control for what was happening in the last 14 days or what was happening in the last 21 days. Not only are we controlling for the fact that New York generally has more cases than Iowa. Maybe April 30th has more cases than April 15th. We're also controlling for the fact that it looks like in this state there seems to be an increasing number of cases and in this state there seems to be a decreasing number of cases, things like that. So we're trying as best as we can to account for these trends. We do a number of different things.

Another thing that we do in the appendix is we actually control directly for the epidemiological predictions that were made by scientists and were probably being viewed by governors and by policymakers in deciding whether or not to implement the shelter in place orders. So we're trying to account as best as possible for the fact that we know that states are implementing these policies when they expect cases to go up. Even when we do all of that, it still looks like we can't find any evidence that these shelter in place orders are actually reducing cases.

Wiola Dziuda: 

And how do you account for delay? So if you shelter in place today, there still is some amount of time during which we discover that we have been infected before we shelter in place and we die. So how do you deal with this problem?

Anthony Fowler: 

Yes. No, you're right. For some of these outcomes, the effect should not be immediate. I think for mobility the effect should be pretty close to immediate. If you implement a shelter in place order, people are supposed to comply. They're supposed to stay at home on that day that the order comes into place. You even give people a little bit of warning time and tell them, "This order's coming in place in two days." One way we deal with this is we test to see what are these effects over time, so instead of just asking what's the average effect of having a shelter in place order, we can actually run a slightly more complicated flexible specification that estimates what's the effect of having a shelter order in place today, what's the effect of having had it yesterday and today, what's the effect of having had it for three consecutive days, et cetera. So we can make a plot that's in the paper of, "Here's the estimated effect of shelter in place orders on cases after the order's been in place for X number of days."

We can go up to multiple weeks, and we can see does it look like maybe there's no effect right away but eventually there's an effect that starts to kick in. There are some interesting changes. So for cases and deaths, we essentially don't find much evidence. What you might've thought is maybe there's not an effect early on but after a couple weeks then you start to see the beneficial effects of shelter in place. We don't see that for cases and deaths. We just see nothing for cases and deaths. For mobility, it goes the other way around. For mobility, there is an effect right away, and it goes away after about 10 days. Even then, the mobility effect is relatively small. It looks like shelter in place orders caused people to reduce their mobility by something like two percent, and then by 10 days, they're back to doing whatever they would've been doing in the absence of shelter in place orders. Then, for unemployment, it looks like the adverse effects of shelter in place on unemployment, you don't really see them until the shelter in place order has been in place for a couple weeks.

Wiola Dziuda: 

So I want to clarify something. Of course, people stopped driving. I remember, a few weeks after shelter in place, I went for a drive, and the streets were empty. So what you're just telling us is that there's no effect that we can attribute to shelter in place, that the day before shelter in place, the mobility was more or less the same as the day after or a week before was more or less the same as the week after?

Anthony Fowler: 

Yes.

Wiola Dziuda: 

You're not comparing this to baseline mobility that we experience right now or we experienced three years ago? Is that correct?

Anthony Fowler: 

Right. So it's worth pointing out that we are not... Yes. We are not saying that the pandemic had little effect on any of these things. Of course, the pandemic as a whole had huge effects. There were huge national trends. We have a graph in the paper that just shows you, for example, with mobility, nationwide, it looks like mobility dropped nationwide for everybody around the same time and then eventually started to return back to normal after the pandemic had been going for a little while. That was true in the places that were adopting shelter in place orders early on. That was also true in states that never adopted shelter in place orders.

I'm sure, obviously, some of that is just mechanical, like they're working from home or they're laid off from their job and so they're not commuting to work anymore. So that's a nationwide thing. When you say that you complied with shelter in place orders, you actually don't really know, right? You know that you stayed at home when everyone told you you were supposed to, but you probably would've done that if you lived in Wyoming and your state had not implemented a shelter in place order. I know for myself-

Wiola Dziuda: 

I definitely would.

Anthony Fowler: 

For my wife and myself, I think we sheltered in place before there was a shelter in place order. So I think in that list of people who were always takers, who were sheltering in place regardless, we're certainly not saying that public health guidance had no effect or national reporting of the pandemic had no effect. It surely did. But it doesn't look like these very invasive, costly shelter in place orders that were issued by state and local governments had any additional effect over and above that.

Will Howell: 

I think it's worth underscoring that the basis for that conclusion is to do something much more careful and thoughtful than what journalists are constantly asking us to do when they say, "What's the effect of masking," or, "What's the effect of any particular action that a state might take," by comparing, say, COVID deaths in one moment in time in one state versus COVID deaths at the same time in another state. We're asked to do this all the time. Look at how, in the South, everybody's dying and, in the North, people aren't dying at quite such a big rate, and what do you know? The policies are very different in these two places or the kinds of recommendations that are handed down are very different.

What you're doing is you're saying, "Wait, wait, wait. That's not a fair comparison at all." We need to account for, in the models that you're estimating, these common shocks that are being played out over time. We need to account for trends within states. We need to think about conditioning present outcomes on the basis of past outcomes if what we're going to do is back out an estimate that meaningfully speaks to the efficacy of a policy instrument, in this case being a shelter in place order. So the naïve thing to do, you might generate a big effect, but when you do this kind of careful thing and you say, "All right, I don't care"... You're not arguing, as you say, the pandemic didn't matter or that all kinds of remediation efforts on the basis of either cities or states or the private decisions that individuals make don't matter. But this one particular intervention that was incredibly costly didn't appear to have a material effect on health outcomes, at least as measured by COVID infection rates and COVID deaths.

Anthony Fowler: 

Yes. Yes. I agree with all of that, yeah. I would add one thing to that. So I agree with what you said about, essentially, bias in design, which is you can't just compare the North and the South, you can't just compare one state versus another. There are obviously lots of differences. You also can't just compare one day to another. You've got to account for time trends. So we're trying to correct for those biases. You also can't cherry-pick your cases. That's another thing that we saw a lot of. "Hey, let's look at Sweden at... But we only looked at Sweden when the comparison was good for the claim we were trying to make, and then other times we ignored"... That kind of thing. If you get to pick your two cases, you can tell whatever story you want. So there's lots of these studies out there that I think are just not very reliable, and it might not be easy for just a news consumer to be able to wade through the details of a study and know whether or not it's reliable. But I think it's a problem because some results are clearly more desirable than others, and people were eager to publish results saying, "Oh, look, these policies are working."

Wiola Dziuda: 

So I would like to linger more on two statements that both of you just made. One statement is shelter in place orders do not work. I want to unpack it a little bit. The other statement that you made is that shelter in place were costly. So can we talk a little bit about this, in which sense shelter in place did not work? Definitely, what your study's suggestion is that just the fact that the governor issues a shelter a place on, let's say, March 24th does not affect the evolution of the deaths and cases in this particular state. But should we then conclude that we would have been in exactly the same situation had none of the governors issued shelter in place order? Can you tell me how you think about this? Because I don't think that's the conclusion we should draw.

Anthony Fowler:

Yeah. So I guess one alternative explanation you might have in mind is there's actually a big effect of these orders but maybe it's a nationwide effect, right, that the first few states that implemented these shelter in place orders, that signaled to the whole country that, "Oh, this is a serious problem." Suppose that California implements their shelter in place order. The people in Oregon, they also start sheltering in place, and now you actually have a bias in my study because now the people in Oregon... I was previously thinking of them as almost a control unit for California, but, actually, they're affected by the shelter in place order. Is that the kind of-

Wiola Dziuda: 

Yeah. So that would be one thing to think about-

Anthony Fowler: 

... story you have in mind?

Wiola Dziuda: 

... and you don't even have to think about California or Oregon or Washington, but you can think even about Europe or China. We hear that Wuhan's completely closed down, and then there's shelter in place in Italy, and then immediately we start adjusting our behavior. So perhaps one particular order you have to stay in place doesn't matter, but the public message that's somehow conveyed by different places imposing shelter in place could cumulatively affect our behavior and saves us from bad outcomes.

Anthony Fowler: 

Yes. I mean, I'm sure you're right that our public health messaging matters. I'm sure you're right about that, and I'm sure there is some sort of cumulative effect, that the more you see and the more consensus there is and so on, probably the bigger effect that has on people's behavior. Of course, we probably should at some point talk about the fact that this pandemic was extremely politicized. We've talked about that before. So that public health messaging was perhaps not as effective as you might've thought. I think that's the other part of the equation as to why shelter in place orders had such little effect on mobility, is because there were just as many people probably who didn't care that there were shelter in place orders and just ignored them.

But, anyway, so I think you're absolutely right that we're not saying that public health messaging does not matter. Of course, it matters, and, of course, to the extent that actually sheltering in place helped mitigate the spread of COVID... Surely, it does. Surely, the most obvious way to prevent the spread of a virus like COVID-19 is to try to avoid people who have it and prevent them from spreading it to other people. But if you're thinking about the question of did these particular state and county policies that were costly... And we'll come back to that in a second. If you're thinking about what the effects of those were, it still makes sense to evaluate them and see whether or not they were effective or not.

So the spillover question is something we tried to assess directly. We tried to ask, "Is there any evidence of these kinds of spillovers? Is there any evidence that maybe the early states had a nationwide effect, and therefore we actually... Suppose we just focus on the later moving states." Or we tested is it something about neighboring states. We tried different ways of analyzing that question. It actually does look like there is some modest spillover effect in neighboring states. So when Illinois implements their shelter in place order, it does look like it will affect Wisconsin and Indiana a little bit. But when we try to account for that, it doesn't change the bottom line result, which is that we still don't find any evidence that it affected cases or deaths.

Will Howell: 

There are roughly two categories of explanation for why you observe a null effect. One has to do with the efficacy of the change in behavior, people making decisions about whether or not to go outside and interact with one another, and the other one having to do with the efficacy of a change in policy. You're emphasizing the latter, that clearly it mattered that people didn't interact with one another as much because we know some basic facts about the spread of disease and reducing that but, at the margin, the declaration of these policies didn't. We don't have direct evidence on the change in behavior.

Imagine the following kind of story. What people... When they saw the spread of the disease and they heard about these outcomes and they said, "I'm going to stay home," they significantly reduced their mobility genuinely down to a few people, and that there are real benefits from doing that, right? But that then when the stay at home order was handed down, then they locked their doors and shut their windows, and that additional reduction in their mobility rates, that there were no health benefits from, that the initial adjustment that we're able to measure had real benefits but the add-on did not. We can't distinguish that possibility, no, from the possibility that, no, it's just about the change in policy. It's very hard to tease out the difference between these things. What is generating the effect, the change in behavior of people as opposed to just the change in policy? It's possible, at least, that there weren't real health benefit returns in terms of the health benefits from changes in actual individual behavior associated with this policy.

Anthony Fowler: 

Yes, I understand the spirit of the point, and I think you're right in some sense, that it could be the shelter in place orders changed behavior but they changed the kind of behavior that didn't matter for the pandemic. But, in fact, even with the GPS data, we find very little evidence that there was an effect on behavior. It was a small effect on behavior, at least as measured by this mobility data. It was a small effect on mobility that dissipated after a week and a half.

So I think the easier way to think about it is that people just did not comply with the shelter in place orders. There are two different kinds of ways in which it might not affect you. One way is you were already sheltering in place in the absence of the shelter in place order. You were already paying very close attention to the national news, and you were doing what the public health people told you to do regardless of what your local government told you to do, and there were surely other people. We know there were such other people because we know there were plenty of people moving around, we know that the disease continued to spread, who were still continuing to see each other, and they went about living their normal lives. Maybe instead of meeting their friends at the movie theater or the restaurant, they met their friends for house parties. That could be potentially maybe even worse than if they had met in public places, and they were spreading the disease.

Wiola Dziuda: 

I think your finding is very interesting because it shows that people respond to information that they have about the severity of the disease and the outcomes that can occur. They respond spontaneously to that. They do not need a politician to tell them, "Now, stay at home." Now, of course, there are people who do not respond to that, who do not take into account that they have externality on other people, that if they get sick themselves, it's no longer their health, it's also health of other people. As you said, your data seems to suggest those people kept on driving and meeting other people. They kept on being mobile despite the official shelter in place policy. That maybe is not so surprising given that these shelter in place policies were really more or less recommendations.

Wiola Dziuda: 

I don't remember now the details, but you tell me more. But my impression was that there was really not much that the state could do to you if you violated it. So, in a sense, if you are the person who is a never responder, who is not scared of COVID, who thinks COVID doesn't exist or is really not dangerous for you personally, they don't care about externalities, it's not so surprising you did not respond to shelter in place.

Anthony Fowler: 

Yes. No, I think that's absolutely right. There were rare cases of people getting fined and so forth for blatant violations of the shelter in place orders, but you're absolutely right. So I think a reasonable reaction to my paper is to say, "Probably whatever we did was not the right thing." So some people might say, "We should've gone further," right? The right answer was to actually be more authoritarian and actually have police officers checking in to make sure people were in fact home or something and finding them if they weren't. Then another reaction would be to say, "We just shouldn't have had them at all, and we should've let people do what they wanted," and maybe that would've not had as much of an adverse effect on the economy, and we would've had the same health outcomes.

Both reactions are defensible reactions, and it comes down to your own personal philosophy and how comfortable you are with government being more and more involved in people's lives. But I think the current middle ground that we chose, which was we're going to shut down the economy but we won't really enforce too strongly the actual public health guidance, that seems to not have worked very well.

Wiola Dziuda: 

I want to go back to this question of costly. How do we know it was costly? Within your paper, you do find an effect on mobility, but it's not a large effect, and as you said, it dissipates very quickly. So just, within your paper, can we really say it was costly? If people didn't comply with that, if they complied with that ex ante because they themselves decided to stay at home, to what extent just the shelter in place policy was costly?

Anthony Fowler: 

You could think about the cost in a few different ways. One is we do find evidence on adverse economic effects. So people lost jobs. It's worth saying that the unemployment effect is not nearly as large as the overall drop in unemployment over the course of the pandemic. So the pandemic itself caused a major increase in unemployment. It looks like shelter in place orders contributed to that but maybe only contributed something like maybe 10% of that effect, something like that. I mean, you could think of other senses in which shelter in place is costly, even if people aren't really complying with it very much.

It could be that people are still getting together for dinner with their friends, but they're getting together in their small, one-bedroom apartments instead of going out to dinner somewhere at a restaurant, let's say. That's a cost, too, in the sense that it's... You're not doing what you wanted to do. The government is basically restricting what you can do without actually providing any clear benefits. So I think it's fair to say that any time the government is putting in place some severe restriction on what you can do, I think that's reasonable to think of that as a cost. But there's also just the direct effects on the economy that we can pick up statistically, on unemployment, for example.

Will Howell: 

I think it would behoove us, as you do in the paper, to read into the record some cautions against bad interpretations of your findings. So one bad interpretation or one bad lesson that one might draw is that what elites say or what governments do when it comes to a pandemic are neither here nor there, that they have no effects, or even, more specifically, that shelter in place orders never affect health outcomes, that what you're doing is you're looking at particular moment, this spring of 2020, when all kinds of things were being learned about a spreading pandemic and all kinds of policies were being handed down... This wasn't the only thing that were being done. There were 75 other things being done, with a particular enforcement regime associated with it, and you're not counseling health officials to abandon this policy instrument in all contexts, right? What you're saying is we don't see a return from this particular intervention as it was done in the United States in the spring of 2020 in terms of health outcomes as we were accustomed to measuring them vis-a-vis COVID infections and COVID deaths.

Anthony Fowler: 

Yes. I agree with all of that. You're right. I mean, yes, of course, there are lots of ways to misinterpret the paper. Yes, we're not saying that sheltering in place, per se, in terms of behavior, has no effect. Surely, it does have an effect on mitigating the spread of a disease like COVID. We're not saying that shelter in place orders could never potentially have an effect. Surely, they could. But in this particular instance, they didn't, and we can think about why they didn't, although I do think it's really interesting to note, and I think it probably is an important social science finding, that this might just not be the kind of thing that the US government is good at doing. Maybe that's okay. Maybe we're okay with that.

Maybe I prefer, on the whole, to live in a country where the government can't just sooner force us to shut down and stop doing whatever we want. That might mean that we're less prepared to deal with a pandemic like this, but that might mean, in the future, we're also less worried that some terrible autocrat's going to come along and abuse their power in some way. So there's trade-offs, obviously. So, yes, there are lots of caveats. We're also not saying that nothing the government did was effective. Of course, public health messaging surely did have an effect on people's behavior. You see this huge nationwide drop in mobility that surely was a result of that, plus lots of businesses voluntarily having their employees work from home. Things like that surely had an effect, and we're not evaluating those particular mitigation strategies.

This is sometimes a disagreement that we have on this show. Sometimes, we like to have lots of caveats and we like to say, "No, of course, this study only applies to this one time in this one place," and so on. But if you think there's any point at all to doing empirical social science, and I do because that's what I do for a living and I think there are things we learn from it, then you must think that there's something we can generalize. There's something we can learn from this. We didn't know ex ante exactly what this was going to look like, and we've now learned that this is not the kind of thing that's effective. When governments just tell people to shelter in place, people don't necessarily comply with that. That seems like a good thing to know.

If a government was going to consider a similar policy in the future, I would probably advise them against it on the grounds that probably it's not effective. We're either going to have to come up with another way to get the outcome that we want, or we're going to have to accept that maybe we're not actually good at getting people to follow our orders. So you can go either direction with it depending on what your personal ideology is, but this is not the kind of thing that's likely to be effective in the future when we're talking about Americans who care about their personal liberty, many of whom don't trust the government to do the right thing, and so on. So this kind of strategy's probably not an effective one.

Wiola Dziuda: 

You could say a third thing, that perhaps they could have gotten the benefit of this public messaging without the backlash that they got for trying to impose their will on citizens. They could have messaged that, "We recommend you to shelter in place. We really think this is going to be good for your health and the health of your family and your loved ones. But we are not imposing an order."

Anthony Fowler: 

Yes. I think I agree with that. Now, this is, of course, outside the scope of our paper, but I also don't want to... Of course, we don't have any direct evidence on public health messaging, but I also don't want to just go around and say, "Oh, public health messaging was fantastic," because we know that wasn't very fantastic either, right? We know that-

Wiola Dziuda: 

That deserves an episode of its own, I think. Yeah.

Anthony Fowler: 

Yeah. We know that this past year and a half has been horrible for the reputation of science and the New York Times and lots of otherwise respectable institutions.

Wiola Dziuda: 

I would not say bad for the reputation of science but institutions that we thought were supposed to follow science. Yeah.

Anthony Fowler: 

Right. Right. And I think there were plenty of people who now think, "Okay, well, I don't really know what to think, but I don't trust what Anthony Fauci says." We're not saying the public health messaging was fantastic. It surely could have been better, and, in fact, probably one of the reasons why there was such low compliance to shelter in place orders was the fact that this pandemic became politicized so early on in such an unfortunate way.

Will Howell: 

But, wait, you don't have direct evidence on compliance. Your story is consistent with massive non-compliance, that is policymakers were whistling in the wind when they said, "Stay at home." It's also consistent with a story in which everybody was staying at home. Everybody, they didn't need to hear the instruction.

Anthony Fowler: 

Oh, but we know it's somewhere in between, right? We know it's a mix of both.

Will Howell: 

Do we?

Anthony Fowler: 

Because if everyone had stayed at home, we wouldn't still be in the middle of this pandemic, right?

Will Howell: 

Sure.

Wiola Dziuda: 

No, because a lot of people were not supposed to stay at home, even after the-

Anthony Fowler: 

Yes, we have data on this, and we know that they weren't staying at home. We know that the cases and deaths continued to proliferate and so on.

Wiola Dziuda: 

But we also know that the shelter in place had a lot of exceptions. There were a lot of people that were exempt from shelter in place, and in your data, you can't really distinguish who is moving, who is not moving. I agree with you that I think you are right. It's true that there were a lot of people who didn't comply. But we don't know that for a fact. It could have been that everyone who could have stayed at home stayed at home even before shelter in place, and that's why we didn't see there any drop at the shelter in place order in the people who kept moving around and the people who are bus drivers, grocery workers, medical workers, and so on.

Anthony Fowler: 

I mean, I guess. There are things that... Yes, the paper does not explicitly rule out the possibility, but we know that there were lots of people who were not complying with these orders in the sense that there are anecdotes, there is the data, there's the... You can just look at the sheer number of people moving around and rule out that you can put a bound on these things. I'm pretty comfortable saying there were lots of people who weren't complying with public health guidance and caused the pandemic to proliferate in the way that it did.

Will Howell: 

There are two things, right? One is the general proliferation of the pandemic, and then there is the efficacy of a particular policy intervention. Those aren't one and the same, and your paper speaks to the inefficacy of these shelter in place orders, and the extent to which we want to attribute that to non-compliance by people who were insisting on going out anyway as opposed to people who they were going to stay home regardless is just unclear. Earlier, I tried to put some bounds on how far do we want to take this paper. I, too, am inclined to say, "Well, there are other ways in which we might want to run with it," in that you're not showing null effects from a little thing that the state might do. All right?

Will Howell: 

This was at a time when lots of things were being happened, but this was an unbelievably draconian state intervention. This was not the state offering on a website some technical advice about don't touch your face and then showing, well, what do you know, that didn't seem to save lives. This was arguably one of the most draconian things that the government has done in a very long time, but your evidence suggests otherwise. We ought to take that into account. Death rates vis-a-vis COVID is really striking.

Anthony Fowler: 

I agree. I agree. I think a reasonable thing to have expected would be to say, "We should've"... It would've been completely reasonable to say, "What we should see is some effect on cases and deaths but then also maybe some adverse effects on the economy," and then we'd sit around-

Will Howell: 

And then we'd have to balance those.

Anthony Fowler: 

... and weigh those trade-offs and we'd say which... Right, and that's not what we find.

Will Howell: 

That's the situation we were in.

Anthony Fowler: 

Mm-hmm (affirmative). Mm-hmm (affirmative).

Will Howell: 

Yeah. Because, look, this is an unbelievably, we've already gestured towards this, an unbelievably politicized issue. On our show, we've talked about other things that have been unbelievably politicized, like voter ID laws, and then we try to estimate what's the effect of voter ID laws in particular, and they tend to have small effects that don't comport with the grand claims on either side of the political divide. You're doing a similar kind of thing here. You're saying, "There are big claims being made, and they just don't stand up to careful empirical scrutiny." What kind of a reception has this paper gotten?

Anthony Fowler: 

Yeah, it's an interesting question. The paper's published in PNAS. It's a very respectable science journal and so on. We've talked to a handful of reporters, and we've gotten some news coverage, but I would say there's been not much reaction at all, especially relative to other papers that have more politically desirable result. If you're a science reporter, you're much more eager to write the paper about how, "Look, you guys, you need to listen to your government because it's in your best interests," whereas the study that says, "Hey, this thing we did didn't work," that's not a very pleasing result to a lot of people who want to believe that these things worked. So we get a lot of either negative reactions or we just get a lot of people ignoring us altogether and hoping...

Will Howell: 

So the lament that I have, if I could get onto a soapbox for a moment, those who would like to leverage the government in the service of solving public problems, and count me among them, be it issues involving health or any number of other issues, I think genuinely do an inadequate job of pausing and saying that the demonstration of action or the deployment of resources is not enough, that one should pause and say, "Well, did it actually work? And if it didn't work, what do we need to do to adjust our behavior?" I mean, the lesson that I would like to draw from this paper that you've written, Anthony, is that the way... There was something about the way that it was done that led to it not yielding meaningful outcomes.

So it may be locked in that this simply can't be done in a way that would yield health returns. It could also be, no, we need to rethink the messaging, the lack of enforcement, the kinds of resources that were deployed, how it was coupled with other policy interventions that were happening at the same time. Because just because the state does something, does not and, in fact, we should... I think, in domain, there's lots of evidence and lots of policy domains that much of what the state does yields null effects. I mean, null effects is the norm, that what our government does, in many ways, is ineffective in all kinds of domains.

Anthony Fowler: 

Yeah. I think I agree with all of that. I think it's a perfectly reasonable reaction to say, "Okay, we shouldn't give up on state and local governments trying to do what they can to address major public health crises." But we need to rethink how we go about these things, and just issuing these orders is not enough. I think that's perfectly fair. But I also think I also kind of sympathize with the other side of the debate, which says, "Maybe we don't want our governments to be so authoritarian that they can control our movements and tell us when we can leave our houses and so forth." Even if that means that we're not as equipped to deal with a once in a 100 year pandemic, that means in the other 99 years out of the century, we have a little bit more liberty and we feel like we have more rights and so on. I see both of those arguments, and I see the other side of the argument, which is to say maybe state and local governments, this isn't the kind of thing they should be trying to do.

Will Howell: 

So, Wiola, does Anthony get to have a bottom line on his own paper, or is it just us?

Wiola Dziuda: 

Yes. Yes. Of course. That's going to be the most fun part of the episode. What's your bottom line, Anthony?

Anthony Fowler: 

I didn't think I would have to have a bottom line. I've written what I wanted to say in the paper, so maybe that's my bottom line. The thing that is maybe the most interesting to say is just that I am... The thing that's most upsetting about all of this discussion over all of the science around COVID is just how politicized it is, and it's really unfortunate that it is that way. We have to be willing to have reasoned discussions. Just think about the sheer number of things where you're not even supposed to discuss them on the pages of the New York Times or on YouTube or wherever. You'll essentially get shouted down by one side or the other just for even discussing it. We've got to stop this, and I don't know how to stop it. I would like, at some point, for us to get back to real science and real evidence. Let's just weigh things. Let's just talk about things.

I would love for a public health official to say, "You know what? It looks like there's this effect of this policy, and it looks like there's risks out there. I, a public health scholar... I don't know what the right thing to do is because there's all kinds of other things that we have to trade off, but here's what we know. Most importantly, here's what we're uncertain about." That would be fantastic, and we haven't heard much of that during the pandemic. I think that has led lots of people to just not trust in experts and scholars and scientists because they say, "Oh, those people, they're just ideologues, so I'm just going to do whatever I want." That's not a great thing either, is to just not trust... "Let's not listen to scientists at all and just do whatever we want because we know that they're so politicized." So, anyway, all of this is terrible. I'd like somehow for us to get back on track. I don't know how that happens.

Will Howell: 

Yeah. So I'm with you on getting back on track. When we think about the politicization of science, I don't think it's just about scientists themselves toeing the party line. It's that there are all kinds of organized efforts to marginalize scientists and, at the moment a scientist comes out and says there's uncertainty in this space, to use that as evidence that scientists don't know what the hell they're talking about and to further marginalize them. So it's a very tricky thing, how you actually break through. But a piece of it is writing papers like this, which is trying to reasonably... Look, what is the optimal messaging that we need to deploy in order to induce changes in behavior in the faces of a pandemic?

It isn't obvious that what that optimal message is is just at every step of the way reading the latest abstract of the latest studies but that there ought to be a space, nonetheless, wherein serious-minded folk think about what policy instruments work and which instruments don't. Your paper, I think, really constructively contributes to that. And, as I say, I was sure we were living in a world in which it was all about trade-offs. These interventions decidedly save lives. The question is whether or not the downside in terms of the economic costs were worth it, and we can think of other downside costs as well.

Anthony Fowler: 

Thanks for listening to Not Another Politics Podcast.

Wiola Dziuda: 

Our show is a podcast from the Harris School of Public Policy and is produced by Matt Hodapp. Thanks for listening.