In the United States, there has not been a COVID-19 response so much as a multitude of micro-responses. With no comprehensive national plan to unify them, local and state governments have deployed an assortment of measures – some aggressive, others less restrictive – in various combinations in their efforts to control the spread of the disease.
Not surprisingly, the results have been similarly mixed, with some communities battling record cases even as others flatten to baselines. But the heterogeneity of U.S. responses and outcomes does have one upside: It’s helping researchers learn which strategies worked, and which did little to protect communities from the virus’ relentless march.
“It’s almost like a randomized trial,” says William Pan, associate professor of global environmental health at Duke. “We realized we could take advantage of the phased implementation occurring to pull out some of the effect sizes of these different policies.”
Over the summer, Pan and a team of analysts sifted through data from all 3,142 U.S. counties and the District of Columbia, testing for relationships between the timing of control policies in place between January and May and the resulting number of COVID-19 cases and deaths during that time. Their findings, published online, provide some of the most complete evidence yet of what drove some communities’ success in beating back the virus.
A striking detail is that only the most aggressive control measures, such as stay-at-home orders and mask mandates, produced across-the-board reductions in both COVID-19 cases and deaths. Researchers categorized response policies into four tiers of increasing rigidity, with soft measures like emergency declarations at the bottom and stay-at-home orders at the top. Counties that adopted policies in the highest tier saw a 50% decline in new daily cases within an average of six days, and death rates declined by 15% for each day the measures were imposed.
The top tier was also the only one to produce declines in cases and deaths in every region of the country. That wasn’t the case for the second-highest tier, including such measures as closing businesses and limiting the size of social gatherings, which reduced cases and deaths in some parts of the country, but not others. The two lowest tiers failed to yield any significant decline in cases or deaths, and in many instances actually led to continued growth, albeit at slower rates.
Pan says the findings sound a note of caution as many states and local governments begin to ease COVID-19 restrictions.
“We found that those lower-level policies were not enough to keep the virus from propagating,” he says. “And with the number of cases we are still seeing, it basically means that depending on which county you live in, the policies that are in place now may not be holding coronavirus back at all.”
In fact, Pan bypassed the standard journal publication process, which can involve weeks of peer reviews and revisions, to ensure the data was accessible more quickly. “We were hesitant to release this paper prior to completion of the peer-review process, but with universities and schools opening up, we felt it was important to share these results so that people can use it in their modeling and decision-making.”
During the early phases of the pandemic, policymakers have had little hard evidence to help them gauge the merits of particular interventions, Pan says. He notes that the widely used models created by the Institute for Health Metrics and Evaluation (IHME), which forecast the effect of policies such as mask mandates, are based on data from a handful of counties.
“These models are useful, but they are based on an extremely small sample size,” Pan says. “We really felt there needed to be a better analysis from a larger, country-wide point of view on how effective these policies are.”
Pan’s study adds another ingredient the IHME models may miss: regional variance. By examining outcomes from every U.S. county, the Duke team identified several cases where policies that worked in one location don’t yield the same results elsewhere. Among counties that adopted the strictest measures, for example, the COVID-19 death rate declined by 25.5% in New England and Middle Atlantic states, but only 5.5% in North Central states. In the second-highest tier, the death rate declined among Pacific states, but increased everywhere else.
One factor may be differing levels of compliance with local orders, Pan notes. “There is a lot of heterogeneity in terms of how people follow the rules,” he says.
While the new statistical analysis cannot evaluate compliance, Pan says the results give policymakers an ability to gauge how different policies may affect COVID-19 risk in their communities, one that is based on the reality of how and where those policies play out when applied. “If you have better data going in to modeling,” he says, “you’re likely to have better decision-making as a result.”
