Action-Oriented Social Science: Urban Labs Conference Explores Bringing Solutions to Scale

Research led by the Education Lab, one of five Urban Labs  at the University of Chicago Harris School of Public Policy, found that intensive tutoring can double or even triple the amount of math a high school student can learn in a single year, pointing out a way to mitigate learning loss brought on by the pandemic.

That research relied on a model from an innovative nonprofit, the participation of Chicago Public Schools, and funding from corporate, philanthropic, and government sources. It also points to the power and impact of public–private partnerships — a topic explored in a recent conference hosted by the Urban Labs and Harris School of Public Policy.

Titled Creating Social Change Through Public-Private Partnerships, the conference was held at the Keller Center and broadcast to a remote audience on November 1 and November 2, gathering researchers, policymakers, government officials, and funders. Panelists, speakers, and moderators included Chicago Mayor Lori Lightfoot, Obama Foundation CEO Valerie Jarrett, and experts from university research labs across the country, all digging in on action-oriented social science.

“Problems that society faces have become ones which no one institution has anything approaching the capacity to be able to address on its own,” said University of Chicago President Paul Alivisatos, whose keynote address opened the conference.

Those kinds of problems include climate change, massive housing challenges, violence in U.S. cities, health disparities, and educational inequality. “We, as a university,” Alivisatos said, “have a special obligation and need to become a place that helps foster larger-scale partnerships.” 

Key to building those partnerships are the Urban Labs, which take academic research off the shelf and into action.

 Examples — like the intensive (also known as “high-dosage”) tutoring initiative and health equity work being done at a Stanford University lab — were featured at the final full session at the conference, which looked at how to scale for impact.

“You can think about the scaling challenge in a number of ways,” said Tom Dunn, a University of Chicago trustee and former CEO of New Holland Capital, who moderated the Scaling Panel. “There's the scaling of that first R&D phase where you're trying to demonstrate that an idea can be implemented at scale. But then once you've learned that, how do you then spread it to other contexts, other cities or other places?”

Panelists looked at scaling and its challenges from academic, governmental, nonprofit, and commercial business angles.

Professor Daniel Ho, director of the Regulation, Evaluation, and Governance Lab (RegLab) at Stanford University, discussed the promise and limits of using technology to scale public-private partnerships.  

One limit, he said, “is that the government often lacks the ability to draw on some of the most cutting-edge forms of data science and artificial intelligence.” 

Government information systems are frequently outdated, he said, referencing a 2016 report from the Government Accountability Office which noted that the Defense Department was still using 8-inch floppy disks.

The RegLab, he said, has been trying to bridge the gulf between universities and government agencies that would benefit from academics’ research. He shared examples including how technology helped California’s Santa Clara County, where the population is one-quarter Latino, solve a health equity issue.

As it scaled COVID-19 contact tracing, the county had no way to match the English and/or Spanish language skill sets of contact tracers to incoming case calls, he said. “We took really sparse information from incoming laboratory reports, merged that with census data and other administrative datasets to build out a language-matching system. That allowed us to predict on an incoming basis what the likelihood was that this was someone with Spanish language needs, and then match them to a language specialty team,” he added. 

Stanford's RegLab has since been working with California officials to determine how to embed the language-matching system statewide, Ho said. “But ultimately we'd love to see other people take the code that we're making available open source.”

One of Ho’s fellow panelists was Mary Ann Bates, a senior fellow at the Office of Management and Budget on leave from her role as executive director of the Abdul Latif Jameel Poverty Action Lab (J-PAL) North America at the Massachusetts Institute of Technology.

Bates’ presentation centered on a scaling challenge she labeled The Generalizability Puzzle, which is work she did with Rachel Glennerster, an associate professor at the University of Chicago Division of the Social Sciences.

At a similar conference several years ago, Bates said, “we had a superintendent who was sitting on a panel like this and he literally said ‘If a study was not conducted in my own school district, I can't act on it. The study has to be done in my context. Every place is unique. My school district is unique, so I can't act on evidence if it wasn't done there.’ 

“Yet, as we've been discussing,” she added, “we all know we won't achieve the impact that we need to if we can't learn from studies beyond their original context. So that's the heart of The Generalizability Puzzle — it’s that we all know that we need to figure out how to take evidence that was generated here in Chicago and figure out how that works in Los Angeles or Sacramento or D.C. or in the rural South.” 

“One example that I find particularly inspiring,” she added, “is how that evidence on high-dosage tutoring was really powerfully taken up and acted on during the pandemic.”

“We have never been in exactly this environment before. And so by definition, we didn't have evidence from the COVID-19 pandemic at the start. And yet … there was so much we could draw on from prior evidence and from prior studies.”

Researchers’ pulling together the big-picture view of why intensive tutoring works and determining what’s needed to scale up such programs led to action, including California “making a massive investment (up to $2.6 billion) in high-dosage tutoring,” she said.

That shows, she said that researchers should think from the beginning about how to “generalize across context.”

Panelist Josh Wright, executive director of ideas42, a nonprofit behavioral design firm (which Wright described as an urban lab outside of academia), turned the attention to the infrastructure he believes is necessary for scaling. 

He said ideas42, whose work ranges from water conservation efforts in Costa Rica to keeping people out of jail in New York, identified four paths to scale. That happened, he said, after the firm realized it was pleased with many of the innovations it was coming up with, but few were getting to scale — defined by ideas42 as reaching hundreds of thousands to millions of people.  

Those four paths are: “scale baked in,” which is engaging in innovation or research with a partner that already has scale (like a government); “venture,” which involves building a business that solves a problem for someone willing to pay for it; “practice change,” achieving adoption or adaptation of innovation by those that control the processes and practices (such as a city); and “policy,” or effecting a change to federal or state policy. 

Policy, he added, “is a place where there has to be a lot of partnership.” 

That partnership message also resonated at the Health and Environment Panel where the goal was to explore the “innovative ways that academia can partner with government to help shape priorities, policies, and process through data and evidence-based practice,” said moderator Dr. Mai Tuyet Pho, a University of Chicago associate professor of medicine.

Panelists included David Meltzer, the Pritzker Director of Harris’ Health Lab, another of the Urban Labs. He described the lab’s Supportive Release Center (SRC), which offers individuals with mental illness a safe place to stay the night and links to social services after their release from Cook County Jail. 

Building off of lessons learned from the SRC randomized control trial, the Health Lab has launched several new initiatives, said Harold Pollack, the Health Lab faculty co-director. That includes the Reducing Opioid Mortality in Illinois (ROMI) project, which he and Pho are working on.

“We're trying to help people who are leaving jails and prisons who have opioid use disorders and who are very vulnerable to having a bad outcome right when they leave,” Pollack said. When individuals with an opioid use history are freed from jail, they’ve effectively detoxed, he said. “What's gonna happen in the next three hours after they leave — that’s the kind of challenge that trials like the ROMI are trying to deal with.”

“It's a real challenge,” because how, he asked, “do you relate to people in a human way that engages them into an intervention at that critical moment when they really need help?”

Solutions to such challenges require “a whole ecosystem,” said Dr. Nancy Potok, former chief statistician of the United States, in closing remarks. “The researchers at universities, the practitioners, the data producers, the facilities where we can bring the data together … the funders … all of these need to come together. One single element of this pie is never going to make it work.”

“Nobody,” she added, “does this alone.”