Q&A: Professor Ethan Bueno de Mesquita on "Leading Evidence-Based Decisions"

How did the “Leading Evidence-Based Decisions” executive education program come about?

Jacob Shapiro from Princeton and I put together this class to fill a particular need. From talking with professionals working on national security policy, development policy, and in the intelligence community, it became clear there was a desire for a different kind of leadership training.  

With the datafication of society, effective policy decision-making increasingly requires leaders, who are not themselves experts in data analysis, to make sound decisions informed by evidence created or provided by their own or others’ teams of analysts. We believe policy schools have focused too much on creating those analysts (which is, of course, important). We wanted to do something to help develop effective leaders who can critically evaluate, interpret and reflect on data analysis to make more effective decisions.

The people at the top don’t need to be specialists; they just need to have the conceptual tools that allow them to make the best possible decisions based on what the specialists are saying. With those tools, they can be sophisticated, skeptical consumers of the analysis they are getting, which will help them lead and better shape policy.

What is the goal of the program?

This program trains policy leaders, government executives, and other professionals in the principles of evidence-based decision making. Participants will gain a working knowledge of these key principles, enabling them to ask smarter questions of their analysts and make better decisions. 

Imagine you’re in a room with a bunch of data analysts, all throwing numbers at you, and you have to decide how to act. Maybe it’s a question of war and peace. Maybe it’s debating a new social safety net program. Does the data really mean what the analysts say it means? How do you ask the right questions to find out?  

The goal is to give you as a leader the tools to understand data analysis when it’s presented, and to recognize common mistakes and fallacies people fall into when interpreting evidence. A leader is only as effective as the information that he or she is given, and his or her ability to make sense of it.

Who should consider taking this course?

Jake and I were initially inspired by our interactions with people in the national security, intelligence, and development communities. But the course is broader than that. It is designed for a leader who relies on evidence-based analysis and recommendations to make important decisions – it’s not a class in data analysis, but in understanding and evaluating data analysis when it’s presented. It is meant for senior civilian government officials, senior military staff and commanders, strategic advisors, executive officers, and nonprofit and social sector senior leadership staff members – in a nutshell, professionals who receive and utilize evidence to weigh their decision-making. Or those who have to describe those decisions, like journalists or speechwriters. 

At the end of this course, you should be able to spot warranted versus unwarranted conclusions, be able to ask the right questions, and demand evidence that actually supports the conclusions that are being presented to you.

What will participants learn?

Participants will learn key questions to ask whenever making an evidence-informed decision, how our institutions and standard operating procedures force us to make basic decision-making errors and how to correct this, how to turn statistics into substance, the importance of measuring your mission, and other, simple tools for probing the credibility of an evidentiary claim.

Through a combination of in-class instruction, timely examples, and experiential group exercises, participants should leave with a transformed understanding of how to extract credible, substantive, and actionable information from evidence-based analyses.

What have people said about the program?

It’s really opened some eyes about the ways we use data. One participant said that in his extensive experience in Washington, DC, nobody ever questions how data is presented in the way that we teach our students to. Another, a former speech writer, realized how many fallacies (and it’s a lot) she had accidentally put into a former cabinet secretary’s speeches, without even realizing it. 

We’ve gotten really positive feedback about the student-centered approach mixing lectures, interactivities, and team work. It’s a really activated classroom. We get to talk about issues across many policy areas, forcing students to talk about the big concepts, not getting bogged down in minutiae and missing the forest for the trees. So for me, it’s a lot of fun to teach.

Related Content:

Learn more about the Leading Evidence-Based Decisions program