Each day will introduce core evidence-based decision making principles with practical exercises that will address how to apply these to security, conflict, and a broad range of policy challenges. Lessons will combine in-class instruction, case studies, and hands-on small team analytical exercises.

90% of 2016 attendees said the course changed their approach to managing information processing.

Day One: Correlation and Causation

Morning (9 a.m. - 12 p.m.)

Key Takeaway

It can be difficult to distinguish between correlations and causal effects. Knowing how to do this is essential because the stakes can be high when we get it wrong.

Agenda

• Introduction (Lecture)

• Is this causal? (Activity, Part 1)

• BREAK

• Correlations and Causal Effects: What they are and what they are useful for (Lecture, Part 1)

• Is this causal? (Activity, Part 2)

• Correlations and Causal Effects: Why knowing the difference matters (Lecture, Part 2)

Lunch

Afternoon (1 p.m. - 5 p.m.)

Key Takeaway

Establishing a correlation is not as easy as we may think. You have to look at the right kinds of evidence.

Agenda

• Correlations and Lessons Learned (Prelude Activity)

• Introduction to the afternoon (Lecture)

• Is this a correlation? (Activity)

• Correlation Requires Variation: Challenges to establishing correlation (Lecture)

• BREAK

• Revisit the prelude (Activity)

• What about when success/failure is not binary? (Lecture)

• Forecasting (Activity)

• Correlations and Causal Effects: Tying it all together (Lecture)

• Closing remarks (Lecture)

Welcome Reception

Day Two: Question Causality

Morning (9 a.m. - 12 p.m.)

Key Takeaways

Sometimes correlation is causation, but you can’t trust an interpretation of causal relationships without accounting for confounders. When using evidence of a correlation to gauge the effect of your planned actions, it is critical to assess whether a causal interpretation of that correlation is credible and plausible. To do so, ask: Are there unaccounted confounders? Is there reverse causality?

Agenda

• Frame the day (Lecture)

• Confounding (Activity)

• Making valid comparisons, drawing valid conclusions (Interactive Lecture)

• Confounding/Selection (Activity)

• BREAK

• Now what? Questions you should ask when someone purports to give you a causal relationship (Lecture)

• Signing the Bias: The first thing to do when you have a confounded relationship (Lecture)

• Signing the Bias (Activity)

Lunch

Afternoon (1 p.m. - 5 p.m.)

Key Takeaways

There are a range of approaches to pulling causal relationships out of potentially confounded correlations.  Four of them are easy to implement and do not require particularly sophisticated analyses: signing the bias, controlling, differencing, and elaborating.

Agenda

• Frame the afternoon (Lecture)

• Controlling (Lecture)

• Controlling (Activity)

• BREAK

• What can you do when you can’t measure the confounders? Differencing (Lecture)

• Thinking through confounders (Activity)

• BREAK

• How do you establish causality when you can’t difference or control? Elaboration (Lecture)

• Pulling it all together (Activity)

Day Three: Principles and Leadership

Morning (9 a.m. - 1 p.m.)

Key Takeaways

Before making an evidence-based decision, be sure to:

• Turn statistics into substance

• Measure your mission

• Make valid analogies

• Build effective teams

• Establish the right culture/norms/processes

• Gather the right data

Agenda

• Frame the day (Lecture)

• Turn statistics into substance (Lecture)

• Measure your mission (Interactive Lecture)

• Did you measure your mission? (Activity)

• BREAK

• Make Valid Analogies: External validity (Interactive Lecture)

• General Principles/Information Processing Best Practices (Lecture)

• Leading evidence-based decisions (Lecture)

• Closing remarks (Lecture)

• LUNCH

The last day is a half-day only.