Causal Inference and Experimentation

class: center, middle, inverse, title-slide

.title[
# Causal Inference and Experimentation
]
.subtitle[
## University of Melbourne
]
.author[
### Kyle Peyton
]
.date[
### 27 March 2023
]

---

count: false
class: hide-count
<style type="text/css">
.inverse .remark-slide-number {
  display: none;
}
.hide-count .remark-slide-number {
  display: none;
}
.my-slide h2 + p {
  margin-top: 0;
  margin-bottom: 0;
}

.custom-left {
  float: left;
  width: 40%; /* Adjust this percentage for the image */
}

.custom-right {
  float: right;
  width: 60%; /* Adjust this percentage for the bullet points */
}

.custom-left-rev {
  float: left;
  width: 60%; /* Adjust this percentage for the image */
}

.custom-right-rev {
  float: right;
  width: 40%; /* Adjust this percentage for the bullet points */
}

.custom-pull-left {
  float: left;
  width: 50%; /* Adjust this percentage for the image */
}

.custom-pull-right {
  float: right;
  width: 50%; /* Adjust this percentage for the bullet points */
}
</style>

<div style = "position:fixed; visibility: hidden">
`$$\require{color}\definecolor{yellow}{rgb}{1, 0.8, 0.16078431372549}$$`
`$$\require{color}\definecolor{orange}{rgb}{0.96078431372549, 0.525490196078431, 0.203921568627451}$$`
`$$\require{color}\definecolor{green}{rgb}{0, 0.474509803921569, 0.396078431372549}$$`
</div>

# Roadmap for today
--

- Introduction to causal inference and experimentation

- Description of classroom activity to facilitate student learning

- Overview of my research agenda and potential contributions to the School and the University of Melbourne

## Learning objectives:

1. Introduction to counterfactuals and the logic of causal inference

2. Description of the fundamental problem of causal inference

3. Understand why randomization "solves" the problem

4. Discover a new causal effect together!

---
count: false
# What is Causal Inference?

--
<div style="display: flex; justify-content: center; align-items: center; width: 100%;">
  <div style="width: 50%; padding-left: 0px; padding-right: 10px">
    <img src="img/frost_roads.png" style="max-width: 100%; max-height: 100%;">
  </div>

<div style="width: 50%; padding-right: 0px; padding-left: 5px; text-align: left; white-space: nowrap;">
    Two roads diverged in a yellow wood,<br>
    And sorry I could not travel both<br>
    And be one traveler, long I stood<br>
    And looked down one as far as I could<br>
    To where it bent in the undergrowth;<br>
    <br>
    -Robert Frost, <em>The Road Not Taken</em>
  </div>
</div>

We can only observe what happens in reality

We do not observe what would have happened if we made different decisions

We can nevertheless **imagine** these counterfactual worlds

Causal inference involves comparisons between **potential outcomes**

---
# What is Causal Inference?

--
<div style="display: flex; justify-content: center; align-items: center;">
  <div class="slide: data-xaringan-increment" style="background-color: black; color: white; padding: 40px; width: 200px; text-align: center; margin-right: 20px; border-radius: 10px;">
    Factual
  </div>
--
  <span class="slide: data-xaringan-increment" style="font-size: 2em;">vs</span>
  <div class="slide: data-xaringan-increment" style="background-color: white; color: black; padding: 40px; width: 200px; text-align: center; margin-left: 20px; border-radius: 10px; border: 2px dashed black;">
    Counterfactual
  </div>
</div>
--

## Examples of Causal Questions

.remark-slide-content ul li {
    margin-bottom: 0.25em;
  }
    
  /* Decrease the space between the second-level bullet points */
  .remark-slide-content li ul li {
    margin-bottom: 0.25em; 
    margin-top: 0.5em;
  }
  
  .reveal ul ul, .reveal ol ol {
  font-size: inherit;
  }
  
</style>
--

- Do mRNA vaccines (e.g., "Spikevax") reduce the chances of getting Covid?

- Infection rates decreased after a vaccination program was implemented

--
  - Would infection rates have gone up if the vaccination program *was not* implemented?

- Do community policing practices lead to increased public trust and a greater willingness to cooperate with the police?

--
  - Police-community relations are characterized by conflict and distrust in places where the police do not use community policing

--
  - Would things be different in those places if the police *did* use community policing?
  
--

**Fundamental problem of causal inference:** we can never observe counterfactual outcomes, they must be inferred

---
class: left, top
# Example: Covid-19 Vaccination Study
<div style="display: flex; justify-content: center; align-items: center; width: 100%;">
  <div style="width: 50%; padding-left:0; padding-right:5px">
    <img src="img/moderna_dose.png" class="presentation-image" style="max-width: 100%; max-height: 100%;">
    <figcaption style="font-size: small; margin-top: 0px;">Source: <em> Ted S. Warren/Associated Press, </em>2020</figcaption>
  </div>
  <div style="width: 50%; padding-left:5px; padding-right:0px">
    <img src="img/moderna_result.png" class="presentation-image" style="max-width: 100%; max-height: 100%;">
<figcaption style="font-size: small; margin-top: 0px;">Source: Baden et al.,<em> New England Journal of Medicine, </em>2020</figcaption>
  </div>
</div>

images.forEach(function(img) {
      img.onload = function() {
        maxHeight = Math.max(maxHeight, img.clientHeight);
        images.forEach(function(innerImg) {
          innerImg.style.height = maxHeight + 'px';
          innerImg.style.objectFit = 'contain';
        });
      };
    });
  }

setEqualHeight(document.querySelectorAll('.presentation-image'));
</script>

- Can mRNA vaccines reduce the incidence of Covid-19 infections and prevent illness, including severe disease?

- Randomly assign individuals to either receive a shot with the active vaccine ("treatment") or a shot with saline placebo ("control")
  
--

- Compare the number of new symptomatic COVID-19 cases that were reported among participants in each group

---
class: left, top

# Example: Community Policing Study
<div style="display: flex; justify-content: center; align-items: center; width: 100%;">
  <div style="width: 50%; padding-left:0; padding-right:5px">
    <img src="img/door_knock.png" class="presentation-image" style="max-width: 100%; max-height: 100%;">
    <figcaption style="font-size: small; margin-top: 0px;">Source: fieldwork conducted in New Haven, Connecticut, 2019</figcaption>
  </div>
  <div style="width: 50%; padding-left:5px; padding-right:0px">
    <img src="img/pnas_abstract.png" class="presentation-image" style="max-width: 100%; max-height: 100%;">
    <figcaption style="font-size: small; margin-top: 0px;">Source: Peyton et al.,<em> Proceedings of the National Academy of Sciences </em>, 2019</figcaption>
  </div>
</div>

setEqualHeight(document.querySelectorAll('.presentation-image'));
</script>

- Can changing police behavior in the field increase trust and cooperation between officers and the communities they serve?

- Randomly assign households to experience positive, non-enforcement contact with uniformed patrol officers ("treatment") or no contact ("control")

- Compare attitudes towards police (e.g., trust, willingness to cooperate) among residents in each group

---
class:custom
# Measuring Causal Effects: Individual Comparisons

--
<div class="custom-right" style="display: flex; justify-content: center;">
  <p style="margin-top: 2.5em; text-align: left; width: 90%;">Can we observe <strong>both</strong> potential outcomes for the same individual at the exact same time under the same circumstance?</p>
</div>

--
<div data-xaringan-increment class="custom-right" style="display: flex; justify-content: center; align-items: center;">
  <img src="img/nopecat.png" style="width: 40%; height: 6%;">
</div>
<div class="custom-right" style="display: flex; justify-content: center;">
  <p style="margin-bottom: 0; text-align: left; width: 90%;"><strong>Fundamental Problem of Causal Inference</strong></p>
</div>
--
<div class="custom-right" style="display: flex; justify-content: center;">
  <p style="margin-top: 1em; text-align: left; width: 90%;"><strong>Solution</strong>: find "good" approximations for counterfactual outcomes</p>
</div>

---
# Measuring Causal Effects: Individual Comparisons
<br>
<table style="border-collapse: collapse; width: 60%; padding-top: 10px">
  <tr style="background-color: #c6c6c6; border-bottom: 0.5px solid; border-top: 1.5px solid; padding-top: 0px; padding-bottom: 0px;">
    <th style="background-color: #c6c6c6; text-align: left; font-weight: bold; width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 1.2em; line-height: 1.2em; margin-top: 0em;"></div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Community policing visit </div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Trust in police</div></th>
  </tr>
  <tr>
    <td style="text-align: left;">Person 1</td>
    <td style="text-align: center;">Yes</td>
    <td style="text-align: center;">High</td>
  </tr>
  <tr>
    <td style="text-align: left;">Person 2</td>
    <td style="text-align: center;">No</td>
    <td style="text-align: center;">Low</td>
  </tr>
</table>
<br>

Did Person 1 report a high level of trust **because** they received the community policing visit?

<br>
<table style="border-collapse: collapse; width: 90%; padding-top: 10px">
  <tr style="background-color: #c6c6c6; border-bottom: 0.5px solid; border-top: 1.5px solid; padding-top: 0px; padding-bottom: 0px;">
    <th style="background-color: #c6c6c6; text-align: left; font-weight: bold; width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 1.2em; line-height: 1.2em; margin-top: 0em;"></div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Community policing visit </div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Trust in police</div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Race/ethnicity</div></th>
  </tr>
  <tr>
    <td style="text-align: left;">Person 1</td>
    <td style="text-align: center;">Yes</td>
    <td style="text-align: center;">High</td>
    <td style="text-align: center;">White</td>
  </tr>
  <tr>
    <td style="text-align: left;">Person 2</td>
    <td style="text-align: center;">No</td>
    <td style="text-align: center;">Low</td>
    <td style="text-align: center;">Black</td>
  </tr>
</table>
<br>

---
count: false
# Measuring Causal Effects: Individual Comparisons
<br>
<table style="border-collapse: collapse; width: 60%; padding-top: 10px">
  <tr style="background-color: #c6c6c6; border-bottom: 0.5px solid; border-top: 1.5px solid; padding-top: 0px; padding-bottom: 0px;">
    <th style="background-color: #c6c6c6; text-align: left; font-weight: bold; width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 1.2em; line-height: 1.2em; margin-top: 0em;"></div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Community policing visit </div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Trust in police</div></th>
  </tr>
  <tr>
    <td style="text-align: left;">Person 1</td>
    <td style="text-align: center;">Yes</td>
    <td style="text-align: center;">High</td>
  </tr>
  <tr>
    <td style="text-align: left;">Person 2</td>
    <td style="text-align: center;">No</td>
    <td style="text-align: center;">Low</td>
  </tr>
</table>
<br>

Did Person 1 report a high level of trust **because** they received the community policing visit?

<br>
<table style="border-collapse: collapse; width: 90%; padding-top: 10px">
  <tr style="background-color: #c6c6c6; border-bottom: 0.5px solid; border-top: 1.5px solid; padding-top: 0px; padding-bottom: 0px;">
    <th style="background-color: #c6c6c6; text-align: left; font-weight: bold; width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 1.2em; line-height: 1.2em; margin-top: 0em;"></div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Community policing visit </div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Trust in police</div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Race/ethnicity</div></th>
    <th style="width: 100px; word-wrap: break-word; vertical-align: middle;"><div style="height: 2.4em; line-height: 1.2em; margin-top: 0em; display: flex; align-items: center; justify-content: center;">Prior contact</div></th>
  </tr>
  <tr>
    <td style="text-align: left;">Person 1</td>
    <td style="text-align: center;">Yes</td>
    <td style="text-align: center;">High</td>
    <td style="text-align: center;">White</td>
    <td style="text-align: center;">No</td>
  </tr>
  <tr>
    <td style="text-align: left;">Person 2</td>
    <td style="text-align: center;">No</td>
    <td style="text-align: center;">Low</td>
    <td style="text-align: center;">Black</td>
    <td style="text-align: center;">Yes</td>
  </tr>
</table>
<br>

---
class:custom
# Measuring Causal Effects: Overcoming Challenges

<div class="custom-left-rev" style="display: flex; justify-content: center;">
  <p style="margin-top: 2.5em; text-align: left; width: 90%;"><strong>Goal:</strong> compare people that that are identical on all dimensions <em>except for </em> the treatment they received!</p>
</div>

--
<div class="custom-right-rev">
  <img src="img/twins.png" style="width: 100%; height: 100%; margin-top:1.5em; margin-bottom:0em">
</div>

<div class="custom-left-rev" style="display: flex; justify-content: center;">
  <p style="margin-top: 0.5em; text-align: left; width: 90%;"><strong>Problem:</strong> we can't "control for" everything </p>
</div>

<div class="custom-left-rev" style="display: flex; justify-content: center;">
  <p style="margin-top: .5em; text-align: left; width: 90%;"><strong>Solution:</strong> random assignment of treatment!</p>
</div>

<div class="custom-left-rev" style="display: flex; justify-content: center;">
  <p style="margin-top: .5em; text-align: left; width: 90%;"><strong>Randomized experiment:</strong> research design that ensures treatment and control groups are, <em>on average</em>, identical to each other on all observed (and unobserved) background characteristics</p>
</div>

<div class="custom-left-rev" style="display: flex; justify-content: center;">
  <p style="margin-top: .5em; text-align: left; width: 90%;"><strong>Implication:</strong> the only thing that distinguishes the two groups, besides the treatment itself, is chance. </p>
</div>

---
# Randomization in the Policing Experiment

--
.center[
<iframe src="https://kyle-peyton.shinyapps.io/BalanceR/" width="100%" height="600" frameborder="0"></iframe>
]

---
# Classroom Activity: Survey Experiment

- "An experiment that looks like a survey" (Gelman and Nolan. <em> Teaching Statistics.</em> Oxford University Press)

- Short survey before class first asks a few background questions (e.g., age) 
--

- Next, half the students would see the following question, with **Z = 10**. The other half would see the same question, but with **Z = 65**:

.center[
<div style="display: flex;">
  <div class="incremental" style="flex: 1; padding-top: .5em">
    <img src="img/student_survey_low.png" style="width: 60%">
  </div>
</div>
]

---
# Classroom Activity: Discussion and Analysis

**Question 1**: will the two groups be balanced on average age, etc.?

Of course! The groups were created by random assignment.

**Question 2**: will the value of **Z** that students were assigned affect the average guess?

Of course not! Everyone was told it was just a randomly generated number...

However, the average guess when **Z = 10** will be *lower* than when **Z = 65**
--
<img src="peyton_melbourne_27March_V2_files/figure-html/example-plot-1.png" width="720" style="display: block; margin: auto;" />

---
# Overview of research agenda

**Broad interests**: how intergroup relations and attitudes toward government shape beliefs, preferences, and behaviors.

**Empirical approach**: statistics, causal inference, and qualitative field research.

### Intergroup Conflict and Cooperation

--
How does interaction between different social groups influence trust, cooperation, and the dynamics between these groups in various political and social contexts?

### Trust in Government and Inequality

--
How do economic disparities, perceptions of fairness, and peoples' trust in political institutions interact to shape policy preferences and political engagement?

### Fairness, Prejudice, and Political Resentment

--
How do perceptions of fairness and prejudiced attitudes contribute to political resentment and shape the dynamics of group-based conflicts?

---
# Potential contributions

--
**Research and collaboration:**

- "Problem solving" approach uses qualitative and quantitative empiricial methods to address normatively significant, real-world problems

- Strong publication record, and demonstrated capacity to attract research funding

- Experience forging external partnerships and interdisciplinary collaborations

**Teaching and student supervision:**

- Demonstrated excellence in teaching research methods to students at all levels, from a wide variety of disciplines

- Committment to student mentoring and research training using the collaborative lab-style research model

**Service and engagement:**

- Highly skilled at leading complex, large-scale research projects and organising conferences/seminars for variety of audiences

- Keen to contribute to programs/courses in research methods training at all levels
  
---
class: center, middle, inverse
count: false
# Thank you!

<div style="font-size: 1.5em; margin-bottom: 0.5em; margin-top: 0.5em;">
  <span style="display: inline-block; vertical-align: middle;"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg">  <path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"></path></svg> <a href="mailto:kyle.peyton@acu.edu.au">kyle.peyton@acu.edu.au</a></span>
</div>

<div style="font-size: 1.5em; margin-bottom: 0.5em;">
  <span style="display: inline-block; vertical-align: middle;"><svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg">  <path d="M0 352a160 160 0 0 0 160 160h64a160 160 0 0 0 160-160V224H0zM176 0h-16A160 160 0 0 0 0 160v32h176zm48 0h-16v192h176v-32A160 160 0 0 0 224 0z"></path></svg> <a href="https://kyle-peyton.com/">https://kyle-peyton.com/</a></span>
</div>

<div style="font-size: 1.5em; margin-bottom: 0.5em;">
  <span style="display: inline-block; vertical-align: middle;"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg">  <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg> <a href="https://twitter.com/pylekeyton">@pylekeyton</a></span>
</div>