# auditing-human-experts

This repository contains the supplementary appendix (Auditing_for_Human_Expertise_Appendix.pdf) and code to replicate the experiments in our paper "Auditing for Human Expertise".

The entry point to replicate the case study of gastrointestinal hospitalization decisions (Section 5) is available in gib_experiment.R. This can be run interactively, and assumes the working directory is set to the './code/' folder.

The entry point to replicate the numerical simulations (Appendix F) is figures.R. This can also be run interactively, and assumes the working directory is set to the './code/' folder.

Data for the experiments, including saved copies of the numerical simulation data, are available in the ./data/ folder. All simulated data and other results can be regenerated by disabling the 'use.saved.data' flag at the top of figures.R and/or gib_experiment.R. Running the code without disabling this flag will regenerate the figures from saved results. Generating the figures takes only a few seconds; running the entire set of simulations takes approximately 30-60 minutes on a standard personal laptop. Qualitatively similar results can be obtained more quickly by reducing the number of iterations (the 'reps' argument) for each simulation to 10.

Figures and tables are written to ./figures/.

ExpertTest is implemented in logic.R. Additional helper methods are in helpers.R.

Our code depends on the following packages: 

library(tidyverse)
library(data.table)
library(ggplot2)
library(glue)
library(Rfast) # provides a (very!) efficient implementation of the NxN distance calculation
