## Installation Instructions

- Install dependencies with Anaconda: `conda env create -f environment.yml`.
- Make sure pytorch >= 1.10 is installed.

- Install stochastic_offline_envs 
```
cd stochastic_offline_envs
pip install -e .
```




## Dataset

Download dataset from [[Google Drive](https://drive.google.com/drive/folders/1rcLXgCDpLr7ak7NWdD6iT_lXuQFBP3L7?usp=sharing)] and unzip datasets files (not folders) to the folder `offline_data`


## Training and testing

### Parameters
#### Environments names 
The parameter of environment names
- Single-stage game: `toy`
- Gambling : `gambling`
- Multi-stage game: 'mstoy'
- Connect Four: `connect_four`
- Mujoco: `halfcheetah` or `hopper` or `walker2d`

#### Model Type
ADT or DT, representing conditioning on the past adversarial actions or not.

#### Algorithm
ardt, dt, esper

### Running scripts

Environment with full data coverage: the parameters from left to right: seed,  algorithm and environment name. For example running multi-stage game: `sh run_toy.sh 0 ardt mstoy`.

Connect four: `sh run_c4.sh 0 ardt`. 

Mujoco `sh run_mujoco.sh 0 0 halfcheetah dt 1 dt`, where the parameters from left to right random seed, GPU device, environments parameters names, algorithm name, proportion of mixing two datasets, and model type. For details please refer to the scripts.