<h1>Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Image</h1>

# Installation

Please follow [ICON](https://github.com/YuliangXiu/ICON/blob/master/docs/installation.md) to initialize the environment, download the extra data, such as HPS and SMPL (using `fetch_hps.sh` and `fetch_data.sh`).

```bash
pip install -r requirements.txt
```

The 130 line of `config/structured.py` need to be the global path of the directory, for example `/home/{user}/MHCDiff`.

Download smpl files and pretrained model for [ProPose](https://github.com/NetEase-GameAI/ProPose) and place them at `hps/ProPose/model_files`

# Train

Sampling points from the raw scan.

```bash
python fps_sampling_thuman.py
```

Train the model

```bash
python main.py run.name={name} dataset.hps=gt
```

# Inference

## CAPE dataset

Random mask the CAPE dataset

```
python random_masking.py
```

```bash
python main.py run.name={name} run.job=sample checkpoint.resume={global path for pretrained checkpoint} dataset.type=cape dataset.hps=propose
```

## MultiHuman dataset

Download [MultiHuman dataset](https://github.com/y-zheng18/MultiHuman-Dataset)

The Original Structure (MultiHuman.zip)

```
MultiHuman
├──single
├──single_occluded
├──three
├──two_closely_inter
├──two_naturally_interactive
```

Place the folder on `data/MultiHuman`

Render the scan data to image

```bash
bash render_multi.sh
```

Inference

```bash
python main.py run.name={name} run.job=sample checkpoint.resume={pretrained checkpoint path} dataset.type=multihuman dataset.category=single dataset.hps=propose

python main.py run.name={name} run.job=sample checkpoint.resume={pretrained checkpoint path} dataset.type=multihuman dataset.category=single_occluded dataset.hps=propose

python main.py run.name={name} run.job=sample checkpoint.resume={pretrained checkpoint path} dataset.type=multihuman dataset.category=three dataset.hps=propose

python main.py run.name={name} run.job=sample checkpoint.resume={pretrained checkpoint path} dataset.type=multihuman dataset.category=two_closely_inter dataset.hps=propose

python main.py run.name={name} run.job=sample checkpoint.resume={pretrained checkpoint path} dataset.type=multihuman dataset.category=two_naturally_interactive dataset.hps=propose
```
