# Layer-wise Orthogonal Training (LOT)

**1. Getting code:**

```bash
git clone git@github.com:taltstidl/lot-x.git
cd lot-x
git checkout auxiliary
```

**2. Creating virtual environment:**

Assumes Python 3.10 is available (may need to load via `module load python/3.10-anaconda` depending on system)

```bash
python -m venv .venv
source .venv/bin/activate
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install einops
```

**3. Installing [NVIDIA apex](https://github.com/NVIDIA/apex)**

```bash
cd ..
git clone git@github.com:NVIDIA/apex.git
pip install -v --disable-pip-version-check --no-cache-dir ./
```

**4. Running commands:**

Uses Slurm's `sbatch` to submit jobs, preferably should run on Nivida A100 nodes.

```bash
# launch.sh SLURM params probably need updating beforehand!
./experiments.sh
# or to launch individual experiments (args: aux scheduler lr epochs block-size)
launch.sh 1m multi-step 0.1 200 4
```

**5. Retrieving results:**

Code will generate files named `output.log`, and best and last models in different folders.

```bash
zip -r results.zip LipConv*
```
