# Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling
![MRConv](assets/mrconv.png "MRConv")

This repository provides implementations and experiments for MRConv and subsequent experiments.


## Setup

### Requirements
This repository requires Python 3.8+ and Pytorch 1.10+.
Other packages are listed in [requirements.txt](./requirements.txt).

## Experiments

Adding `wandb=null` to any command line turns off logging.

Some of these datasets may require downloading and preparing data, documented in the [src/dataloaders](./src/dataloaders/) subdirectory.

### Long Range Arena (LRA)

You can use these configs to reproduce the best results on LRA using a random initialization.

```
python -m train experiment=lra/mrconv-base/fourier-lra-listops
python -m train experiment=lra/mrconv-base/fourier-lra-imdb
python -m train experiment=lra/mrconv-base/fourier-lra-cifar
python -m train experiment=lra/mrconv-base/long-conv-lra-aan
python -m train experiment=lra/mrconv-base/dilated-fourier-lra-pathfinder
```

## Acknowledgements

This repo is based on HazyResearch's [safari](https://github.com/HazyResearch/safari) repo and borrows its structure and training scripts.
