# L smoothness computation experiments for PV-tuning framework
This sub repository contains small-scale experiments for PV tuning framework.

## Algorithm: Approximate Matrix-free Algorithm for Computing $|\nabla f(x)\|$

This algorithm computes an approximation of the norm of the gradient of a function using matrix-free methods.

### Parameters
- Point $x ∈ ℝ^d$
- Fixed $γ ∈ ℝ$ such as $γ = 10^{-5} * x$ for numerical stability.

### Algorithm Steps
1. Initialize $r^0$ randomly uniformly from $ℝ^d$.
2. Calculate the gradient $g = ∇f(x)$.

   For $k = 0, 1, ..., K$:
   - Normalize $r^k$ to get $\hat{r^k} = r^k / \|r^k\|$
   - Update $r^{k+1} = 1/γ * (∇f(x + γ * \hat{r^k}) - g)$ 
     - This step approximates the computation of $r^{k+1} ≈ ∇^2 f * \hat{r^k}$.

### Output
- Returns the approximate eigenvector $r^{K+1} / \|r^{K+1}\|$.
- Corresponding to $|λ_{max}| ≈ \|r^{K+1}\|$.


## Experiments

* [L_smoothness_compute_exp_1.ipynb](L_smoothness_compute_exp_1.ipynb)
* [L_smoothness_compute_exp_2.ipynb](L_smoothness_compute_exp_2.ipynb)
* [L_smoothness_compute_exp_3.ipynb](L_smoothness_compute_exp_3.ipynb)
* [L_smoothness_compute_exp_4.ipynb](L_smoothness_compute_exp_4.ipynb)

## Requirements

Run

```pip install -r requirements.txt```
