Arch: resnet50_pt
Bs trn: 128
Bs val: 128
Hidden dim: 256
Dataset: celebA
Resample class: 
Slice with: rep
Rep cluster method: gmm
Num anchor: 32
Num positive: 32
Num negative: 32
Num negative easy: 0
Weight anc by loss: False
Weight pos by loss: False
Weight neg by loss: False
Anc loss temp: 0.5
Pos loss temp: 0.5
Neg loss temp: 0.5
Data wide pos: False
Target sample ratio: 1
Balance targets: False
Additional negatives: False
Hard negative factor: 0
Full contrastive: False
Train encoder: False
No projection head: False
Projection dim: 128
Batch factor: None
Temperature: 0.05
Single pos: False
Supervised linear scale up: False
Supervised update delay: 0
Contrastive weight: 0.5
Classifier update interval: 8
Optim: sgd
Max epoch: 50
Lr: 0.0001
Momentum: 0.9
Weight decay: 0.1
Weight decay c: 0.1
Stopping window: 30
Load encoder: 
Freeze encoder: False
Finetune epochs: 0
Clip grad norm: False
Lr scheduler classifier: 
Lr scheduler: 
Grad clip grad norm: False
Erm: False
Erm only: False
Pretrained spurious path: ./model/celebA/config/stage_one_erm/seed25/stage_one_erm_model_b_epoch0_seed25.pt
Max epoch s: 1
Bs trn s: 32
Lr s: 0.001
Momentum s: 0.9
Weight decay s: 0.0005
Slice temp: 10
Log loss interval: 10
Checkpoint interval: 50
Grad checkpoint interval: 50
Log visual interval: 100
Log grad visual interval: 50
Verbose: True
Seed: 25
Replicate: 0
No cuda: False
Resume: False
New slice: False
Num workers: 12
Evaluate: False
Data cmap: hsv
Test cmap: 
P correlation: 0.9
P corr by class: None
Train classes: ['blond', 'nonblond']
Train class ratios: None
Test shift: random
Flipped: False
Q: 0.7
Pretrained bmodel: True
Cosine: False
Exp: ours
Tau: 1.1
Gamma: None
Remove label noise: False
Model for remove samples: 
Remove ratio: 0.03
Supervised contrast: True
Prioritize spurious pos: False
Contrastive type: cnc
Compute auroc: False
Model type: resnet50_pt_cnc
Criterion: cross_entropy
Pretrained: False
Max grad norm: 1.0
Adam epsilon: 1e-08
Warmup steps: 0
Max grad norm s: 1.0
Adam epsilon s: 1e-08
Warmup steps s: 0
Grad max grad norm: 1.0
Grad adam epsilon: 1e-08
Grad warmup steps: 0
Device: cuda
Img file type: .png
Display image: False
Image path: ./images/celebA/celebA/config/contrastive_umaps
Log interval: 1
Log path: ./logs/celebA/config
Results path: ./results/celebA/config
Model path: ./model/celebA/config
Loss factor: 1
Supersample labels: False
Subsample labels: False
Weigh slice samples by loss: True
Val split: 0.2
Spurious train split: 0.2
Subsample groups: False
Train method: sc
Max robust acc: -1
Max robust epoch: -1
Max robust group acc: (None, None)
Root dir: ./datasets/data/CelebA/
Target name: Blond_Hair
Confounder names: ['Male']
Image mean: 0.449
Image std: 0.226
Augment data: False
Task: celebA
Num classes: 2
Experiment configs: config
Experiment name: cnc-celebA-sw=re-na=32-np=32-nn=32-nne=0-tsr=1-t=0.05-bf=None-cw=0.5-sud=0-me=50-bst=128-o=sgd-lr=0.0001-mo=0.9-wd=0.1-wdc=0.1-spur-me=1-bst=32-lr=0.001-mo=0.9-wd=0.0005-sts=0.2-s=25-r=0
Mi resampled: None

Loading checkpoints for train split:
[-1 -1 -1 ... -1 -1 -1]
<class 'numpy.ndarray'>
[0 1 2 3] [71629 66874 22880  1387]
Loading checkpoints for val split:
[-1 -1 -1 ... -1  1 -1]
<class 'numpy.ndarray'>
[0 1 2 3] [8535 8276 2874  182]
Loading checkpoints for test split:
[-1 -1 -1 ... -1 -1  1]
<class 'numpy.ndarray'>
[0 1 2 3] [9767 7535 2480  180]
Train dataset:
    Blond_Hair = 0, Male = 0 : n = 71629
    Blond_Hair = 0, Male = 1 : n = 66874
    Blond_Hair = 1, Male = 0 : n = 22880
    Blond_Hair = 1, Male = 1 : n = 1387
Val dataset:
    Blond_Hair = 0, Male = 0 : n = 8535
    Blond_Hair = 0, Male = 1 : n = 8276
    Blond_Hair = 1, Male = 0 : n = 2874
    Blond_Hair = 1, Male = 1 : n = 182
Test dataset:
    Blond_Hair = 0, Male = 0 : n = 9767
    Blond_Hair = 0, Male = 1 : n = 7535
    Blond_Hair = 1, Male = 0 : n = 2480
    Blond_Hair = 1, Male = 1 : n = 180
------------------------
> Loading spurious model
------------------------
Pretrained model loaded from ./model/celebA/config/stage_one_erm/seed25/stage_one_erm_model_b_epoch0_seed25.pt
======
# Calculate probability ...
======
======
p_y_a:  tensor([[0.8256, 0.0253],
        [0.1347, 0.0144]])
p_y:  tensor([0.8509, 0.1491])
# Load biased model ...
======
Epoch:   1 | Train Loss: 0.002 | Train Acc: 90.464 | Val Loss: 0.003 | Val Acc: 81.895
Training:
Accuracies by groups:
0, 0  acc:  8229 / 17019 =  48.352
0, 1  acc:  4749 /  8535 =  55.641
1, 0  acc: 126354 / 128788 =  98.110
1, 1  acc:  7916 /  8428 =  93.925
--------------------------------------
Average acc: 147248 / 162770 =  90.464
Robust  acc:  8229 / 17019 =  48.352
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6353 /  8535 =  74.435
0, 1  acc:  6905 /  8276 =  83.434
1, 0  acc:  2844 /  2874 =  98.956
1, 1  acc:   168 /   182 =  92.308
------------------------------------
Average acc: 16270 / 19867 =  81.895
Robust  acc:  6353 /  8535 =  74.435
------------------------------------
New max robust acc: 74.43468072642062
debias model - Saving best checkpoint at epoch 0
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 83.864
Robust Acc: 79.810 | Best Acc: 98.790
-------------------------------------
Training, Epoch 0:
Accuracies by groups:
0, 0  acc:  7795 /  9767 =  79.810
0, 1  acc:  6328 /  7535 =  83.981
1, 0  acc:  2450 /  2480 =  98.790
1, 1  acc:   168 /   180 =  93.333
------------------------------------
Average acc: 16741 / 19962 =  83.864
Robust  acc:  7795 /  9767 =  79.810
------------------------------------
Accuracies by groups:
0, 0  acc:  7795 /  9767 =  79.810
0, 1  acc:  6328 /  7535 =  83.981
1, 0  acc:  2450 /  2480 =  98.790
1, 1  acc:   168 /   180 =  93.333
------------------------------------
Average acc: 16741 / 19962 =  83.864
Robust  acc:  7795 /  9767 =  79.810
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7795 /  9767 =  79.810
0, 1  acc:  6328 /  7535 =  83.981
1, 0  acc:  2450 /  2480 =  98.790
1, 1  acc:   168 /   180 =  93.333
------------------------------------
Average acc: 16741 / 19962 =  83.864
Robust  acc:  7795 /  9767 =  79.810
------------------------------------
Epoch:   2 | Train Loss: 0.001 | Train Acc: 94.561 | Val Loss: 0.003 | Val Acc: 85.499
Training:
Accuracies by groups:
0, 0  acc: 11870 / 16979 =  69.910
0, 1  acc:  6795 /  8507 =  79.875
1, 0  acc: 127666 / 129116 =  98.877
1, 1  acc:  7586 /  8168 =  92.875
--------------------------------------
Average acc: 153917 / 162770 =  94.561
Robust  acc: 11870 / 16979 =  69.910
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6594 /  8535 =  77.258
0, 1  acc:  7377 /  8276 =  89.137
1, 0  acc:  2843 /  2874 =  98.921
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 16986 / 19867 =  85.499
Robust  acc:  6594 /  8535 =  77.258
------------------------------------
New max robust acc: 77.25834797891036
debias model - Saving best checkpoint at epoch 1
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 87.030
Robust Acc: 82.298 | Best Acc: 98.629
-------------------------------------
Training, Epoch 1:
Accuracies by groups:
0, 0  acc:  8038 /  9767 =  82.298
0, 1  acc:  6726 /  7535 =  89.263
1, 0  acc:  2446 /  2480 =  98.629
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 17373 / 19962 =  87.030
Robust  acc:  8038 /  9767 =  82.298
------------------------------------
Accuracies by groups:
0, 0  acc:  8038 /  9767 =  82.298
0, 1  acc:  6726 /  7535 =  89.263
1, 0  acc:  2446 /  2480 =  98.629
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 17373 / 19962 =  87.030
Robust  acc:  8038 /  9767 =  82.298
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8038 /  9767 =  82.298
0, 1  acc:  6726 /  7535 =  89.263
1, 0  acc:  2446 /  2480 =  98.629
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 17373 / 19962 =  87.030
Robust  acc:  8038 /  9767 =  82.298
------------------------------------
Epoch:   3 | Train Loss: 0.001 | Train Acc: 95.177 | Val Loss: 0.002 | Val Acc: 87.905
Training:
Accuracies by groups:
0, 0  acc: 12168 / 16866 =  72.145
0, 1  acc:  7236 /  8534 =  84.790
1, 0  acc: 127802 / 129075 =  99.014
1, 1  acc:  7713 /  8295 =  92.984
--------------------------------------
Average acc: 154919 / 162770 =  95.177
Robust  acc: 12168 / 16866 =  72.145
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6828 /  8535 =  80.000
0, 1  acc:  7631 /  8276 =  92.206
1, 0  acc:  2833 /  2874 =  98.573
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 17464 / 19867 =  87.905
Robust  acc:  6828 /  8535 =  80.000
------------------------------------
New max robust acc: 80.0
debias model - Saving best checkpoint at epoch 2
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 88.809
Robust Acc: 84.253 | Best Acc: 98.145
-------------------------------------
Training, Epoch 2:
Accuracies by groups:
0, 0  acc:  8229 /  9767 =  84.253
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2434 /  2480 =  98.145
1, 1  acc:   154 /   180 =  85.556
------------------------------------
Average acc: 17728 / 19962 =  88.809
Robust  acc:  8229 /  9767 =  84.253
------------------------------------
Accuracies by groups:
0, 0  acc:  8229 /  9767 =  84.253
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2434 /  2480 =  98.145
1, 1  acc:   154 /   180 =  85.556
------------------------------------
Average acc: 17728 / 19962 =  88.809
Robust  acc:  8229 /  9767 =  84.253
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8229 /  9767 =  84.253
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2434 /  2480 =  98.145
1, 1  acc:   154 /   180 =  85.556
------------------------------------
Average acc: 17728 / 19962 =  88.809
Robust  acc:  8229 /  9767 =  84.253
------------------------------------
Epoch:   4 | Train Loss: 0.001 | Train Acc: 95.731 | Val Loss: 0.002 | Val Acc: 88.337
Training:
Accuracies by groups:
0, 0  acc: 12831 / 17198 =  74.608
0, 1  acc:  7480 /  8557 =  87.414
1, 0  acc: 127588 / 128642 =  99.181
1, 1  acc:  7923 /  8373 =  94.626
--------------------------------------
Average acc: 155822 / 162770 =  95.731
Robust  acc: 12831 / 17198 =  74.608
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6907 /  8535 =  80.926
0, 1  acc:  7643 /  8276 =  92.351
1, 0  acc:  2828 /  2874 =  98.399
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 17550 / 19867 =  88.337
Robust  acc:  6907 /  8535 =  80.926
------------------------------------
New max robust acc: 80.92560046865846
debias model - Saving best checkpoint at epoch 3
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 89.194
Robust Acc: 84.857 | Best Acc: 97.984
-------------------------------------
Training, Epoch 3:
Accuracies by groups:
0, 0  acc:  8288 /  9767 =  84.857
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   158 /   180 =  87.778
------------------------------------
Average acc: 17805 / 19962 =  89.194
Robust  acc:  8288 /  9767 =  84.857
------------------------------------
Accuracies by groups:
0, 0  acc:  8288 /  9767 =  84.857
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   158 /   180 =  87.778
------------------------------------
Average acc: 17805 / 19962 =  89.194
Robust  acc:  8288 /  9767 =  84.857
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8288 /  9767 =  84.857
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   158 /   180 =  87.778
------------------------------------
Average acc: 17805 / 19962 =  89.194
Robust  acc:  8288 /  9767 =  84.857
------------------------------------
Epoch:   5 | Train Loss: 0.001 | Train Acc: 96.204 | Val Loss: 0.002 | Val Acc: 90.079
Training:
Accuracies by groups:
0, 0  acc: 13115 / 17192 =  76.285
0, 1  acc:  7534 /  8414 =  89.541
1, 0  acc: 127930 / 128810 =  99.317
1, 1  acc:  8012 /  8354 =  95.906
--------------------------------------
Average acc: 156591 / 162770 =  96.204
Robust  acc: 13115 / 17192 =  76.285
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7141 /  8535 =  83.667
0, 1  acc:  7771 /  8276 =  93.898
1, 0  acc:  2816 /  2874 =  97.982
1, 1  acc:   168 /   182 =  92.308
------------------------------------
Average acc: 17896 / 19867 =  90.079
Robust  acc:  7141 /  8535 =  83.667
------------------------------------
New max robust acc: 83.6672524897481
debias model - Saving best checkpoint at epoch 4
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 90.572
Robust Acc: 84.444 | Best Acc: 97.581
-------------------------------------
Training, Epoch 4:
Accuracies by groups:
0, 0  acc:  8466 /  9767 =  86.680
0, 1  acc:  7042 /  7535 =  93.457
1, 0  acc:  2420 /  2480 =  97.581
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 18080 / 19962 =  90.572
Robust  acc:   152 /   180 =  84.444
------------------------------------
Accuracies by groups:
0, 0  acc:  8466 /  9767 =  86.680
0, 1  acc:  7042 /  7535 =  93.457
1, 0  acc:  2420 /  2480 =  97.581
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 18080 / 19962 =  90.572
Robust  acc:   152 /   180 =  84.444
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8466 /  9767 =  86.680
0, 1  acc:  7042 /  7535 =  93.457
1, 0  acc:  2420 /  2480 =  97.581
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 18080 / 19962 =  90.572
Robust  acc:   152 /   180 =  84.444
------------------------------------
Epoch:   6 | Train Loss: 0.001 | Train Acc: 96.811 | Val Loss: 0.002 | Val Acc: 90.557
Training:
Accuracies by groups:
0, 0  acc: 13386 / 17017 =  78.663
0, 1  acc:  7659 /  8457 =  90.564
1, 0  acc: 128561 / 129124 =  99.564
1, 1  acc:  7973 /  8172 =  97.565
--------------------------------------
Average acc: 157579 / 162770 =  96.811
Robust  acc: 13386 / 17017 =  78.663
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7167 /  8535 =  83.972
0, 1  acc:  7842 /  8276 =  94.756
1, 0  acc:  2818 /  2874 =  98.051
1, 1  acc:   164 /   182 =  90.110
------------------------------------
Average acc: 17991 / 19867 =  90.557
Robust  acc:  7167 /  8535 =  83.972
------------------------------------
New max robust acc: 83.97188049209139
debias model - Saving best checkpoint at epoch 5
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 91.018
Robust Acc: 80.556 | Best Acc: 97.379
-------------------------------------
Training, Epoch 5:
Accuracies by groups:
0, 0  acc:  8492 /  9767 =  86.946
0, 1  acc:  7117 /  7535 =  94.453
1, 0  acc:  2415 /  2480 =  97.379
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18169 / 19962 =  91.018
Robust  acc:   145 /   180 =  80.556
------------------------------------
Accuracies by groups:
0, 0  acc:  8492 /  9767 =  86.946
0, 1  acc:  7117 /  7535 =  94.453
1, 0  acc:  2415 /  2480 =  97.379
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18169 / 19962 =  91.018
Robust  acc:   145 /   180 =  80.556
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8492 /  9767 =  86.946
0, 1  acc:  7117 /  7535 =  94.453
1, 0  acc:  2415 /  2480 =  97.379
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18169 / 19962 =  91.018
Robust  acc:   145 /   180 =  80.556
------------------------------------
Epoch:   7 | Train Loss: 0.001 | Train Acc: 97.401 | Val Loss: 0.002 | Val Acc: 91.755
Training:
Accuracies by groups:
0, 0  acc: 13887 / 17058 =  81.410
0, 1  acc:  7879 /  8542 =  92.238
1, 0  acc: 128461 / 128767 =  99.762
1, 1  acc:  8313 /  8403 =  98.929
--------------------------------------
Average acc: 158540 / 162770 =  97.401
Robust  acc: 13887 / 17058 =  81.410
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7364 /  8535 =  86.280
0, 1  acc:  7903 /  8276 =  95.493
1, 0  acc:  2799 /  2874 =  97.390
1, 1  acc:   163 /   182 =  89.560
------------------------------------
Average acc: 18229 / 19867 =  91.755
Robust  acc:  7364 /  8535 =  86.280
------------------------------------
New max robust acc: 86.28002343292326
debias model - Saving best checkpoint at epoch 6
replace: True
-> Updating checkpoint debias-wga-best_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-wga-best_seed25.pt
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 92.085
Robust Acc: 78.889 | Best Acc: 96.653
-------------------------------------
Training, Epoch 6:
Accuracies by groups:
0, 0  acc:  8674 /  9767 =  88.809
0, 1  acc:  7169 /  7535 =  95.143
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 18382 / 19962 =  92.085
Robust  acc:   142 /   180 =  78.889
------------------------------------
Accuracies by groups:
0, 0  acc:  8674 /  9767 =  88.809
0, 1  acc:  7169 /  7535 =  95.143
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 18382 / 19962 =  92.085
Robust  acc:   142 /   180 =  78.889
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8674 /  9767 =  88.809
0, 1  acc:  7169 /  7535 =  95.143
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 18382 / 19962 =  92.085
Robust  acc:   142 /   180 =  78.889
------------------------------------
Epoch:   8 | Train Loss: 0.001 | Train Acc: 97.889 | Val Loss: 0.001 | Val Acc: 92.933
Training:
Accuracies by groups:
0, 0  acc: 14634 / 17377 =  84.215
0, 1  acc:  8127 /  8649 =  93.965
1, 0  acc: 128272 / 128406 =  99.896
1, 1  acc:  8301 /  8338 =  99.556
--------------------------------------
Average acc: 159334 / 162770 =  97.889
Robust  acc: 14634 / 17377 =  84.215
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7542 /  8535 =  88.366
0, 1  acc:  8018 /  8276 =  96.883
1, 0  acc:  2754 /  2874 =  95.825
1, 1  acc:   149 /   182 =  81.868
------------------------------------
Average acc: 18463 / 19867 =  92.933
Robust  acc:   149 /   182 =  81.868
------------------------------------
-------------------------------------------
Avg Test Loss: 0.001 | Avg Test Acc: 93.157
Robust Acc: 73.889 | Best Acc: 96.510
-------------------------------------
Training, Epoch 7:
Accuracies by groups:
0, 0  acc:  8813 /  9767 =  90.232
0, 1  acc:  7272 /  7535 =  96.510
1, 0  acc:  2378 /  2480 =  95.887
1, 1  acc:   133 /   180 =  73.889
------------------------------------
Average acc: 18596 / 19962 =  93.157
Robust  acc:   133 /   180 =  73.889
------------------------------------
Accuracies by groups:
0, 0  acc:  8813 /  9767 =  90.232
0, 1  acc:  7272 /  7535 =  96.510
1, 0  acc:  2378 /  2480 =  95.887
1, 1  acc:   133 /   180 =  73.889
------------------------------------
Average acc: 18596 / 19962 =  93.157
Robust  acc:   133 /   180 =  73.889
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8813 /  9767 =  90.232
0, 1  acc:  7272 /  7535 =  96.510
1, 0  acc:  2378 /  2480 =  95.887
1, 1  acc:   133 /   180 =  73.889
------------------------------------
Average acc: 18596 / 19962 =  93.157
Robust  acc:   133 /   180 =  73.889
------------------------------------
Epoch:   9 | Train Loss: 0.001 | Train Acc: 98.298 | Val Loss: 0.001 | Val Acc: 93.401
Training:
Accuracies by groups:
0, 0  acc: 14849 / 17108 =  86.796
0, 1  acc:  7987 /  8394 =  95.151
1, 0  acc: 128816 / 128906 =  99.930
1, 1  acc:  8347 /  8362 =  99.821
--------------------------------------
Average acc: 159999 / 162770 =  98.298
Robust  acc: 14849 / 17108 =  86.796
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7631 /  8535 =  89.408
0, 1  acc:  8077 /  8276 =  97.595
1, 0  acc:  2717 /  2874 =  94.537
1, 1  acc:   131 /   182 =  71.978
------------------------------------
Average acc: 18556 / 19867 =  93.401
Robust  acc:   131 /   182 =  71.978
------------------------------------
-------------------------------------------
Avg Test Loss: 0.001 | Avg Test Acc: 93.643
Robust Acc: 65.556 | Best Acc: 97.518
-------------------------------------
Training, Epoch 8:
Accuracies by groups:
0, 0  acc:  8909 /  9767 =  91.215
0, 1  acc:  7348 /  7535 =  97.518
1, 0  acc:  2318 /  2480 =  93.468
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18693 / 19962 =  93.643
Robust  acc:   118 /   180 =  65.556
------------------------------------
Accuracies by groups:
0, 0  acc:  8909 /  9767 =  91.215
0, 1  acc:  7348 /  7535 =  97.518
1, 0  acc:  2318 /  2480 =  93.468
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18693 / 19962 =  93.643
Robust  acc:   118 /   180 =  65.556
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8909 /  9767 =  91.215
0, 1  acc:  7348 /  7535 =  97.518
1, 0  acc:  2318 /  2480 =  93.468
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18693 / 19962 =  93.643
Robust  acc:   118 /   180 =  65.556
------------------------------------
Epoch:  10 | Train Loss: 0.001 | Train Acc: 98.606 | Val Loss: 0.001 | Val Acc: 94.302
Training:
Accuracies by groups:
0, 0  acc: 15517 / 17338 =  89.497
0, 1  acc:  8250 /  8561 =  96.367
1, 0  acc: 128459 / 128567 =  99.916
1, 1  acc:  8275 /  8304 =  99.651
--------------------------------------
Average acc: 160501 / 162770 =  98.606
Robust  acc: 15517 / 17338 =  89.497
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7849 /  8535 =  91.963
0, 1  acc:  8100 /  8276 =  97.873
1, 0  acc:  2652 /  2874 =  92.276
1, 1  acc:   134 /   182 =  73.626
------------------------------------
Average acc: 18735 / 19867 =  94.302
Robust  acc:   134 /   182 =  73.626
------------------------------------
-------------------------------------------
Avg Test Loss: 0.001 | Avg Test Acc: 94.129
Robust Acc: 65.556 | Best Acc: 97.306
-------------------------------------
Training, Epoch 9:
Accuracies by groups:
0, 0  acc:  9083 /  9767 =  92.997
0, 1  acc:  7332 /  7535 =  97.306
1, 0  acc:  2257 /  2480 =  91.008
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18790 / 19962 =  94.129
Robust  acc:   118 /   180 =  65.556
------------------------------------
Accuracies by groups:
0, 0  acc:  9083 /  9767 =  92.997
0, 1  acc:  7332 /  7535 =  97.306
1, 0  acc:  2257 /  2480 =  91.008
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18790 / 19962 =  94.129
Robust  acc:   118 /   180 =  65.556
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9083 /  9767 =  92.997
0, 1  acc:  7332 /  7535 =  97.306
1, 0  acc:  2257 /  2480 =  91.008
1, 1  acc:   118 /   180 =  65.556
------------------------------------
Average acc: 18790 / 19962 =  94.129
Robust  acc:   118 /   180 =  65.556
------------------------------------
Epoch:  11 | Train Loss: 0.001 | Train Acc: 98.615 | Val Loss: 0.001 | Val Acc: 94.151
Training:
Accuracies by groups:
0, 0  acc: 15190 / 16912 =  89.818
0, 1  acc:  8088 /  8405 =  96.228
1, 0  acc: 128887 / 129048 =  99.875
1, 1  acc:  8351 /  8405 =  99.358
--------------------------------------
Average acc: 160516 / 162770 =  98.615
Robust  acc: 15190 / 16912 =  89.818
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7767 /  8535 =  91.002
0, 1  acc:  8120 /  8276 =  98.115
1, 0  acc:  2689 /  2874 =  93.563
1, 1  acc:   129 /   182 =  70.879
------------------------------------
Average acc: 18705 / 19867 =  94.151
Robust  acc:   129 /   182 =  70.879
------------------------------------
-------------------------------------------
Avg Test Loss: 0.001 | Avg Test Acc: 94.124
Robust Acc: 64.444 | Best Acc: 97.717
-------------------------------------
Training, Epoch 10:
Accuracies by groups:
0, 0  acc:  9025 /  9767 =  92.403
0, 1  acc:  7363 /  7535 =  97.717
1, 0  acc:  2285 /  2480 =  92.137
1, 1  acc:   116 /   180 =  64.444
------------------------------------
Average acc: 18789 / 19962 =  94.124
Robust  acc:   116 /   180 =  64.444
------------------------------------
Accuracies by groups:
0, 0  acc:  9025 /  9767 =  92.403
0, 1  acc:  7363 /  7535 =  97.717
1, 0  acc:  2285 /  2480 =  92.137
1, 1  acc:   116 /   180 =  64.444
------------------------------------
Average acc: 18789 / 19962 =  94.124
Robust  acc:   116 /   180 =  64.444
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9025 /  9767 =  92.403
0, 1  acc:  7363 /  7535 =  97.717
1, 0  acc:  2285 /  2480 =  92.137
1, 1  acc:   116 /   180 =  64.444
------------------------------------
Average acc: 18789 / 19962 =  94.124
Robust  acc:   116 /   180 =  64.444
------------------------------------
Epoch:  12 | Train Loss: 0.001 | Train Acc: 98.235 | Val Loss: 0.002 | Val Acc: 91.378
Training:
Accuracies by groups:
0, 0  acc: 14950 / 16993 =  87.977
0, 1  acc:  8356 /  8729 =  95.727
1, 0  acc: 128492 / 128835 =  99.734
1, 1  acc:  8099 /  8213 =  98.612
--------------------------------------
Average acc: 159897 / 162770 =  98.235
Robust  acc: 14950 / 16993 =  87.977
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7372 /  8535 =  86.374
0, 1  acc:  7859 /  8276 =  94.961
1, 0  acc:  2767 /  2874 =  96.277
1, 1  acc:   156 /   182 =  85.714
------------------------------------
Average acc: 18154 / 19867 =  91.378
Robust  acc:   156 /   182 =  85.714
------------------------------------
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 91.845
Robust Acc: 80.556 | Best Acc: 95.524
-------------------------------------
Training, Epoch 11:
Accuracies by groups:
0, 0  acc:  8676 /  9767 =  88.830
0, 1  acc:  7144 /  7535 =  94.811
1, 0  acc:  2369 /  2480 =  95.524
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18334 / 19962 =  91.845
Robust  acc:   145 /   180 =  80.556
------------------------------------
Accuracies by groups:
0, 0  acc:  8676 /  9767 =  88.830
0, 1  acc:  7144 /  7535 =  94.811
1, 0  acc:  2369 /  2480 =  95.524
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18334 / 19962 =  91.845
Robust  acc:   145 /   180 =  80.556
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8676 /  9767 =  88.830
0, 1  acc:  7144 /  7535 =  94.811
1, 0  acc:  2369 /  2480 =  95.524
1, 1  acc:   145 /   180 =  80.556
------------------------------------
Average acc: 18334 / 19962 =  91.845
Robust  acc:   145 /   180 =  80.556
------------------------------------
Epoch:  13 | Train Loss: 0.001 | Train Acc: 97.860 | Val Loss: 0.003 | Val Acc: 87.361
Training:
Accuracies by groups:
0, 0  acc: 14768 / 17113 =  86.297
0, 1  acc:  8028 /  8471 =  94.770
1, 0  acc: 128280 / 128816 =  99.584
1, 1  acc:  8210 /  8370 =  98.088
--------------------------------------
Average acc: 159286 / 162770 =  97.860
Robust  acc: 14768 / 17113 =  86.297
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6706 /  8535 =  78.571
0, 1  acc:  7675 /  8276 =  92.738
1, 0  acc:  2815 /  2874 =  97.947
1, 1  acc:   160 /   182 =  87.912
------------------------------------
Average acc: 17356 / 19867 =  87.361
Robust  acc:  6706 /  8535 =  78.571
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 87.662
Robust Acc: 78.889 | Best Acc: 97.621
-------------------------------------
Training, Epoch 12:
Accuracies by groups:
0, 0  acc:  8007 /  9767 =  81.980
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2421 /  2480 =  97.621
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 17499 / 19962 =  87.662
Robust  acc:   142 /   180 =  78.889
------------------------------------
Accuracies by groups:
0, 0  acc:  8007 /  9767 =  81.980
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2421 /  2480 =  97.621
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 17499 / 19962 =  87.662
Robust  acc:   142 /   180 =  78.889
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8007 /  9767 =  81.980
0, 1  acc:  6929 /  7535 =  91.958
1, 0  acc:  2421 /  2480 =  97.621
1, 1  acc:   142 /   180 =  78.889
------------------------------------
Average acc: 17499 / 19962 =  87.662
Robust  acc:   142 /   180 =  78.889
------------------------------------
Epoch:  14 | Train Loss: 0.001 | Train Acc: 97.291 | Val Loss: 0.003 | Val Acc: 86.138
Training:
Accuracies by groups:
0, 0  acc: 14332 / 17193 =  83.360
0, 1  acc:  8014 /  8557 =  93.654
1, 0  acc: 127887 / 128645 =  99.411
1, 1  acc:  8127 /  8375 =  97.039
--------------------------------------
Average acc: 158360 / 162770 =  97.291
Robust  acc: 14332 / 17193 =  83.360
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6782 /  8535 =  79.461
0, 1  acc:  7316 /  8276 =  88.400
1, 0  acc:  2840 /  2874 =  98.817
1, 1  acc:   175 /   182 =  96.154
------------------------------------
Average acc: 17113 / 19867 =  86.138
Robust  acc:  6782 /  8535 =  79.461
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 87.471
Robust Acc: 83.690 | Best Acc: 98.347
-------------------------------------
Training, Epoch 13:
Accuracies by groups:
0, 0  acc:  8174 /  9767 =  83.690
0, 1  acc:  6682 /  7535 =  88.679
1, 0  acc:  2439 /  2480 =  98.347
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17461 / 19962 =  87.471
Robust  acc:  8174 /  9767 =  83.690
------------------------------------
Accuracies by groups:
0, 0  acc:  8174 /  9767 =  83.690
0, 1  acc:  6682 /  7535 =  88.679
1, 0  acc:  2439 /  2480 =  98.347
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17461 / 19962 =  87.471
Robust  acc:  8174 /  9767 =  83.690
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8174 /  9767 =  83.690
0, 1  acc:  6682 /  7535 =  88.679
1, 0  acc:  2439 /  2480 =  98.347
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17461 / 19962 =  87.471
Robust  acc:  8174 /  9767 =  83.690
------------------------------------
Epoch:  15 | Train Loss: 0.001 | Train Acc: 96.871 | Val Loss: 0.004 | Val Acc: 77.380
Training:
Accuracies by groups:
0, 0  acc: 13880 / 17099 =  81.174
0, 1  acc:  7848 /  8571 =  91.565
1, 0  acc: 128082 / 128921 =  99.349
1, 1  acc:  7867 /  8179 =  96.185
--------------------------------------
Average acc: 157677 / 162770 =  96.871
Robust  acc: 13880 / 17099 =  81.174
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5844 /  8535 =  68.471
0, 1  acc:  6492 /  8276 =  78.444
1, 0  acc:  2856 /  2874 =  99.374
1, 1  acc:   181 /   182 =  99.451
------------------------------------
Average acc: 15373 / 19867 =  77.380
Robust  acc:  5844 /  8535 =  68.471
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 78.444
Robust Acc: 73.206 | Best Acc: 99.315
-------------------------------------
Training, Epoch 14:
Accuracies by groups:
0, 0  acc:  7150 /  9767 =  73.206
0, 1  acc:  5872 /  7535 =  77.930
1, 0  acc:  2463 /  2480 =  99.315
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15659 / 19962 =  78.444
Robust  acc:  7150 /  9767 =  73.206
------------------------------------
Accuracies by groups:
0, 0  acc:  7150 /  9767 =  73.206
0, 1  acc:  5872 /  7535 =  77.930
1, 0  acc:  2463 /  2480 =  99.315
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15659 / 19962 =  78.444
Robust  acc:  7150 /  9767 =  73.206
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7150 /  9767 =  73.206
0, 1  acc:  5872 /  7535 =  77.930
1, 0  acc:  2463 /  2480 =  99.315
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15659 / 19962 =  78.444
Robust  acc:  7150 /  9767 =  73.206
------------------------------------
Epoch:  16 | Train Loss: 0.001 | Train Acc: 96.349 | Val Loss: 0.002 | Val Acc: 93.693
Training:
Accuracies by groups:
0, 0  acc: 13537 / 17241 =  78.516
0, 1  acc:  7681 /  8508 =  90.280
1, 0  acc: 127831 / 128899 =  99.171
1, 1  acc:  7778 /  8122 =  95.765
--------------------------------------
Average acc: 156827 / 162770 =  96.349
Robust  acc: 13537 / 17241 =  78.516
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7754 /  8535 =  90.849
0, 1  acc:  8034 /  8276 =  97.076
1, 0  acc:  2687 /  2874 =  93.493
1, 1  acc:   139 /   182 =  76.374
------------------------------------
Average acc: 18614 / 19867 =  93.693
Robust  acc:   139 /   182 =  76.374
------------------------------------
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 93.873
Robust Acc: 71.667 | Best Acc: 96.616
-------------------------------------
Training, Epoch 15:
Accuracies by groups:
0, 0  acc:  9044 /  9767 =  92.598
0, 1  acc:  7280 /  7535 =  96.616
1, 0  acc:  2286 /  2480 =  92.177
1, 1  acc:   129 /   180 =  71.667
------------------------------------
Average acc: 18739 / 19962 =  93.873
Robust  acc:   129 /   180 =  71.667
------------------------------------
Accuracies by groups:
0, 0  acc:  9044 /  9767 =  92.598
0, 1  acc:  7280 /  7535 =  96.616
1, 0  acc:  2286 /  2480 =  92.177
1, 1  acc:   129 /   180 =  71.667
------------------------------------
Average acc: 18739 / 19962 =  93.873
Robust  acc:   129 /   180 =  71.667
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9044 /  9767 =  92.598
0, 1  acc:  7280 /  7535 =  96.616
1, 0  acc:  2286 /  2480 =  92.177
1, 1  acc:   129 /   180 =  71.667
------------------------------------
Average acc: 18739 / 19962 =  93.873
Robust  acc:   129 /   180 =  71.667
------------------------------------
Epoch:  17 | Train Loss: 0.001 | Train Acc: 95.935 | Val Loss: 0.001 | Val Acc: 94.800
Training:
Accuracies by groups:
0, 0  acc: 13161 / 17183 =  76.593
0, 1  acc:  7672 /  8617 =  89.033
1, 0  acc: 127283 / 128492 =  99.059
1, 1  acc:  8038 /  8478 =  94.810
--------------------------------------
Average acc: 156154 / 162770 =  95.935
Robust  acc: 13161 / 17183 =  76.593
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  8151 /  8535 =  95.501
0, 1  acc:  8194 /  8276 =  99.009
1, 0  acc:  2384 /  2874 =  82.951
1, 1  acc:   105 /   182 =  57.692
------------------------------------
Average acc: 18834 / 19867 =  94.800
Robust  acc:   105 /   182 =  57.692
------------------------------------
-------------------------------------------
Avg Test Loss: 0.001 | Avg Test Acc: 94.870
Robust Acc: 50.000 | Best Acc: 98.673
-------------------------------------
Training, Epoch 16:
Accuracies by groups:
0, 0  acc:  9410 /  9767 =  96.345
0, 1  acc:  7435 /  7535 =  98.673
1, 0  acc:  2003 /  2480 =  80.766
1, 1  acc:    90 /   180 =  50.000
------------------------------------
Average acc: 18938 / 19962 =  94.870
Robust  acc:    90 /   180 =  50.000
------------------------------------
Accuracies by groups:
0, 0  acc:  9410 /  9767 =  96.345
0, 1  acc:  7435 /  7535 =  98.673
1, 0  acc:  2003 /  2480 =  80.766
1, 1  acc:    90 /   180 =  50.000
------------------------------------
Average acc: 18938 / 19962 =  94.870
Robust  acc:    90 /   180 =  50.000
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9410 /  9767 =  96.345
0, 1  acc:  7435 /  7535 =  98.673
1, 0  acc:  2003 /  2480 =  80.766
1, 1  acc:    90 /   180 =  50.000
------------------------------------
Average acc: 18938 / 19962 =  94.870
Robust  acc:    90 /   180 =  50.000
------------------------------------
Epoch:  18 | Train Loss: 0.001 | Train Acc: 95.555 | Val Loss: 0.002 | Val Acc: 91.232
Training:
Accuracies by groups:
0, 0  acc: 12785 / 17157 =  74.518
0, 1  acc:  7466 /  8479 =  88.053
1, 0  acc: 127305 / 128699 =  98.917
1, 1  acc:  7979 /  8435 =  94.594
--------------------------------------
Average acc: 155535 / 162770 =  95.555
Robust  acc: 12785 / 17157 =  74.518
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7350 /  8535 =  86.116
0, 1  acc:  7830 /  8276 =  94.611
1, 0  acc:  2786 /  2874 =  96.938
1, 1  acc:   159 /   182 =  87.363
------------------------------------
Average acc: 18125 / 19867 =  91.232
Robust  acc:  7350 /  8535 =  86.116
------------------------------------
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 92.100
Robust Acc: 79.444 | Best Acc: 96.653
-------------------------------------
Training, Epoch 17:
Accuracies by groups:
0, 0  acc:  8673 /  9767 =  88.799
0, 1  acc:  7172 /  7535 =  95.182
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18385 / 19962 =  92.100
Robust  acc:   143 /   180 =  79.444
------------------------------------
Accuracies by groups:
0, 0  acc:  8673 /  9767 =  88.799
0, 1  acc:  7172 /  7535 =  95.182
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18385 / 19962 =  92.100
Robust  acc:   143 /   180 =  79.444
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8673 /  9767 =  88.799
0, 1  acc:  7172 /  7535 =  95.182
1, 0  acc:  2397 /  2480 =  96.653
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18385 / 19962 =  92.100
Robust  acc:   143 /   180 =  79.444
------------------------------------
Epoch:  19 | Train Loss: 0.001 | Train Acc: 95.198 | Val Loss: 0.003 | Val Acc: 89.319
Training:
Accuracies by groups:
0, 0  acc: 12495 / 17075 =  73.177
0, 1  acc:  7158 /  8317 =  86.065
1, 0  acc: 127545 / 129083 =  98.809
1, 1  acc:  7756 /  8295 =  93.502
--------------------------------------
Average acc: 154954 / 162770 =  95.198
Robust  acc: 12495 / 17075 =  73.177
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6995 /  8535 =  81.957
0, 1  acc:  7778 /  8276 =  93.983
1, 0  acc:  2812 /  2874 =  97.843
1, 1  acc:   160 /   182 =  87.912
------------------------------------
Average acc: 17745 / 19867 =  89.319
Robust  acc:  6995 /  8535 =  81.957
------------------------------------
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 90.116
Robust Acc: 83.889 | Best Acc: 97.500
-------------------------------------
Training, Epoch 18:
Accuracies by groups:
0, 0  acc:  8333 /  9767 =  85.318
0, 1  acc:  7087 /  7535 =  94.054
1, 0  acc:  2418 /  2480 =  97.500
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   151 /   180 =  83.889
------------------------------------
Accuracies by groups:
0, 0  acc:  8333 /  9767 =  85.318
0, 1  acc:  7087 /  7535 =  94.054
1, 0  acc:  2418 /  2480 =  97.500
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   151 /   180 =  83.889
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8333 /  9767 =  85.318
0, 1  acc:  7087 /  7535 =  94.054
1, 0  acc:  2418 /  2480 =  97.500
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   151 /   180 =  83.889
------------------------------------
Epoch:  20 | Train Loss: 0.001 | Train Acc: 94.833 | Val Loss: 0.003 | Val Acc: 87.522
Training:
Accuracies by groups:
0, 0  acc: 12272 / 17187 =  71.403
0, 1  acc:  7237 /  8546 =  84.683
1, 0  acc: 127046 / 128700 =  98.715
1, 1  acc:  7804 /  8337 =  93.607
--------------------------------------
Average acc: 154359 / 162770 =  94.833
Robust  acc: 12272 / 17187 =  71.403
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6923 /  8535 =  81.113
0, 1  acc:  7479 /  8276 =  90.370
1, 0  acc:  2815 /  2874 =  97.947
1, 1  acc:   171 /   182 =  93.956
------------------------------------
Average acc: 17388 / 19867 =  87.522
Robust  acc:  6923 /  8535 =  81.113
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 89.074
Robust Acc: 85.492 | Best Acc: 97.823
-------------------------------------
Training, Epoch 19:
Accuracies by groups:
0, 0  acc:  8350 /  9767 =  85.492
0, 1  acc:  6849 /  7535 =  90.896
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   156 /   180 =  86.667
------------------------------------
Average acc: 17781 / 19962 =  89.074
Robust  acc:  8350 /  9767 =  85.492
------------------------------------
Accuracies by groups:
0, 0  acc:  8350 /  9767 =  85.492
0, 1  acc:  6849 /  7535 =  90.896
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   156 /   180 =  86.667
------------------------------------
Average acc: 17781 / 19962 =  89.074
Robust  acc:  8350 /  9767 =  85.492
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8350 /  9767 =  85.492
0, 1  acc:  6849 /  7535 =  90.896
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   156 /   180 =  86.667
------------------------------------
Average acc: 17781 / 19962 =  89.074
Robust  acc:  8350 /  9767 =  85.492
------------------------------------
Epoch:  21 | Train Loss: 0.002 | Train Acc: 94.763 | Val Loss: 0.002 | Val Acc: 93.019
Training:
Accuracies by groups:
0, 0  acc: 12150 / 17126 =  70.945
0, 1  acc:  6955 /  8332 =  83.473
1, 0  acc: 127329 / 128962 =  98.734
1, 1  acc:  7812 /  8350 =  93.557
--------------------------------------
Average acc: 154246 / 162770 =  94.763
Robust  acc: 12150 / 17126 =  70.945
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7671 /  8535 =  89.877
0, 1  acc:  7954 /  8276 =  96.109
1, 0  acc:  2717 /  2874 =  94.537
1, 1  acc:   138 /   182 =  75.824
------------------------------------
Average acc: 18480 / 19867 =  93.019
Robust  acc:   138 /   182 =  75.824
------------------------------------
-------------------------------------------
Avg Test Loss: 0.002 | Avg Test Acc: 93.673
Robust Acc: 75.000 | Best Acc: 95.926
-------------------------------------
Training, Epoch 20:
Accuracies by groups:
0, 0  acc:  9016 /  9767 =  92.311
0, 1  acc:  7228 /  7535 =  95.926
1, 0  acc:  2320 /  2480 =  93.548
1, 1  acc:   135 /   180 =  75.000
------------------------------------
Average acc: 18699 / 19962 =  93.673
Robust  acc:   135 /   180 =  75.000
------------------------------------
Accuracies by groups:
0, 0  acc:  9016 /  9767 =  92.311
0, 1  acc:  7228 /  7535 =  95.926
1, 0  acc:  2320 /  2480 =  93.548
1, 1  acc:   135 /   180 =  75.000
------------------------------------
Average acc: 18699 / 19962 =  93.673
Robust  acc:   135 /   180 =  75.000
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9016 /  9767 =  92.311
0, 1  acc:  7228 /  7535 =  95.926
1, 0  acc:  2320 /  2480 =  93.548
1, 1  acc:   135 /   180 =  75.000
------------------------------------
Average acc: 18699 / 19962 =  93.673
Robust  acc:   135 /   180 =  75.000
------------------------------------
Epoch:  22 | Train Loss: 0.002 | Train Acc: 94.396 | Val Loss: 0.004 | Val Acc: 78.713
Training:
Accuracies by groups:
0, 0  acc: 11813 / 17034 =  69.350
0, 1  acc:  6965 /  8564 =  81.329
1, 0  acc: 127022 / 128725 =  98.677
1, 1  acc:  7848 /  8447 =  92.909
--------------------------------------
Average acc: 153648 / 162770 =  94.396
Robust  acc: 11813 / 17034 =  69.350
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5959 /  8535 =  69.818
0, 1  acc:  6646 /  8276 =  80.304
1, 0  acc:  2856 /  2874 =  99.374
1, 1  acc:   177 /   182 =  97.253
------------------------------------
Average acc: 15638 / 19867 =  78.713
Robust  acc:  5959 /  8535 =  69.818
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 80.628
Robust Acc: 75.131 | Best Acc: 99.556
-------------------------------------
Training, Epoch 21:
Accuracies by groups:
0, 0  acc:  7338 /  9767 =  75.131
0, 1  acc:  6116 /  7535 =  81.168
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 16095 / 19962 =  80.628
Robust  acc:  7338 /  9767 =  75.131
------------------------------------
Accuracies by groups:
0, 0  acc:  7338 /  9767 =  75.131
0, 1  acc:  6116 /  7535 =  81.168
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 16095 / 19962 =  80.628
Robust  acc:  7338 /  9767 =  75.131
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7338 /  9767 =  75.131
0, 1  acc:  6116 /  7535 =  81.168
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 16095 / 19962 =  80.628
Robust  acc:  7338 /  9767 =  75.131
------------------------------------
Epoch:  23 | Train Loss: 0.002 | Train Acc: 94.302 | Val Loss: 0.003 | Val Acc: 87.985
Training:
Accuracies by groups:
0, 0  acc: 11977 / 17345 =  69.052
0, 1  acc:  6883 /  8576 =  80.259
1, 0  acc: 126871 / 128545 =  98.698
1, 1  acc:  7765 /  8304 =  93.509
--------------------------------------
Average acc: 153496 / 162770 =  94.302
Robust  acc: 11977 / 17345 =  69.052
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7038 /  8535 =  82.460
0, 1  acc:  7473 /  8276 =  90.297
1, 0  acc:  2797 /  2874 =  97.321
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 17480 / 19867 =  87.985
Robust  acc:  7038 /  8535 =  82.460
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 89.380
Robust Acc: 86.721 | Best Acc: 96.492
-------------------------------------
Training, Epoch 22:
Accuracies by groups:
0, 0  acc:  8470 /  9767 =  86.721
0, 1  acc:  6822 /  7535 =  90.537
1, 0  acc:  2393 /  2480 =  96.492
1, 1  acc:   157 /   180 =  87.222
------------------------------------
Average acc: 17842 / 19962 =  89.380
Robust  acc:  8470 /  9767 =  86.721
------------------------------------
Accuracies by groups:
0, 0  acc:  8470 /  9767 =  86.721
0, 1  acc:  6822 /  7535 =  90.537
1, 0  acc:  2393 /  2480 =  96.492
1, 1  acc:   157 /   180 =  87.222
------------------------------------
Average acc: 17842 / 19962 =  89.380
Robust  acc:  8470 /  9767 =  86.721
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8470 /  9767 =  86.721
0, 1  acc:  6822 /  7535 =  90.537
1, 0  acc:  2393 /  2480 =  96.492
1, 1  acc:   157 /   180 =  87.222
------------------------------------
Average acc: 17842 / 19962 =  89.380
Robust  acc:  8470 /  9767 =  86.721
------------------------------------
Epoch:  24 | Train Loss: 0.002 | Train Acc: 94.277 | Val Loss: 0.003 | Val Acc: 86.072
Training:
Accuracies by groups:
0, 0  acc: 11606 / 17138 =  67.721
0, 1  acc:  6911 /  8619 =  80.183
1, 0  acc: 126966 / 128487 =  98.816
1, 1  acc:  7971 /  8526 =  93.490
--------------------------------------
Average acc: 153454 / 162770 =  94.277
Robust  acc: 11606 / 17138 =  67.721
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6622 /  8535 =  77.586
0, 1  acc:  7477 /  8276 =  90.346
1, 0  acc:  2831 /  2874 =  98.504
1, 1  acc:   170 /   182 =  93.407
------------------------------------
Average acc: 17100 / 19867 =  86.072
Robust  acc:  6622 /  8535 =  77.586
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 87.095
Robust Acc: 81.796 | Best Acc: 97.823
-------------------------------------
Training, Epoch 23:
Accuracies by groups:
0, 0  acc:  7989 /  9767 =  81.796
0, 1  acc:  6810 /  7535 =  90.378
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17386 / 19962 =  87.095
Robust  acc:  7989 /  9767 =  81.796
------------------------------------
Accuracies by groups:
0, 0  acc:  7989 /  9767 =  81.796
0, 1  acc:  6810 /  7535 =  90.378
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17386 / 19962 =  87.095
Robust  acc:  7989 /  9767 =  81.796
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7989 /  9767 =  81.796
0, 1  acc:  6810 /  7535 =  90.378
1, 0  acc:  2426 /  2480 =  97.823
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17386 / 19962 =  87.095
Robust  acc:  7989 /  9767 =  81.796
------------------------------------
Epoch:  25 | Train Loss: 0.002 | Train Acc: 94.267 | Val Loss: 0.003 | Val Acc: 87.487
Training:
Accuracies by groups:
0, 0  acc: 11635 / 17149 =  67.847
0, 1  acc:  6581 /  8306 =  79.232
1, 0  acc: 127457 / 128993 =  98.809
1, 1  acc:  7765 /  8322 =  93.307
--------------------------------------
Average acc: 153438 / 162770 =  94.267
Robust  acc: 11635 / 17149 =  67.847
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6783 /  8535 =  79.473
0, 1  acc:  7621 /  8276 =  92.086
1, 0  acc:  2813 /  2874 =  97.878
1, 1  acc:   164 /   182 =  90.110
------------------------------------
Average acc: 17381 / 19867 =  87.487
Robust  acc:  6783 /  8535 =  79.473
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 88.278
Robust Acc: 83.352 | Best Acc: 96.734
-------------------------------------
Training, Epoch 24:
Accuracies by groups:
0, 0  acc:  8141 /  9767 =  83.352
0, 1  acc:  6930 /  7535 =  91.971
1, 0  acc:  2399 /  2480 =  96.734
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 17622 / 19962 =  88.278
Robust  acc:  8141 /  9767 =  83.352
------------------------------------
Accuracies by groups:
0, 0  acc:  8141 /  9767 =  83.352
0, 1  acc:  6930 /  7535 =  91.971
1, 0  acc:  2399 /  2480 =  96.734
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 17622 / 19962 =  88.278
Robust  acc:  8141 /  9767 =  83.352
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8141 /  9767 =  83.352
0, 1  acc:  6930 /  7535 =  91.971
1, 0  acc:  2399 /  2480 =  96.734
1, 1  acc:   152 /   180 =  84.444
------------------------------------
Average acc: 17622 / 19962 =  88.278
Robust  acc:  8141 /  9767 =  83.352
------------------------------------
Epoch:  26 | Train Loss: 0.002 | Train Acc: 94.143 | Val Loss: 0.003 | Val Acc: 91.544
Training:
Accuracies by groups:
0, 0  acc: 11349 / 17043 =  66.590
0, 1  acc:  6725 /  8584 =  78.343
1, 0  acc: 127411 / 128931 =  98.821
1, 1  acc:  7752 /  8212 =  94.398
--------------------------------------
Average acc: 153237 / 162770 =  94.143
Robust  acc: 11349 / 17043 =  66.590
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7713 /  8535 =  90.369
0, 1  acc:  7808 /  8276 =  94.345
1, 0  acc:  2548 /  2874 =  88.657
1, 1  acc:   118 /   182 =  64.835
------------------------------------
Average acc: 18187 / 19867 =  91.544
Robust  acc:   118 /   182 =  64.835
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 92.365
Robust Acc: 61.111 | Best Acc: 94.413
-------------------------------------
Training, Epoch 25:
Accuracies by groups:
0, 0  acc:  9037 /  9767 =  92.526
0, 1  acc:  7114 /  7535 =  94.413
1, 0  acc:  2177 /  2480 =  87.782
1, 1  acc:   110 /   180 =  61.111
------------------------------------
Average acc: 18438 / 19962 =  92.365
Robust  acc:   110 /   180 =  61.111
------------------------------------
Accuracies by groups:
0, 0  acc:  9037 /  9767 =  92.526
0, 1  acc:  7114 /  7535 =  94.413
1, 0  acc:  2177 /  2480 =  87.782
1, 1  acc:   110 /   180 =  61.111
------------------------------------
Average acc: 18438 / 19962 =  92.365
Robust  acc:   110 /   180 =  61.111
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9037 /  9767 =  92.526
0, 1  acc:  7114 /  7535 =  94.413
1, 0  acc:  2177 /  2480 =  87.782
1, 1  acc:   110 /   180 =  61.111
------------------------------------
Average acc: 18438 / 19962 =  92.365
Robust  acc:   110 /   180 =  61.111
------------------------------------
Epoch:  27 | Train Loss: 0.002 | Train Acc: 94.076 | Val Loss: 0.004 | Val Acc: 81.874
Training:
Accuracies by groups:
0, 0  acc: 11399 / 17264 =  66.028
0, 1  acc:  6678 /  8615 =  77.516
1, 0  acc: 127156 / 128527 =  98.933
1, 1  acc:  7894 /  8364 =  94.381
--------------------------------------
Average acc: 153127 / 162770 =  94.076
Robust  acc: 11399 / 17264 =  66.028
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6355 /  8535 =  74.458
0, 1  acc:  6894 /  8276 =  83.301
1, 0  acc:  2842 /  2874 =  98.887
1, 1  acc:   175 /   182 =  96.154
------------------------------------
Average acc: 16266 / 19867 =  81.874
Robust  acc:  6355 /  8535 =  74.458
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 83.444
Robust Acc: 80.004 | Best Acc: 98.589
-------------------------------------
Training, Epoch 26:
Accuracies by groups:
0, 0  acc:  7814 /  9767 =  80.004
0, 1  acc:  6228 /  7535 =  82.654
1, 0  acc:  2445 /  2480 =  98.589
1, 1  acc:   170 /   180 =  94.444
------------------------------------
Average acc: 16657 / 19962 =  83.444
Robust  acc:  7814 /  9767 =  80.004
------------------------------------
Accuracies by groups:
0, 0  acc:  7814 /  9767 =  80.004
0, 1  acc:  6228 /  7535 =  82.654
1, 0  acc:  2445 /  2480 =  98.589
1, 1  acc:   170 /   180 =  94.444
------------------------------------
Average acc: 16657 / 19962 =  83.444
Robust  acc:  7814 /  9767 =  80.004
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7814 /  9767 =  80.004
0, 1  acc:  6228 /  7535 =  82.654
1, 0  acc:  2445 /  2480 =  98.589
1, 1  acc:   170 /   180 =  94.444
------------------------------------
Average acc: 16657 / 19962 =  83.444
Robust  acc:  7814 /  9767 =  80.004
------------------------------------
Epoch:  28 | Train Loss: 0.002 | Train Acc: 93.936 | Val Loss: 0.006 | Val Acc: 68.586
Training:
Accuracies by groups:
0, 0  acc: 11242 / 17217 =  65.296
0, 1  acc:  6597 /  8645 =  76.310
1, 0  acc: 127305 / 128688 =  98.925
1, 1  acc:  7756 /  8220 =  94.355
--------------------------------------
Average acc: 152900 / 162770 =  93.936
Robust  acc: 11242 / 17217 =  65.296
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5223 /  8535 =  61.195
0, 1  acc:  5358 /  8276 =  64.741
1, 0  acc:  2864 /  2874 =  99.652
1, 1  acc:   181 /   182 =  99.451
------------------------------------
Average acc: 13626 / 19867 =  68.586
Robust  acc:  5223 /  8535 =  61.195
------------------------------------
-------------------------------------------
Avg Test Loss: 0.006 | Avg Test Acc: 70.594
Robust Acc: 65.481 | Best Acc: 99.556
-------------------------------------
Training, Epoch 27:
Accuracies by groups:
0, 0  acc:  6512 /  9767 =  66.673
0, 1  acc:  4934 /  7535 =  65.481
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   177 /   180 =  98.333
------------------------------------
Average acc: 14092 / 19962 =  70.594
Robust  acc:  4934 /  7535 =  65.481
------------------------------------
Accuracies by groups:
0, 0  acc:  6512 /  9767 =  66.673
0, 1  acc:  4934 /  7535 =  65.481
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   177 /   180 =  98.333
------------------------------------
Average acc: 14092 / 19962 =  70.594
Robust  acc:  4934 /  7535 =  65.481
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  6512 /  9767 =  66.673
0, 1  acc:  4934 /  7535 =  65.481
1, 0  acc:  2469 /  2480 =  99.556
1, 1  acc:   177 /   180 =  98.333
------------------------------------
Average acc: 14092 / 19962 =  70.594
Robust  acc:  4934 /  7535 =  65.481
------------------------------------
Epoch:  29 | Train Loss: 0.002 | Train Acc: 93.840 | Val Loss: 0.003 | Val Acc: 87.879
Training:
Accuracies by groups:
0, 0  acc: 11089 / 17203 =  64.460
0, 1  acc:  6635 /  8764 =  75.707
1, 0  acc: 127181 / 128504 =  98.970
1, 1  acc:  7838 /  8299 =  94.445
--------------------------------------
Average acc: 152743 / 162770 =  93.840
Robust  acc: 11089 / 17203 =  64.460
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6886 /  8535 =  80.680
0, 1  acc:  7600 /  8276 =  91.832
1, 0  acc:  2810 /  2874 =  97.773
1, 1  acc:   163 /   182 =  89.560
------------------------------------
Average acc: 17459 / 19867 =  87.879
Robust  acc:  6886 /  8535 =  80.680
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 89.054
Robust Acc: 83.889 | Best Acc: 97.339
-------------------------------------
Training, Epoch 28:
Accuracies by groups:
0, 0  acc:  8301 /  9767 =  84.990
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2414 /  2480 =  97.339
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17777 / 19962 =  89.054
Robust  acc:   151 /   180 =  83.889
------------------------------------
Accuracies by groups:
0, 0  acc:  8301 /  9767 =  84.990
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2414 /  2480 =  97.339
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17777 / 19962 =  89.054
Robust  acc:   151 /   180 =  83.889
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8301 /  9767 =  84.990
0, 1  acc:  6911 /  7535 =  91.719
1, 0  acc:  2414 /  2480 =  97.339
1, 1  acc:   151 /   180 =  83.889
------------------------------------
Average acc: 17777 / 19962 =  89.054
Robust  acc:   151 /   180 =  83.889
------------------------------------
Epoch:  30 | Train Loss: 0.002 | Train Acc: 94.010 | Val Loss: 0.005 | Val Acc: 77.264
Training:
Accuracies by groups:
0, 0  acc: 11231 / 17228 =  65.190
0, 1  acc:  6480 /  8508 =  76.164
1, 0  acc: 127607 / 128879 =  99.013
1, 1  acc:  7702 /  8155 =  94.445
--------------------------------------
Average acc: 153020 / 162770 =  94.010
Robust  acc: 11231 / 17228 =  65.190
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5764 /  8535 =  67.534
0, 1  acc:  6556 /  8276 =  79.217
1, 0  acc:  2851 /  2874 =  99.200
1, 1  acc:   179 /   182 =  98.352
------------------------------------
Average acc: 15350 / 19867 =  77.264
Robust  acc:  5764 /  8535 =  67.534
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 79.100
Robust Acc: 72.980 | Best Acc: 99.194
-------------------------------------
Training, Epoch 29:
Accuracies by groups:
0, 0  acc:  7128 /  9767 =  72.980
0, 1  acc:  6030 /  7535 =  80.027
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 15790 / 19962 =  79.100
Robust  acc:  7128 /  9767 =  72.980
------------------------------------
Accuracies by groups:
0, 0  acc:  7128 /  9767 =  72.980
0, 1  acc:  6030 /  7535 =  80.027
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 15790 / 19962 =  79.100
Robust  acc:  7128 /  9767 =  72.980
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7128 /  9767 =  72.980
0, 1  acc:  6030 /  7535 =  80.027
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   172 /   180 =  95.556
------------------------------------
Average acc: 15790 / 19962 =  79.100
Robust  acc:  7128 /  9767 =  72.980
------------------------------------
Epoch:  31 | Train Loss: 0.002 | Train Acc: 93.982 | Val Loss: 0.004 | Val Acc: 81.376
Training:
Accuracies by groups:
0, 0  acc: 10965 / 17085 =  64.179
0, 1  acc:  6366 /  8422 =  75.588
1, 0  acc: 127629 / 128854 =  99.049
1, 1  acc:  8014 /  8409 =  95.303
--------------------------------------
Average acc: 152974 / 162770 =  93.982
Robust  acc: 10965 / 17085 =  64.179
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6189 /  8535 =  72.513
0, 1  acc:  6959 /  8276 =  84.087
1, 0  acc:  2842 /  2874 =  98.887
1, 1  acc:   177 /   182 =  97.253
------------------------------------
Average acc: 16167 / 19867 =  81.376
Robust  acc:  6189 /  8535 =  72.513
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 82.677
Robust Acc: 77.373 | Best Acc: 98.185
-------------------------------------
Training, Epoch 30:
Accuracies by groups:
0, 0  acc:  7557 /  9767 =  77.373
0, 1  acc:  6349 /  7535 =  84.260
1, 0  acc:  2435 /  2480 =  98.185
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 16504 / 19962 =  82.677
Robust  acc:  7557 /  9767 =  77.373
------------------------------------
Accuracies by groups:
0, 0  acc:  7557 /  9767 =  77.373
0, 1  acc:  6349 /  7535 =  84.260
1, 0  acc:  2435 /  2480 =  98.185
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 16504 / 19962 =  82.677
Robust  acc:  7557 /  9767 =  77.373
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7557 /  9767 =  77.373
0, 1  acc:  6349 /  7535 =  84.260
1, 0  acc:  2435 /  2480 =  98.185
1, 1  acc:   163 /   180 =  90.556
------------------------------------
Average acc: 16504 / 19962 =  82.677
Robust  acc:  7557 /  9767 =  77.373
------------------------------------
Epoch:  32 | Train Loss: 0.002 | Train Acc: 94.015 | Val Loss: 0.005 | Val Acc: 77.948
Training:
Accuracies by groups:
0, 0  acc: 11185 / 17256 =  64.818
0, 1  acc:  6594 /  8656 =  76.178
1, 0  acc: 127422 / 128631 =  99.060
1, 1  acc:  7827 /  8227 =  95.138
--------------------------------------
Average acc: 153028 / 162770 =  94.015
Robust  acc: 11185 / 17256 =  64.818
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5725 /  8535 =  67.077
0, 1  acc:  6726 /  8276 =  81.271
1, 0  acc:  2857 /  2874 =  99.408
1, 1  acc:   178 /   182 =  97.802
------------------------------------
Average acc: 15486 / 19867 =  77.948
Robust  acc:  5725 /  8535 =  67.077
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 79.721
Robust Acc: 73.154 | Best Acc: 99.355
-------------------------------------
Training, Epoch 31:
Accuracies by groups:
0, 0  acc:  7145 /  9767 =  73.154
0, 1  acc:  6134 /  7535 =  81.407
1, 0  acc:  2464 /  2480 =  99.355
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15914 / 19962 =  79.721
Robust  acc:  7145 /  9767 =  73.154
------------------------------------
Accuracies by groups:
0, 0  acc:  7145 /  9767 =  73.154
0, 1  acc:  6134 /  7535 =  81.407
1, 0  acc:  2464 /  2480 =  99.355
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15914 / 19962 =  79.721
Robust  acc:  7145 /  9767 =  73.154
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7145 /  9767 =  73.154
0, 1  acc:  6134 /  7535 =  81.407
1, 0  acc:  2464 /  2480 =  99.355
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15914 / 19962 =  79.721
Robust  acc:  7145 /  9767 =  73.154
------------------------------------
Epoch:  33 | Train Loss: 0.002 | Train Acc: 93.977 | Val Loss: 0.006 | Val Acc: 67.917
Training:
Accuracies by groups:
0, 0  acc: 11156 / 17228 =  64.755
0, 1  acc:  6509 /  8530 =  76.307
1, 0  acc: 127461 / 128771 =  98.983
1, 1  acc:  7840 /  8241 =  95.134
--------------------------------------
Average acc: 152966 / 162770 =  93.977
Robust  acc: 11156 / 17228 =  64.755
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  4543 /  8535 =  53.228
0, 1  acc:  5903 /  8276 =  71.327
1, 0  acc:  2866 /  2874 =  99.722
1, 1  acc:   181 /   182 =  99.451
------------------------------------
Average acc: 13493 / 19867 =  67.917
Robust  acc:  4543 /  8535 =  53.228
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 69.467
Robust Acc: 59.650 | Best Acc: 99.758
-------------------------------------
Training, Epoch 32:
Accuracies by groups:
0, 0  acc:  5826 /  9767 =  59.650
0, 1  acc:  5393 /  7535 =  71.573
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 13867 / 19962 =  69.467
Robust  acc:  5826 /  9767 =  59.650
------------------------------------
Accuracies by groups:
0, 0  acc:  5826 /  9767 =  59.650
0, 1  acc:  5393 /  7535 =  71.573
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 13867 / 19962 =  69.467
Robust  acc:  5826 /  9767 =  59.650
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  5826 /  9767 =  59.650
0, 1  acc:  5393 /  7535 =  71.573
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 13867 / 19962 =  69.467
Robust  acc:  5826 /  9767 =  59.650
------------------------------------
Epoch:  34 | Train Loss: 0.002 | Train Acc: 94.060 | Val Loss: 0.007 | Val Acc: 58.922
Training:
Accuracies by groups:
0, 0  acc: 11113 / 17169 =  64.727
0, 1  acc:  6506 /  8482 =  76.704
1, 0  acc: 127474 / 128723 =  99.030
1, 1  acc:  8009 /  8396 =  95.391
--------------------------------------
Average acc: 153102 / 162770 =  94.060
Robust  acc: 11113 / 17169 =  64.727
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  4148 /  8535 =  48.600
0, 1  acc:  4510 /  8276 =  54.495
1, 0  acc:  2868 /  2874 =  99.791
1, 1  acc:   180 /   182 =  98.901
------------------------------------
Average acc: 11706 / 19867 =  58.922
Robust  acc:  4148 /  8535 =  48.600
------------------------------------
-------------------------------------------
Avg Test Loss: 0.006 | Avg Test Acc: 59.859
Robust Acc: 53.457 | Best Acc: 99.758
-------------------------------------
Training, Epoch 33:
Accuracies by groups:
0, 0  acc:  5269 /  9767 =  53.947
0, 1  acc:  4028 /  7535 =  53.457
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   178 /   180 =  98.889
------------------------------------
Average acc: 11949 / 19962 =  59.859
Robust  acc:  4028 /  7535 =  53.457
------------------------------------
Accuracies by groups:
0, 0  acc:  5269 /  9767 =  53.947
0, 1  acc:  4028 /  7535 =  53.457
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   178 /   180 =  98.889
------------------------------------
Average acc: 11949 / 19962 =  59.859
Robust  acc:  4028 /  7535 =  53.457
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  5269 /  9767 =  53.947
0, 1  acc:  4028 /  7535 =  53.457
1, 0  acc:  2474 /  2480 =  99.758
1, 1  acc:   178 /   180 =  98.889
------------------------------------
Average acc: 11949 / 19962 =  59.859
Robust  acc:  4028 /  7535 =  53.457
------------------------------------
Epoch:  35 | Train Loss: 0.002 | Train Acc: 94.044 | Val Loss: 0.005 | Val Acc: 78.603
Training:
Accuracies by groups:
0, 0  acc: 11259 / 17240 =  65.307
0, 1  acc:  6498 /  8584 =  75.699
1, 0  acc: 127516 / 128718 =  99.066
1, 1  acc:  7802 /  8228 =  94.823
--------------------------------------
Average acc: 153075 / 162770 =  94.044
Robust  acc: 11259 / 17240 =  65.307
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5884 /  8535 =  68.940
0, 1  acc:  6703 /  8276 =  80.993
1, 0  acc:  2850 /  2874 =  99.165
1, 1  acc:   179 /   182 =  98.352
------------------------------------
Average acc: 15616 / 19867 =  78.603
Robust  acc:  5884 /  8535 =  68.940
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 80.438
Robust Acc: 74.711 | Best Acc: 98.911
-------------------------------------
Training, Epoch 34:
Accuracies by groups:
0, 0  acc:  7297 /  9767 =  74.711
0, 1  acc:  6147 /  7535 =  81.579
1, 0  acc:  2453 /  2480 =  98.911
1, 1  acc:   160 /   180 =  88.889
------------------------------------
Average acc: 16057 / 19962 =  80.438
Robust  acc:  7297 /  9767 =  74.711
------------------------------------
Accuracies by groups:
0, 0  acc:  7297 /  9767 =  74.711
0, 1  acc:  6147 /  7535 =  81.579
1, 0  acc:  2453 /  2480 =  98.911
1, 1  acc:   160 /   180 =  88.889
------------------------------------
Average acc: 16057 / 19962 =  80.438
Robust  acc:  7297 /  9767 =  74.711
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7297 /  9767 =  74.711
0, 1  acc:  6147 /  7535 =  81.579
1, 0  acc:  2453 /  2480 =  98.911
1, 1  acc:   160 /   180 =  88.889
------------------------------------
Average acc: 16057 / 19962 =  80.438
Robust  acc:  7297 /  9767 =  74.711
------------------------------------
Epoch:  36 | Train Loss: 0.002 | Train Acc: 94.114 | Val Loss: 0.003 | Val Acc: 92.807
Training:
Accuracies by groups:
0, 0  acc: 11163 / 17100 =  65.281
0, 1  acc:  6377 /  8361 =  76.271
1, 0  acc: 127748 / 128953 =  99.066
1, 1  acc:  7902 /  8356 =  94.567
--------------------------------------
Average acc: 153190 / 162770 =  94.114
Robust  acc: 11163 / 17100 =  65.281
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7742 /  8535 =  90.709
0, 1  acc:  7970 /  8276 =  96.303
1, 0  acc:  2596 /  2874 =  90.327
1, 1  acc:   130 /   182 =  71.429
------------------------------------
Average acc: 18438 / 19867 =  92.807
Robust  acc:   130 /   182 =  71.429
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 93.483
Robust Acc: 72.222 | Best Acc: 96.098
-------------------------------------
Training, Epoch 35:
Accuracies by groups:
0, 0  acc:  9080 /  9767 =  92.966
0, 1  acc:  7241 /  7535 =  96.098
1, 0  acc:  2210 /  2480 =  89.113
1, 1  acc:   130 /   180 =  72.222
------------------------------------
Average acc: 18661 / 19962 =  93.483
Robust  acc:   130 /   180 =  72.222
------------------------------------
Accuracies by groups:
0, 0  acc:  9080 /  9767 =  92.966
0, 1  acc:  7241 /  7535 =  96.098
1, 0  acc:  2210 /  2480 =  89.113
1, 1  acc:   130 /   180 =  72.222
------------------------------------
Average acc: 18661 / 19962 =  93.483
Robust  acc:   130 /   180 =  72.222
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  9080 /  9767 =  92.966
0, 1  acc:  7241 /  7535 =  96.098
1, 0  acc:  2210 /  2480 =  89.113
1, 1  acc:   130 /   180 =  72.222
------------------------------------
Average acc: 18661 / 19962 =  93.483
Robust  acc:   130 /   180 =  72.222
------------------------------------
Epoch:  37 | Train Loss: 0.002 | Train Acc: 94.130 | Val Loss: 0.005 | Val Acc: 77.219
Training:
Accuracies by groups:
0, 0  acc: 11227 / 17140 =  65.502
0, 1  acc:  6621 /  8582 =  77.150
1, 0  acc: 127526 / 128791 =  99.018
1, 1  acc:  7841 /  8257 =  94.962
--------------------------------------
Average acc: 153215 / 162770 =  94.130
Robust  acc: 11227 / 17140 =  65.502
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5895 /  8535 =  69.069
0, 1  acc:  6412 /  8276 =  77.477
1, 0  acc:  2855 /  2874 =  99.339
1, 1  acc:   179 /   182 =  98.352
------------------------------------
Average acc: 15341 / 19867 =  77.219
Robust  acc:  5895 /  8535 =  69.069
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 79.396
Robust Acc: 75.151 | Best Acc: 99.194
-------------------------------------
Training, Epoch 36:
Accuracies by groups:
0, 0  acc:  7340 /  9767 =  75.151
0, 1  acc:  5875 /  7535 =  77.969
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15849 / 19962 =  79.396
Robust  acc:  7340 /  9767 =  75.151
------------------------------------
Accuracies by groups:
0, 0  acc:  7340 /  9767 =  75.151
0, 1  acc:  5875 /  7535 =  77.969
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15849 / 19962 =  79.396
Robust  acc:  7340 /  9767 =  75.151
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7340 /  9767 =  75.151
0, 1  acc:  5875 /  7535 =  77.969
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15849 / 19962 =  79.396
Robust  acc:  7340 /  9767 =  75.151
------------------------------------
Epoch:  38 | Train Loss: 0.002 | Train Acc: 94.160 | Val Loss: 0.005 | Val Acc: 75.160
Training:
Accuracies by groups:
0, 0  acc: 11344 / 17208 =  65.923
0, 1  acc:  6543 /  8535 =  76.661
1, 0  acc: 127407 / 128652 =  99.032
1, 1  acc:  7971 /  8375 =  95.176
--------------------------------------
Average acc: 153265 / 162770 =  94.160
Robust  acc: 11344 / 17208 =  65.923
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5576 /  8535 =  65.331
0, 1  acc:  6326 /  8276 =  76.438
1, 0  acc:  2852 /  2874 =  99.235
1, 1  acc:   178 /   182 =  97.802
------------------------------------
Average acc: 14932 / 19867 =  75.160
Robust  acc:  5576 /  8535 =  65.331
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 76.971
Robust Acc: 71.547 | Best Acc: 99.194
-------------------------------------
Training, Epoch 37:
Accuracies by groups:
0, 0  acc:  6988 /  9767 =  71.547
0, 1  acc:  5743 /  7535 =  76.218
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15365 / 19962 =  76.971
Robust  acc:  6988 /  9767 =  71.547
------------------------------------
Accuracies by groups:
0, 0  acc:  6988 /  9767 =  71.547
0, 1  acc:  5743 /  7535 =  76.218
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15365 / 19962 =  76.971
Robust  acc:  6988 /  9767 =  71.547
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  6988 /  9767 =  71.547
0, 1  acc:  5743 /  7535 =  76.218
1, 0  acc:  2460 /  2480 =  99.194
1, 1  acc:   174 /   180 =  96.667
------------------------------------
Average acc: 15365 / 19962 =  76.971
Robust  acc:  6988 /  9767 =  71.547
------------------------------------
Epoch:  39 | Train Loss: 0.002 | Train Acc: 94.048 | Val Loss: 0.004 | Val Acc: 82.197
Training:
Accuracies by groups:
0, 0  acc: 11428 / 17368 =  65.799
0, 1  acc:  6595 /  8549 =  77.144
1, 0  acc: 127109 / 128419 =  98.980
1, 1  acc:  7950 /  8434 =  94.261
--------------------------------------
Average acc: 153082 / 162770 =  94.048
Robust  acc: 11428 / 17368 =  65.799
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6177 /  8535 =  72.373
0, 1  acc:  7135 /  8276 =  86.213
1, 0  acc:  2843 /  2874 =  98.921
1, 1  acc:   175 /   182 =  96.154
------------------------------------
Average acc: 16330 / 19867 =  82.197
Robust  acc:  6177 /  8535 =  72.373
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 84.100
Robust Acc: 78.325 | Best Acc: 98.669
-------------------------------------
Training, Epoch 38:
Accuracies by groups:
0, 0  acc:  7650 /  9767 =  78.325
0, 1  acc:  6526 /  7535 =  86.609
1, 0  acc:  2447 /  2480 =  98.669
1, 1  acc:   165 /   180 =  91.667
------------------------------------
Average acc: 16788 / 19962 =  84.100
Robust  acc:  7650 /  9767 =  78.325
------------------------------------
Accuracies by groups:
0, 0  acc:  7650 /  9767 =  78.325
0, 1  acc:  6526 /  7535 =  86.609
1, 0  acc:  2447 /  2480 =  98.669
1, 1  acc:   165 /   180 =  91.667
------------------------------------
Average acc: 16788 / 19962 =  84.100
Robust  acc:  7650 /  9767 =  78.325
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7650 /  9767 =  78.325
0, 1  acc:  6526 /  7535 =  86.609
1, 0  acc:  2447 /  2480 =  98.669
1, 1  acc:   165 /   180 =  91.667
------------------------------------
Average acc: 16788 / 19962 =  84.100
Robust  acc:  7650 /  9767 =  78.325
------------------------------------
Epoch:  40 | Train Loss: 0.002 | Train Acc: 94.149 | Val Loss: 0.005 | Val Acc: 74.294
Training:
Accuracies by groups:
0, 0  acc: 11178 / 17031 =  65.633
0, 1  acc:  6557 /  8495 =  77.187
1, 0  acc: 127557 / 128859 =  98.990
1, 1  acc:  7954 /  8385 =  94.860
--------------------------------------
Average acc: 153246 / 162770 =  94.149
Robust  acc: 11178 / 17031 =  65.633
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5364 /  8535 =  62.847
0, 1  acc:  6359 /  8276 =  76.837
1, 0  acc:  2859 /  2874 =  99.478
1, 1  acc:   178 /   182 =  97.802
------------------------------------
Average acc: 14760 / 19867 =  74.294
Robust  acc:  5364 /  8535 =  62.847
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 76.090
Robust Acc: 68.650 | Best Acc: 99.395
-------------------------------------
Training, Epoch 39:
Accuracies by groups:
0, 0  acc:  6705 /  9767 =  68.650
0, 1  acc:  5848 /  7535 =  77.611
1, 0  acc:  2465 /  2480 =  99.395
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15189 / 19962 =  76.090
Robust  acc:  6705 /  9767 =  68.650
------------------------------------
Accuracies by groups:
0, 0  acc:  6705 /  9767 =  68.650
0, 1  acc:  5848 /  7535 =  77.611
1, 0  acc:  2465 /  2480 =  99.395
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15189 / 19962 =  76.090
Robust  acc:  6705 /  9767 =  68.650
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  6705 /  9767 =  68.650
0, 1  acc:  5848 /  7535 =  77.611
1, 0  acc:  2465 /  2480 =  99.395
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 15189 / 19962 =  76.090
Robust  acc:  6705 /  9767 =  68.650
------------------------------------
Epoch:  41 | Train Loss: 0.002 | Train Acc: 94.157 | Val Loss: 0.005 | Val Acc: 77.274
Training:
Accuracies by groups:
0, 0  acc: 11522 / 17336 =  66.463
0, 1  acc:  6748 /  8677 =  77.769
1, 0  acc: 127184 / 128497 =  98.978
1, 1  acc:  7805 /  8260 =  94.492
--------------------------------------
Average acc: 153259 / 162770 =  94.157
Robust  acc: 11522 / 17336 =  66.463
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  5923 /  8535 =  69.397
0, 1  acc:  6410 /  8276 =  77.453
1, 0  acc:  2842 /  2874 =  98.887
1, 1  acc:   177 /   182 =  97.253
------------------------------------
Average acc: 15352 / 19867 =  77.274
Robust  acc:  5923 /  8535 =  69.397
------------------------------------
-------------------------------------------
Avg Test Loss: 0.005 | Avg Test Acc: 79.195
Robust Acc: 75.192 | Best Acc: 98.992
-------------------------------------
Training, Epoch 40:
Accuracies by groups:
0, 0  acc:  7344 /  9767 =  75.192
0, 1  acc:  5843 /  7535 =  77.545
1, 0  acc:  2455 /  2480 =  98.992
1, 1  acc:   167 /   180 =  92.778
------------------------------------
Average acc: 15809 / 19962 =  79.195
Robust  acc:  7344 /  9767 =  75.192
------------------------------------
Accuracies by groups:
0, 0  acc:  7344 /  9767 =  75.192
0, 1  acc:  5843 /  7535 =  77.545
1, 0  acc:  2455 /  2480 =  98.992
1, 1  acc:   167 /   180 =  92.778
------------------------------------
Average acc: 15809 / 19962 =  79.195
Robust  acc:  7344 /  9767 =  75.192
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7344 /  9767 =  75.192
0, 1  acc:  5843 /  7535 =  77.545
1, 0  acc:  2455 /  2480 =  98.992
1, 1  acc:   167 /   180 =  92.778
------------------------------------
Average acc: 15809 / 19962 =  79.195
Robust  acc:  7344 /  9767 =  75.192
------------------------------------
Epoch:  42 | Train Loss: 0.002 | Train Acc: 94.158 | Val Loss: 0.004 | Val Acc: 83.717
Training:
Accuracies by groups:
0, 0  acc: 11309 / 17098 =  66.142
0, 1  acc:  6856 /  8793 =  77.971
1, 0  acc: 127270 / 128607 =  98.960
1, 1  acc:  7826 /  8272 =  94.608
--------------------------------------
Average acc: 153261 / 162770 =  94.158
Robust  acc: 11309 / 17098 =  66.142
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6287 /  8535 =  73.661
0, 1  acc:  7327 /  8276 =  88.533
1, 0  acc:  2846 /  2874 =  99.026
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 16632 / 19867 =  83.717
Robust  acc:  6287 /  8535 =  73.661
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 85.197
Robust Acc: 79.144 | Best Acc: 98.710
-------------------------------------
Training, Epoch 41:
Accuracies by groups:
0, 0  acc:  7730 /  9767 =  79.144
0, 1  acc:  6668 /  7535 =  88.494
1, 0  acc:  2448 /  2480 =  98.710
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17007 / 19962 =  85.197
Robust  acc:  7730 /  9767 =  79.144
------------------------------------
Accuracies by groups:
0, 0  acc:  7730 /  9767 =  79.144
0, 1  acc:  6668 /  7535 =  88.494
1, 0  acc:  2448 /  2480 =  98.710
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17007 / 19962 =  85.197
Robust  acc:  7730 /  9767 =  79.144
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7730 /  9767 =  79.144
0, 1  acc:  6668 /  7535 =  88.494
1, 0  acc:  2448 /  2480 =  98.710
1, 1  acc:   161 /   180 =  89.444
------------------------------------
Average acc: 17007 / 19962 =  85.197
Robust  acc:  7730 /  9767 =  79.144
------------------------------------
Epoch:  43 | Train Loss: 0.002 | Train Acc: 94.257 | Val Loss: 0.004 | Val Acc: 83.536
Training:
Accuracies by groups:
0, 0  acc: 11621 / 17390 =  66.826
0, 1  acc:  6794 /  8633 =  78.698
1, 0  acc: 127201 / 128503 =  98.987
1, 1  acc:  7806 /  8244 =  94.687
--------------------------------------
Average acc: 153422 / 162770 =  94.257
Robust  acc: 11621 / 17390 =  66.826
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6468 /  8535 =  75.782
0, 1  acc:  7125 /  8276 =  86.092
1, 0  acc:  2833 /  2874 =  98.573
1, 1  acc:   170 /   182 =  93.407
------------------------------------
Average acc: 16596 / 19867 =  83.536
Robust  acc:  6468 /  8535 =  75.782
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 85.327
Robust Acc: 81.079 | Best Acc: 97.984
-------------------------------------
Training, Epoch 42:
Accuracies by groups:
0, 0  acc:  7919 /  9767 =  81.079
0, 1  acc:  6518 /  7535 =  86.503
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17033 / 19962 =  85.327
Robust  acc:  7919 /  9767 =  81.079
------------------------------------
Accuracies by groups:
0, 0  acc:  7919 /  9767 =  81.079
0, 1  acc:  6518 /  7535 =  86.503
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17033 / 19962 =  85.327
Robust  acc:  7919 /  9767 =  81.079
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7919 /  9767 =  81.079
0, 1  acc:  6518 /  7535 =  86.503
1, 0  acc:  2430 /  2480 =  97.984
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17033 / 19962 =  85.327
Robust  acc:  7919 /  9767 =  81.079
------------------------------------
Epoch:  44 | Train Loss: 0.002 | Train Acc: 94.218 | Val Loss: 0.003 | Val Acc: 91.126
Training:
Accuracies by groups:
0, 0  acc: 11528 / 17256 =  66.806
0, 1  acc:  6621 /  8456 =  78.299
1, 0  acc: 127499 / 128906 =  98.909
1, 1  acc:  7710 /  8152 =  94.578
--------------------------------------
Average acc: 153358 / 162770 =  94.218
Robust  acc: 11528 / 17256 =  66.806
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7508 /  8535 =  87.967
0, 1  acc:  7744 /  8276 =  93.572
1, 0  acc:  2703 /  2874 =  94.050
1, 1  acc:   149 /   182 =  81.868
------------------------------------
Average acc: 18104 / 19867 =  91.126
Robust  acc:   149 /   182 =  81.868
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 92.145
Robust Acc: 79.444 | Best Acc: 94.253
-------------------------------------
Training, Epoch 43:
Accuracies by groups:
0, 0  acc:  8860 /  9767 =  90.714
0, 1  acc:  7102 /  7535 =  94.253
1, 0  acc:  2289 /  2480 =  92.298
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18394 / 19962 =  92.145
Robust  acc:   143 /   180 =  79.444
------------------------------------
Accuracies by groups:
0, 0  acc:  8860 /  9767 =  90.714
0, 1  acc:  7102 /  7535 =  94.253
1, 0  acc:  2289 /  2480 =  92.298
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18394 / 19962 =  92.145
Robust  acc:   143 /   180 =  79.444
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8860 /  9767 =  90.714
0, 1  acc:  7102 /  7535 =  94.253
1, 0  acc:  2289 /  2480 =  92.298
1, 1  acc:   143 /   180 =  79.444
------------------------------------
Average acc: 18394 / 19962 =  92.145
Robust  acc:   143 /   180 =  79.444
------------------------------------
Epoch:  45 | Train Loss: 0.002 | Train Acc: 94.291 | Val Loss: 0.004 | Val Acc: 85.544
Training:
Accuracies by groups:
0, 0  acc: 11431 / 17130 =  66.731
0, 1  acc:  6752 /  8555 =  78.925
1, 0  acc: 127357 / 128698 =  98.958
1, 1  acc:  7937 /  8387 =  94.635
--------------------------------------
Average acc: 153477 / 162770 =  94.291
Robust  acc: 11431 / 17130 =  66.731
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6750 /  8535 =  79.086
0, 1  acc:  7246 /  8276 =  87.554
1, 0  acc:  2827 /  2874 =  98.365
1, 1  acc:   172 /   182 =  94.505
------------------------------------
Average acc: 16995 / 19867 =  85.544
Robust  acc:  6750 /  8535 =  79.086
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 86.895
Robust Acc: 83.209 | Best Acc: 97.661
-------------------------------------
Training, Epoch 44:
Accuracies by groups:
0, 0  acc:  8127 /  9767 =  83.209
0, 1  acc:  6631 /  7535 =  88.003
1, 0  acc:  2422 /  2480 =  97.661
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17346 / 19962 =  86.895
Robust  acc:  8127 /  9767 =  83.209
------------------------------------
Accuracies by groups:
0, 0  acc:  8127 /  9767 =  83.209
0, 1  acc:  6631 /  7535 =  88.003
1, 0  acc:  2422 /  2480 =  97.661
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17346 / 19962 =  86.895
Robust  acc:  8127 /  9767 =  83.209
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8127 /  9767 =  83.209
0, 1  acc:  6631 /  7535 =  88.003
1, 0  acc:  2422 /  2480 =  97.661
1, 1  acc:   166 /   180 =  92.222
------------------------------------
Average acc: 17346 / 19962 =  86.895
Robust  acc:  8127 /  9767 =  83.209
------------------------------------
Epoch:  46 | Train Loss: 0.002 | Train Acc: 94.266 | Val Loss: 0.003 | Val Acc: 88.785
Training:
Accuracies by groups:
0, 0  acc: 11462 / 17148 =  66.842
0, 1  acc:  6786 /  8604 =  78.870
1, 0  acc: 127499 / 128856 =  98.947
1, 1  acc:  7689 /  8162 =  94.205
--------------------------------------
Average acc: 153436 / 162770 =  94.266
Robust  acc: 11462 / 17148 =  66.842
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7079 /  8535 =  82.941
0, 1  acc:  7609 /  8276 =  91.941
1, 0  acc:  2794 /  2874 =  97.216
1, 1  acc:   157 /   182 =  86.264
------------------------------------
Average acc: 17639 / 19867 =  88.785
Robust  acc:  7079 /  8535 =  82.941
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 90.116
Robust Acc: 82.222 | Best Acc: 96.774
-------------------------------------
Training, Epoch 45:
Accuracies by groups:
0, 0  acc:  8482 /  9767 =  86.843
0, 1  acc:  6959 /  7535 =  92.356
1, 0  acc:  2400 /  2480 =  96.774
1, 1  acc:   148 /   180 =  82.222
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   148 /   180 =  82.222
------------------------------------
Accuracies by groups:
0, 0  acc:  8482 /  9767 =  86.843
0, 1  acc:  6959 /  7535 =  92.356
1, 0  acc:  2400 /  2480 =  96.774
1, 1  acc:   148 /   180 =  82.222
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   148 /   180 =  82.222
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8482 /  9767 =  86.843
0, 1  acc:  6959 /  7535 =  92.356
1, 0  acc:  2400 /  2480 =  96.774
1, 1  acc:   148 /   180 =  82.222
------------------------------------
Average acc: 17989 / 19962 =  90.116
Robust  acc:   148 /   180 =  82.222
------------------------------------
Epoch:  47 | Train Loss: 0.002 | Train Acc: 94.296 | Val Loss: 0.003 | Val Acc: 90.185
Training:
Accuracies by groups:
0, 0  acc: 11426 / 17050 =  67.015
0, 1  acc:  6816 /  8688 =  78.453
1, 0  acc: 127438 / 128769 =  98.966
1, 1  acc:  7805 /  8263 =  94.457
--------------------------------------
Average acc: 153485 / 162770 =  94.296
Robust  acc: 11426 / 17050 =  67.015
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7183 /  8535 =  84.159
0, 1  acc:  7802 /  8276 =  94.273
1, 0  acc:  2782 /  2874 =  96.799
1, 1  acc:   150 /   182 =  82.418
------------------------------------
Average acc: 17917 / 19867 =  90.185
Robust  acc:   150 /   182 =  82.418
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 91.013
Robust Acc: 77.778 | Best Acc: 95.565
-------------------------------------
Training, Epoch 46:
Accuracies by groups:
0, 0  acc:  8532 /  9767 =  87.355
0, 1  acc:  7126 /  7535 =  94.572
1, 0  acc:  2370 /  2480 =  95.565
1, 1  acc:   140 /   180 =  77.778
------------------------------------
Average acc: 18168 / 19962 =  91.013
Robust  acc:   140 /   180 =  77.778
------------------------------------
Accuracies by groups:
0, 0  acc:  8532 /  9767 =  87.355
0, 1  acc:  7126 /  7535 =  94.572
1, 0  acc:  2370 /  2480 =  95.565
1, 1  acc:   140 /   180 =  77.778
------------------------------------
Average acc: 18168 / 19962 =  91.013
Robust  acc:   140 /   180 =  77.778
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8532 /  9767 =  87.355
0, 1  acc:  7126 /  7535 =  94.572
1, 0  acc:  2370 /  2480 =  95.565
1, 1  acc:   140 /   180 =  77.778
------------------------------------
Average acc: 18168 / 19962 =  91.013
Robust  acc:   140 /   180 =  77.778
------------------------------------
Epoch:  48 | Train Loss: 0.002 | Train Acc: 94.378 | Val Loss: 0.003 | Val Acc: 90.165
Training:
Accuracies by groups:
0, 0  acc: 11602 / 17203 =  67.442
0, 1  acc:  6604 /  8384 =  78.769
1, 0  acc: 127554 / 128853 =  98.992
1, 1  acc:  7859 /  8330 =  94.346
--------------------------------------
Average acc: 153619 / 162770 =  94.378
Robust  acc: 11602 / 17203 =  67.442
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7241 /  8535 =  84.839
0, 1  acc:  7732 /  8276 =  93.427
1, 0  acc:  2784 /  2874 =  96.868
1, 1  acc:   156 /   182 =  85.714
------------------------------------
Average acc: 17913 / 19867 =  90.165
Robust  acc:  7241 /  8535 =  84.839
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 91.083
Robust Acc: 81.667 | Best Acc: 96.089
-------------------------------------
Training, Epoch 47:
Accuracies by groups:
0, 0  acc:  8598 /  9767 =  88.031
0, 1  acc:  7054 /  7535 =  93.616
1, 0  acc:  2383 /  2480 =  96.089
1, 1  acc:   147 /   180 =  81.667
------------------------------------
Average acc: 18182 / 19962 =  91.083
Robust  acc:   147 /   180 =  81.667
------------------------------------
Accuracies by groups:
0, 0  acc:  8598 /  9767 =  88.031
0, 1  acc:  7054 /  7535 =  93.616
1, 0  acc:  2383 /  2480 =  96.089
1, 1  acc:   147 /   180 =  81.667
------------------------------------
Average acc: 18182 / 19962 =  91.083
Robust  acc:   147 /   180 =  81.667
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8598 /  9767 =  88.031
0, 1  acc:  7054 /  7535 =  93.616
1, 0  acc:  2383 /  2480 =  96.089
1, 1  acc:   147 /   180 =  81.667
------------------------------------
Average acc: 18182 / 19962 =  91.083
Robust  acc:   147 /   180 =  81.667
------------------------------------
Epoch:  49 | Train Loss: 0.002 | Train Acc: 94.293 | Val Loss: 0.004 | Val Acc: 80.334
Training:
Accuracies by groups:
0, 0  acc: 11504 / 17091 =  67.310
0, 1  acc:  6666 /  8545 =  78.011
1, 0  acc: 127600 / 128938 =  98.962
1, 1  acc:  7711 /  8196 =  94.082
--------------------------------------
Average acc: 153481 / 162770 =  94.293
Robust  acc: 11504 / 17091 =  67.310
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  6130 /  8535 =  71.822
0, 1  acc:  6819 /  8276 =  82.395
1, 0  acc:  2840 /  2874 =  98.817
1, 1  acc:   171 /   182 =  93.956
------------------------------------
Average acc: 15960 / 19867 =  80.334
Robust  acc:  6130 /  8535 =  71.822
------------------------------------
-------------------------------------------
Avg Test Loss: 0.004 | Avg Test Acc: 82.291
Robust Acc: 77.690 | Best Acc: 98.306
-------------------------------------
Training, Epoch 48:
Accuracies by groups:
0, 0  acc:  7588 /  9767 =  77.690
0, 1  acc:  6230 /  7535 =  82.681
1, 0  acc:  2438 /  2480 =  98.306
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 16427 / 19962 =  82.291
Robust  acc:  7588 /  9767 =  77.690
------------------------------------
Accuracies by groups:
0, 0  acc:  7588 /  9767 =  77.690
0, 1  acc:  6230 /  7535 =  82.681
1, 0  acc:  2438 /  2480 =  98.306
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 16427 / 19962 =  82.291
Robust  acc:  7588 /  9767 =  77.690
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  7588 /  9767 =  77.690
0, 1  acc:  6230 /  7535 =  82.681
1, 0  acc:  2438 /  2480 =  98.306
1, 1  acc:   171 /   180 =  95.000
------------------------------------
Average acc: 16427 / 19962 =  82.291
Robust  acc:  7588 /  9767 =  77.690
------------------------------------
Epoch:  50 | Train Loss: 0.002 | Train Acc: 94.389 | Val Loss: 0.003 | Val Acc: 87.774
Training:
Accuracies by groups:
0, 0  acc: 11633 / 17221 =  67.551
0, 1  acc:  6744 /  8495 =  79.388
1, 0  acc: 127439 / 128769 =  98.967
1, 1  acc:  7821 /  8285 =  94.400
--------------------------------------
Average acc: 153637 / 162770 =  94.389
Robust  acc: 11633 / 17221 =  67.551
--------------------------------------
Validating:
Accuracies by groups:
0, 0  acc:  7036 /  8535 =  82.437
0, 1  acc:  7460 /  8276 =  90.140
1, 0  acc:  2778 /  2874 =  96.660
1, 1  acc:   164 /   182 =  90.110
------------------------------------
Average acc: 17438 / 19867 =  87.774
Robust  acc:  7036 /  8535 =  82.437
------------------------------------
-------------------------------------------
Avg Test Loss: 0.003 | Avg Test Acc: 89.325
Robust Acc: 86.111 | Best Acc: 96.331
-------------------------------------
Training, Epoch 49:
Accuracies by groups:
0, 0  acc:  8469 /  9767 =  86.710
0, 1  acc:  6818 /  7535 =  90.484
1, 0  acc:  2389 /  2480 =  96.331
1, 1  acc:   155 /   180 =  86.111
------------------------------------
Average acc: 17831 / 19962 =  89.325
Robust  acc:   155 /   180 =  86.111
------------------------------------
Accuracies by groups:
0, 0  acc:  8469 /  9767 =  86.710
0, 1  acc:  6818 /  7535 =  90.484
1, 0  acc:  2389 /  2480 =  96.331
1, 1  acc:   155 /   180 =  86.111
------------------------------------
Average acc: 17831 / 19962 =  89.325
Robust  acc:   155 /   180 =  86.111
------------------------------------
Testing:
Accuracies by groups:
0, 0  acc:  8469 /  9767 =  86.710
0, 1  acc:  6818 /  7535 =  90.484
1, 0  acc:  2389 /  2480 =  96.331
1, 1  acc:   155 /   180 =  86.111
------------------------------------
Average acc: 17831 / 19962 =  89.325
Robust  acc:   155 /   180 =  86.111
------------------------------------
replace: True
-> Updating checkpoint debias-end_seed25.pt...
Checkpoint saved at ./model/celebA/config/debias-end_seed25.pt
