/home/mmsys/anaconda3/envs/LLFormer/bin/python /media/mmsys/6f1091c9-4ed8-4a10-a03d-2acef144d2e1/SXY/ablation/LLormer-DA/train_psnrssim.py load training yaml file: configs/LOL/train/training_LOL.yaml ==> Build the model ==> Loading datasets ==> Training details: ------------------------------------------------------------------ Restoration mode: LLFormer_LOL-DA Train patches size: 128x128 Val patches size: 128x128 Model parameters: 24549014 Start/End epochs: 1~500 Batch sizes: 1 Learning rate: 1e-4 ------------------------------------------------------------------ ==> Training start: 100%|██████████| 485/485 [03:03<00:00, 2.64it/s] [epoch 1 PSNR: 12.9392 --- best_epoch 1 Best_PSNR 12.9392] [epoch 1 SSIM: 0.2422 --- best_epoch 1 Best_SSIM 0.2422] ------------------------------------------------------------------ Epoch: 1 Time: 236.9543 Loss: 300.6611 LearningRate 0.000067 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:00<00:00, 2.68it/s] [epoch 2 PSNR: 13.2200 --- best_epoch 2 Best_PSNR 13.2200] [epoch 2 SSIM: 0.4242 --- best_epoch 2 Best_SSIM 0.4242] ------------------------------------------------------------------ Epoch: 2 Time: 234.5352 Loss: 155.4707 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:06<00:00, 2.60it/s] [epoch 3 PSNR: 13.2780 --- best_epoch 3 Best_PSNR 13.2780] [epoch 3 SSIM: 0.4221 --- best_epoch 2 Best_SSIM 0.4242] ------------------------------------------------------------------ Epoch: 3 Time: 237.6840 Loss: 127.2522 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 4 PSNR: 13.5298 --- best_epoch 4 Best_PSNR 13.5298] [epoch 4 SSIM: 0.4267 --- best_epoch 4 Best_SSIM 0.4267] ------------------------------------------------------------------ Epoch: 4 Time: 245.9495 Loss: 116.8293 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 5 PSNR: 13.6178 --- best_epoch 5 Best_PSNR 13.6178] [epoch 5 SSIM: 0.4116 --- best_epoch 4 Best_SSIM 0.4267] ------------------------------------------------------------------ Epoch: 5 Time: 242.7179 Loss: 102.6880 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 6 PSNR: 13.4869 --- best_epoch 5 Best_PSNR 13.6178] [epoch 6 SSIM: 0.4293 --- best_epoch 6 Best_SSIM 0.4293] ------------------------------------------------------------------ Epoch: 6 Time: 243.0979 Loss: 100.2496 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.57it/s] [epoch 7 PSNR: 13.6978 --- best_epoch 7 Best_PSNR 13.6978] [epoch 7 SSIM: 0.4231 --- best_epoch 6 Best_SSIM 0.4293] ------------------------------------------------------------------ Epoch: 7 Time: 241.4643 Loss: 98.5784 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 8 PSNR: 13.5177 --- best_epoch 7 Best_PSNR 13.6978] [epoch 8 SSIM: 0.4223 --- best_epoch 6 Best_SSIM 0.4293] ------------------------------------------------------------------ Epoch: 8 Time: 241.2489 Loss: 93.1802 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 9 PSNR: 13.6095 --- best_epoch 7 Best_PSNR 13.6978] [epoch 9 SSIM: 0.4435 --- best_epoch 9 Best_SSIM 0.4435] ------------------------------------------------------------------ Epoch: 9 Time: 243.0430 Loss: 91.0201 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 10 PSNR: 14.2383 --- best_epoch 10 Best_PSNR 14.2383] [epoch 10 SSIM: 0.4534 --- best_epoch 10 Best_SSIM 0.4534] ------------------------------------------------------------------ Epoch: 10 Time: 245.6985 Loss: 87.8424 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 11 PSNR: 14.2507 --- best_epoch 11 Best_PSNR 14.2507] [epoch 11 SSIM: 0.4801 --- best_epoch 11 Best_SSIM 0.4801] ------------------------------------------------------------------ Epoch: 11 Time: 245.8451 Loss: 86.7920 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 12 PSNR: 13.6300 --- best_epoch 11 Best_PSNR 14.2507] [epoch 12 SSIM: 0.4323 --- best_epoch 11 Best_SSIM 0.4801] ------------------------------------------------------------------ Epoch: 12 Time: 241.2346 Loss: 85.7409 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 13 PSNR: 12.4237 --- best_epoch 11 Best_PSNR 14.2507] [epoch 13 SSIM: 0.4517 --- best_epoch 11 Best_SSIM 0.4801] ------------------------------------------------------------------ Epoch: 13 Time: 239.9842 Loss: 83.0741 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 14 PSNR: 14.1661 --- best_epoch 11 Best_PSNR 14.2507] [epoch 14 SSIM: 0.4842 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 14 Time: 244.1370 Loss: 82.9281 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 15 PSNR: 13.3518 --- best_epoch 11 Best_PSNR 14.2507] [epoch 15 SSIM: 0.4711 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 15 Time: 239.9422 Loss: 82.8702 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 16 PSNR: 12.8262 --- best_epoch 11 Best_PSNR 14.2507] [epoch 16 SSIM: 0.4632 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 16 Time: 240.7426 Loss: 81.4726 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 17 PSNR: 13.3852 --- best_epoch 11 Best_PSNR 14.2507] [epoch 17 SSIM: 0.4753 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 17 Time: 240.3797 Loss: 81.9502 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 18 PSNR: 13.0936 --- best_epoch 11 Best_PSNR 14.2507] [epoch 18 SSIM: 0.4560 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 18 Time: 241.5072 Loss: 81.9861 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 19 PSNR: 13.2399 --- best_epoch 11 Best_PSNR 14.2507] [epoch 19 SSIM: 0.4639 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 19 Time: 240.4171 Loss: 80.2475 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 20 PSNR: 14.0860 --- best_epoch 11 Best_PSNR 14.2507] [epoch 20 SSIM: 0.4695 --- best_epoch 14 Best_SSIM 0.4842] ------------------------------------------------------------------ Epoch: 20 Time: 240.2203 Loss: 79.7678 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:13<00:00, 2.51it/s] [epoch 21 PSNR: 14.4695 --- best_epoch 21 Best_PSNR 14.4695] [epoch 21 SSIM: 0.4962 --- best_epoch 21 Best_SSIM 0.4962] ------------------------------------------------------------------ Epoch: 21 Time: 247.5349 Loss: 78.9512 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 22 PSNR: 14.4645 --- best_epoch 21 Best_PSNR 14.4695] [epoch 22 SSIM: 0.4944 --- best_epoch 21 Best_SSIM 0.4962] ------------------------------------------------------------------ Epoch: 22 Time: 239.3161 Loss: 81.3114 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 23 PSNR: 13.9300 --- best_epoch 21 Best_PSNR 14.4695] [epoch 23 SSIM: 0.4905 --- best_epoch 21 Best_SSIM 0.4962] ------------------------------------------------------------------ Epoch: 23 Time: 239.4963 Loss: 79.0123 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 24 PSNR: 13.9197 --- best_epoch 21 Best_PSNR 14.4695] [epoch 24 SSIM: 0.4854 --- best_epoch 21 Best_SSIM 0.4962] ------------------------------------------------------------------ Epoch: 24 Time: 241.5130 Loss: 79.9127 LearningRate 0.000100 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 25 PSNR: 14.5532 --- best_epoch 25 Best_PSNR 14.5532] [epoch 25 SSIM: 0.4972 --- best_epoch 25 Best_SSIM 0.4972] ------------------------------------------------------------------ Epoch: 25 Time: 244.4979 Loss: 78.0317 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:11<00:00, 2.54it/s] [epoch 26 PSNR: 14.3044 --- best_epoch 25 Best_PSNR 14.5532] [epoch 26 SSIM: 0.4942 --- best_epoch 25 Best_SSIM 0.4972] ------------------------------------------------------------------ Epoch: 26 Time: 241.2161 Loss: 80.9282 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 27 PSNR: 15.0169 --- best_epoch 27 Best_PSNR 15.0169] [epoch 27 SSIM: 0.5007 --- best_epoch 27 Best_SSIM 0.5007] ------------------------------------------------------------------ Epoch: 27 Time: 244.5918 Loss: 80.1729 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 28 PSNR: 14.5165 --- best_epoch 27 Best_PSNR 15.0169] [epoch 28 SSIM: 0.4990 --- best_epoch 27 Best_SSIM 0.5007] ------------------------------------------------------------------ Epoch: 28 Time: 240.1564 Loss: 80.0008 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 29 PSNR: 15.0566 --- best_epoch 29 Best_PSNR 15.0566] [epoch 29 SSIM: 0.5079 --- best_epoch 29 Best_SSIM 0.5079] ------------------------------------------------------------------ Epoch: 29 Time: 244.1492 Loss: 80.4654 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 30 PSNR: 15.2334 --- best_epoch 30 Best_PSNR 15.2334] [epoch 30 SSIM: 0.5148 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 30 Time: 245.2543 Loss: 78.8067 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:08<00:00, 2.57it/s] [epoch 31 PSNR: 14.8513 --- best_epoch 30 Best_PSNR 15.2334] [epoch 31 SSIM: 0.5033 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 31 Time: 239.1968 Loss: 77.4602 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 32 PSNR: 14.6775 --- best_epoch 30 Best_PSNR 15.2334] [epoch 32 SSIM: 0.4989 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 32 Time: 239.9141 Loss: 77.7658 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 33 PSNR: 13.3112 --- best_epoch 30 Best_PSNR 15.2334] [epoch 33 SSIM: 0.4867 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 33 Time: 239.5179 Loss: 78.5356 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 34 PSNR: 15.1050 --- best_epoch 30 Best_PSNR 15.2334] [epoch 34 SSIM: 0.5078 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 34 Time: 240.0685 Loss: 75.5973 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:08<00:00, 2.57it/s] [epoch 35 PSNR: 14.4734 --- best_epoch 30 Best_PSNR 15.2334] [epoch 35 SSIM: 0.5004 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 35 Time: 239.3931 Loss: 77.2141 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 36 PSNR: 14.8251 --- best_epoch 30 Best_PSNR 15.2334] [epoch 36 SSIM: 0.5081 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 36 Time: 240.8083 Loss: 77.4241 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 37 PSNR: 13.3368 --- best_epoch 30 Best_PSNR 15.2334] [epoch 37 SSIM: 0.4604 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 37 Time: 240.1561 Loss: 76.3514 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 38 PSNR: 13.7987 --- best_epoch 30 Best_PSNR 15.2334] [epoch 38 SSIM: 0.4749 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 38 Time: 240.5590 Loss: 75.9244 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 39 PSNR: 13.9483 --- best_epoch 30 Best_PSNR 15.2334] [epoch 39 SSIM: 0.4991 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 39 Time: 240.1905 Loss: 76.5482 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:08<00:00, 2.57it/s] [epoch 40 PSNR: 15.2167 --- best_epoch 30 Best_PSNR 15.2334] [epoch 40 SSIM: 0.5117 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 40 Time: 238.3952 Loss: 79.2601 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 41 PSNR: 14.6197 --- best_epoch 30 Best_PSNR 15.2334] [epoch 41 SSIM: 0.4864 --- best_epoch 30 Best_SSIM 0.5148] ------------------------------------------------------------------ Epoch: 41 Time: 240.3929 Loss: 74.8834 LearningRate 0.000099 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 42 PSNR: 15.3286 --- best_epoch 42 Best_PSNR 15.3286] [epoch 42 SSIM: 0.5183 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 42 Time: 245.6409 Loss: 75.7780 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 43 PSNR: 14.4983 --- best_epoch 42 Best_PSNR 15.3286] [epoch 43 SSIM: 0.4913 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 43 Time: 239.8966 Loss: 75.5054 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 44 PSNR: 14.8328 --- best_epoch 42 Best_PSNR 15.3286] [epoch 44 SSIM: 0.5088 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 44 Time: 239.9366 Loss: 78.8608 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 45 PSNR: 13.6886 --- best_epoch 42 Best_PSNR 15.3286] [epoch 45 SSIM: 0.4919 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 45 Time: 241.1952 Loss: 78.4922 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 46 PSNR: 14.1341 --- best_epoch 42 Best_PSNR 15.3286] [epoch 46 SSIM: 0.5049 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 46 Time: 239.8078 Loss: 76.8635 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 47 PSNR: 10.9450 --- best_epoch 42 Best_PSNR 15.3286] [epoch 47 SSIM: 0.4580 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 47 Time: 240.1842 Loss: 77.4210 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 48 PSNR: 13.0694 --- best_epoch 42 Best_PSNR 15.3286] [epoch 48 SSIM: 0.4902 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 48 Time: 239.7309 Loss: 75.9978 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 49 PSNR: 14.0484 --- best_epoch 42 Best_PSNR 15.3286] [epoch 49 SSIM: 0.5027 --- best_epoch 42 Best_SSIM 0.5183] ------------------------------------------------------------------ Epoch: 49 Time: 240.4878 Loss: 76.6017 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 50 PSNR: 15.5054 --- best_epoch 50 Best_PSNR 15.5054] [epoch 50 SSIM: 0.5190 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 50 Time: 244.0712 Loss: 75.7120 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 51 PSNR: 14.7156 --- best_epoch 50 Best_PSNR 15.5054] [epoch 51 SSIM: 0.5118 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 51 Time: 240.3070 Loss: 75.4484 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 52 PSNR: 14.1065 --- best_epoch 50 Best_PSNR 15.5054] [epoch 52 SSIM: 0.4992 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 52 Time: 239.2526 Loss: 75.2371 LearningRate 0.000098 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 53 PSNR: 12.6133 --- best_epoch 50 Best_PSNR 15.5054] [epoch 53 SSIM: 0.4817 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 53 Time: 240.1624 Loss: 76.9093 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 54 PSNR: 14.9350 --- best_epoch 50 Best_PSNR 15.5054] [epoch 54 SSIM: 0.5168 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 54 Time: 240.0452 Loss: 73.0691 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 55 PSNR: 14.3905 --- best_epoch 50 Best_PSNR 15.5054] [epoch 55 SSIM: 0.5058 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 55 Time: 240.7184 Loss: 75.2322 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:11<00:00, 2.54it/s] [epoch 56 PSNR: 14.8735 --- best_epoch 50 Best_PSNR 15.5054] [epoch 56 SSIM: 0.5135 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 56 Time: 241.4215 Loss: 77.4396 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 57 PSNR: 13.7998 --- best_epoch 50 Best_PSNR 15.5054] [epoch 57 SSIM: 0.4969 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 57 Time: 238.8866 Loss: 71.1801 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 58 PSNR: 13.6894 --- best_epoch 50 Best_PSNR 15.5054] [epoch 58 SSIM: 0.5021 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 58 Time: 240.5808 Loss: 75.0319 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 59 PSNR: 14.6827 --- best_epoch 50 Best_PSNR 15.5054] [epoch 59 SSIM: 0.5027 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 59 Time: 239.9513 Loss: 76.0319 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:08<00:00, 2.57it/s] [epoch 60 PSNR: 14.6852 --- best_epoch 50 Best_PSNR 15.5054] [epoch 60 SSIM: 0.5117 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 60 Time: 238.8071 Loss: 73.5957 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 61 PSNR: 13.5616 --- best_epoch 50 Best_PSNR 15.5054] [epoch 61 SSIM: 0.4971 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 61 Time: 239.3586 Loss: 72.0091 LearningRate 0.000097 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 62 PSNR: 14.4594 --- best_epoch 50 Best_PSNR 15.5054] [epoch 62 SSIM: 0.5117 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 62 Time: 240.1853 Loss: 74.3871 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 63 PSNR: 14.6339 --- best_epoch 50 Best_PSNR 15.5054] [epoch 63 SSIM: 0.5157 --- best_epoch 50 Best_SSIM 0.5190] ------------------------------------------------------------------ Epoch: 63 Time: 239.8599 Loss: 74.7018 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 64 PSNR: 15.4361 --- best_epoch 50 Best_PSNR 15.5054] [epoch 64 SSIM: 0.5234 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 64 Time: 242.9569 Loss: 75.1196 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 65 PSNR: 14.1652 --- best_epoch 50 Best_PSNR 15.5054] [epoch 65 SSIM: 0.5033 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 65 Time: 239.3522 Loss: 71.8825 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 66 PSNR: 14.7230 --- best_epoch 50 Best_PSNR 15.5054] [epoch 66 SSIM: 0.5143 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 66 Time: 240.5999 Loss: 72.2329 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 67 PSNR: 14.5734 --- best_epoch 50 Best_PSNR 15.5054] [epoch 67 SSIM: 0.5143 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 67 Time: 239.7080 Loss: 73.6255 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 68 PSNR: 14.9847 --- best_epoch 50 Best_PSNR 15.5054] [epoch 68 SSIM: 0.5170 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 68 Time: 240.7693 Loss: 74.7068 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 69 PSNR: 13.2297 --- best_epoch 50 Best_PSNR 15.5054] [epoch 69 SSIM: 0.4925 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 69 Time: 239.6196 Loss: 76.5027 LearningRate 0.000096 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 70 PSNR: 14.5605 --- best_epoch 50 Best_PSNR 15.5054] [epoch 70 SSIM: 0.5067 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 70 Time: 240.1681 Loss: 74.5328 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 71 PSNR: 15.2959 --- best_epoch 50 Best_PSNR 15.5054] [epoch 71 SSIM: 0.5219 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 71 Time: 240.0417 Loss: 75.2580 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 72 PSNR: 15.5987 --- best_epoch 72 Best_PSNR 15.5987] [epoch 72 SSIM: 0.5197 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 72 Time: 243.1639 Loss: 74.0990 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 73 PSNR: 13.5257 --- best_epoch 72 Best_PSNR 15.5987] [epoch 73 SSIM: 0.4983 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 73 Time: 240.2263 Loss: 72.2422 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 74 PSNR: 14.7637 --- best_epoch 72 Best_PSNR 15.5987] [epoch 74 SSIM: 0.5127 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 74 Time: 241.1158 Loss: 73.6175 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 75 PSNR: 14.9682 --- best_epoch 72 Best_PSNR 15.5987] [epoch 75 SSIM: 0.5180 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 75 Time: 239.6722 Loss: 73.8762 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 76 PSNR: 14.1028 --- best_epoch 72 Best_PSNR 15.5987] [epoch 76 SSIM: 0.5055 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 76 Time: 239.1607 Loss: 72.6612 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 77 PSNR: 13.4040 --- best_epoch 72 Best_PSNR 15.5987] [epoch 77 SSIM: 0.4991 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 77 Time: 239.2110 Loss: 72.8090 LearningRate 0.000095 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 78 PSNR: 13.5966 --- best_epoch 72 Best_PSNR 15.5987] [epoch 78 SSIM: 0.5005 --- best_epoch 64 Best_SSIM 0.5234] ------------------------------------------------------------------ Epoch: 78 Time: 240.1477 Loss: 74.1740 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 79 PSNR: 15.3327 --- best_epoch 72 Best_PSNR 15.5987] [epoch 79 SSIM: 0.5266 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 79 Time: 243.0586 Loss: 73.6473 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.54it/s] [epoch 80 PSNR: 14.6878 --- best_epoch 72 Best_PSNR 15.5987] [epoch 80 SSIM: 0.5138 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 80 Time: 240.5102 Loss: 72.6360 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.57it/s] [epoch 81 PSNR: 14.3763 --- best_epoch 72 Best_PSNR 15.5987] [epoch 81 SSIM: 0.5100 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 81 Time: 239.3289 Loss: 73.7046 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 82 PSNR: 15.6086 --- best_epoch 82 Best_PSNR 15.6086] [epoch 82 SSIM: 0.5175 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 82 Time: 242.1163 Loss: 74.6069 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 83 PSNR: 14.1110 --- best_epoch 82 Best_PSNR 15.6086] [epoch 83 SSIM: 0.5092 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 83 Time: 239.9854 Loss: 72.1454 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 84 PSNR: 15.5376 --- best_epoch 82 Best_PSNR 15.6086] [epoch 84 SSIM: 0.5200 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 84 Time: 239.5732 Loss: 73.6839 LearningRate 0.000094 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 85 PSNR: 14.4315 --- best_epoch 82 Best_PSNR 15.6086] [epoch 85 SSIM: 0.5100 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 85 Time: 241.0931 Loss: 72.7796 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 86 PSNR: 15.0133 --- best_epoch 82 Best_PSNR 15.6086] [epoch 86 SSIM: 0.5163 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 86 Time: 239.6271 Loss: 73.4423 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 87 PSNR: 15.3330 --- best_epoch 82 Best_PSNR 15.6086] [epoch 87 SSIM: 0.5141 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 87 Time: 240.4622 Loss: 71.7696 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.55it/s] [epoch 88 PSNR: 14.4690 --- best_epoch 82 Best_PSNR 15.6086] [epoch 88 SSIM: 0.5059 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 88 Time: 240.0357 Loss: 72.1429 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 89 PSNR: 15.8675 --- best_epoch 89 Best_PSNR 15.8675] [epoch 89 SSIM: 0.5100 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 89 Time: 243.2063 Loss: 73.7085 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:09<00:00, 2.56it/s] [epoch 90 PSNR: 13.9559 --- best_epoch 89 Best_PSNR 15.8675] [epoch 90 SSIM: 0.5073 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 90 Time: 239.4461 Loss: 71.9671 LearningRate 0.000093 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:10<00:00, 2.55it/s] [epoch 91 PSNR: 15.2135 --- best_epoch 89 Best_PSNR 15.8675] [epoch 91 SSIM: 0.5192 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 91 Time: 240.8997 Loss: 69.8153 LearningRate 0.000092 ------------------------------------------------------------------ 100%|██████████| 485/485 [03:11<00:00, 2.54it/s] [epoch 92 PSNR: 15.1205 --- best_epoch 89 Best_PSNR 15.8675] [epoch 92 SSIM: 0.4758 --- best_epoch 79 Best_SSIM 0.5266] ------------------------------------------------------------------ Epoch: 92 Time: 241.0529 Loss: 72.5776 LearningRate 0.000092 ------------------------------------------------------------------ 28%|██▊ | 136/485 [00:53<02:17, 2.53it/s] Traceback (most recent call last): File "/media/mmsys/6f1091c9-4ed8-4a10-a03d-2acef144d2e1/SXY/ablation/LLormer-DA/train_psnrssim.py", line 147, in <module> loss.backward() File "/home/mmsys/anaconda3/envs/LLFormer/lib/python3.7/site-packages/torch/_tensor.py", line 488, in backward self, gradient, retain_graph, create_graph, inputs=inputs File "/home/mmsys/anaconda3/envs/LLFormer/lib/python3.7/site-packages/torch/autograd/__init__.py", line 199, in backward allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass KeyboardInterrupt Process finished with exit code 1