Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.444724
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.897759
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.705424
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.648387
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.610637
Train Epoch: 1 - Loss: 0.7681, Accuracy: 93.15%

Test set: Average loss: 0.5992, Accuracy: 9777/10000 (97.77%)

New best model saved with accuracy: 97.77%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.611852
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.581369
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.591108
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.568814
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.573359
Train Epoch: 2 - Loss: 0.5820, Accuracy: 98.24%

Test set: Average loss: 0.5623, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.562382
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.543117
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.556359
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.553619
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.585253
Train Epoch: 3 - Loss: 0.5585, Accuracy: 98.79%

Test set: Average loss: 0.5493, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.568835
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.555979
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.550147
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.554290
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.551534
Train Epoch: 4 - Loss: 0.5473, Accuracy: 99.06%

Test set: Average loss: 0.5443, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.540198
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.532788
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.537258
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.527116
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.549516
Train Epoch: 5 - Loss: 0.5396, Accuracy: 99.25%

Test set: Average loss: 0.5367, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.529925
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.542195
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.533998
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.532977
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.536133
Train Epoch: 6 - Loss: 0.5340, Accuracy: 99.44%

Test set: Average loss: 0.5337, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.530050
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.522312
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.519112
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.529343
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.524874
Train Epoch: 7 - Loss: 0.5307, Accuracy: 99.49%

Test set: Average loss: 0.5325, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.520827
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.522755
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.528978
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.537210
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.523472
Train Epoch: 8 - Loss: 0.5281, Accuracy: 99.53%

Test set: Average loss: 0.5303, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.525526
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.514599
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.530423
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.520677
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.517562
Train Epoch: 9 - Loss: 0.5257, Accuracy: 99.61%

Test set: Average loss: 0.5286, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.521032
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.514123
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.524397
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.517816
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.526136
Train Epoch: 10 - Loss: 0.5237, Accuracy: 99.65%

Test set: Average loss: 0.5279, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.530224
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.521214
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.513502
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.515283
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.527650
Train Epoch: 11 - Loss: 0.5223, Accuracy: 99.68%

Test set: Average loss: 0.5261, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.519723
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.539863
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.516263
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.517191
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.521295
Train Epoch: 12 - Loss: 0.5214, Accuracy: 99.68%

Test set: Average loss: 0.5265, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.520635
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.513752
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.520229
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.527716
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.517403
Train Epoch: 13 - Loss: 0.5201, Accuracy: 99.74%

Test set: Average loss: 0.5250, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.512906
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.510255
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.519941
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.522508
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.515641
Train Epoch: 14 - Loss: 0.5191, Accuracy: 99.78%

Test set: Average loss: 0.5248, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.513720
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.516451
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.519402
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.511860
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.533799
Train Epoch: 15 - Loss: 0.5184, Accuracy: 99.78%

Test set: Average loss: 0.5244, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.516340
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.521109
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.522135
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.510064
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.514301
Train Epoch: 16 - Loss: 0.5176, Accuracy: 99.80%

Test set: Average loss: 0.5236, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.512315
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.511791
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.508576
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.512693
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.510407
Train Epoch: 17 - Loss: 0.5174, Accuracy: 99.80%

Test set: Average loss: 0.5237, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.512752
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.515130
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.515627
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.518457
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.519278
Train Epoch: 18 - Loss: 0.5169, Accuracy: 99.83%

Test set: Average loss: 0.5235, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.543398
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.519958
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.518078
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.518275
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.524042
Train Epoch: 19 - Loss: 0.5166, Accuracy: 99.82%

Test set: Average loss: 0.5233, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.521011
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.514246
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.512992
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.513652
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.519161
Train Epoch: 20 - Loss: 0.5162, Accuracy: 99.85%

Test set: Average loss: 0.5235, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.513823
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.511242
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.515402
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.509462
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.515643
Train Epoch: 21 - Loss: 0.5160, Accuracy: 99.84%

Test set: Average loss: 0.5234, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.535781
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.512554
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.520351
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.512529
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.513381
Train Epoch: 22 - Loss: 0.5158, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.511624
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.526518
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.509470
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.508710
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.511053
Train Epoch: 23 - Loss: 0.5159, Accuracy: 99.85%

Test set: Average loss: 0.5230, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.528647
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.526100
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.512468
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.510106
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.518474
Train Epoch: 24 - Loss: 0.5158, Accuracy: 99.86%

Test set: Average loss: 0.5230, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.516243
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.516953
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.515837
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.513090
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.510850
Train Epoch: 25 - Loss: 0.5155, Accuracy: 99.85%

Test set: Average loss: 0.5231, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.515631
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.510634
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.511511
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.512759
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.511396
Train Epoch: 26 - Loss: 0.5180, Accuracy: 99.78%

Test set: Average loss: 0.5239, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.510530
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.521433
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.519314
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.515049
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.515796
Train Epoch: 27 - Loss: 0.5173, Accuracy: 99.79%

Test set: Average loss: 0.5230, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.517254
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.517087
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.513697
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.509400
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.513963
Train Epoch: 28 - Loss: 0.5163, Accuracy: 99.81%

Test set: Average loss: 0.5244, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.512568
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519555
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.523484
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.511822
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.515369
Train Epoch: 29 - Loss: 0.5153, Accuracy: 99.83%

Test set: Average loss: 0.5218, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.513532
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.514555
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.510487
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.509944
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.511864
Train Epoch: 30 - Loss: 0.5148, Accuracy: 99.84%

Test set: Average loss: 0.5212, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.518261
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.512961
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.512462
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.511305
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.509868
Train Epoch: 31 - Loss: 0.5138, Accuracy: 99.88%

Test set: Average loss: 0.5211, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.516859
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.518104
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.510204
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.509754
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.532639
Train Epoch: 32 - Loss: 0.5132, Accuracy: 99.89%

Test set: Average loss: 0.5210, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.513623
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.510626
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.507010
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.512178
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.516949
Train Epoch: 33 - Loss: 0.5129, Accuracy: 99.88%

Test set: Average loss: 0.5217, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.509562
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.511688
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.511065
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.511712
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.508374
Train Epoch: 34 - Loss: 0.5124, Accuracy: 99.89%

Test set: Average loss: 0.5216, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.512969
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.509337
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.512144
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.513071
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.506937
Train Epoch: 35 - Loss: 0.5118, Accuracy: 99.92%

Test set: Average loss: 0.5201, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.508885
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.517730
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.520224
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.507530
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.510645
Train Epoch: 36 - Loss: 0.5116, Accuracy: 99.91%

Test set: Average loss: 0.5198, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.510965
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.508668
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.514411
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.509910
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.510139
Train Epoch: 37 - Loss: 0.5112, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.511358
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.512620
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.508969
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.511291
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.509823
Train Epoch: 38 - Loss: 0.5107, Accuracy: 99.92%

Test set: Average loss: 0.5191, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.508098
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.505973
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.506245
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513091
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515601
Train Epoch: 39 - Loss: 0.5105, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.514869
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.506765
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.508640
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.516620
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.517638
Train Epoch: 40 - Loss: 0.5105, Accuracy: 99.93%

Test set: Average loss: 0.5192, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.507771
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.513321
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.508309
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.506136
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.507481
Train Epoch: 41 - Loss: 0.5102, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.506479
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.510647
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.509481
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.507408
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.511127
Train Epoch: 42 - Loss: 0.5099, Accuracy: 99.95%

Test set: Average loss: 0.5189, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.510297
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.507659
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.513045
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.505773
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.505470
Train Epoch: 43 - Loss: 0.5098, Accuracy: 99.94%

Test set: Average loss: 0.5187, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.506544
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.511207
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.510835
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.509378
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.506746
Train Epoch: 44 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5187, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.509647
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.514158
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.508061
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.513205
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.517303
Train Epoch: 45 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5187, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.506661
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.506338
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.507130
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.505419
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.506330
Train Epoch: 46 - Loss: 0.5095, Accuracy: 99.96%

Test set: Average loss: 0.5187, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.510898
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.506461
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.509667
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.511717
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.505367
Train Epoch: 47 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5187, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.509478
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.517492
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.539333
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.515511
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.508232
Train Epoch: 48 - Loss: 0.5094, Accuracy: 99.94%

Test set: Average loss: 0.5188, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.510121
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.506705
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.513262
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.508155
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.537734
Train Epoch: 49 - Loss: 0.5093, Accuracy: 99.96%

Test set: Average loss: 0.5188, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.510037
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.506263
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.512224
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.508285
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.511536
Train Epoch: 50 - Loss: 0.5094, Accuracy: 99.97%

Test set: Average loss: 0.5186, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.509078
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.508612
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.507341
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.507526
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.508795
Train Epoch: 51 - Loss: 0.5106, Accuracy: 99.93%

Test set: Average loss: 0.5202, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.507268
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.510331
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.508110
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.507899
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.514391
Train Epoch: 52 - Loss: 0.5102, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.506733
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.508493
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.507580
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509539
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.511543
Train Epoch: 53 - Loss: 0.5101, Accuracy: 99.94%

Test set: Average loss: 0.5191, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.508686
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.510345
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.507071
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.506811
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.507902
Train Epoch: 54 - Loss: 0.5097, Accuracy: 99.94%

Test set: Average loss: 0.5187, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.509955
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.510924
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.510027
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.507551
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.513429
Train Epoch: 55 - Loss: 0.5094, Accuracy: 99.94%

Test set: Average loss: 0.5187, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.508458
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.509169
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.507801
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.509616
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.509759
Train Epoch: 56 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5195, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.505692
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.506412
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.509306
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.508126
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.509962
Train Epoch: 57 - Loss: 0.5089, Accuracy: 99.95%

Test set: Average loss: 0.5180, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.509669
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.507279
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.510341
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.507246
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506133
Train Epoch: 58 - Loss: 0.5086, Accuracy: 99.96%

Test set: Average loss: 0.5181, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.507572
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.507481
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.508805
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.515692
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.509132
Train Epoch: 59 - Loss: 0.5084, Accuracy: 99.96%

Test set: Average loss: 0.5180, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.505822
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.505961
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.505178
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.511042
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.511852
Train Epoch: 60 - Loss: 0.5082, Accuracy: 99.96%

Test set: Average loss: 0.5179, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.505759
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.506270
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.508085
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.510155
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.506074
Train Epoch: 61 - Loss: 0.5080, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.505486
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.507535
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.504899
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.506710
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.508472
Train Epoch: 62 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.510085
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.516358
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.509231
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510649
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.505815
Train Epoch: 63 - Loss: 0.5076, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.507217
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.505632
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.515762
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.505505
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.507668
Train Epoch: 64 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.505470
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509958
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.504944
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.505648
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506256
Train Epoch: 65 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5173, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.505370
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.505759
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508781
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505917
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506612
Train Epoch: 66 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.505558
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.507224
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.513484
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.506059
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.505016
Train Epoch: 67 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.503950
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.505130
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.518681
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.506694
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.504185
Train Epoch: 68 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.506232
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.505342
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507703
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.509667
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506057
Train Epoch: 69 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.509199
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.507491
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.506397
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.505977
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507727
Train Epoch: 70 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.509393
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.505370
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.505946
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.505517
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.509361
Train Epoch: 71 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.506896
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.505885
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509875
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.505865
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.505765
Train Epoch: 72 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.504688
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.506564
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.504831
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.508069
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.506449
Train Epoch: 73 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.504667
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.507372
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.504862
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.504699
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.506311
Train Epoch: 74 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.507220
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.504961
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.505992
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.505444
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506540
Train Epoch: 75 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.507542
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.506135
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.507990
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.505758
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.533762
Train Epoch: 76 - Loss: 0.5075, Accuracy: 99.96%

Test set: Average loss: 0.5173, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511462
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.507164
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.504332
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.505215
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.504844
Train Epoch: 77 - Loss: 0.5076, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505399
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.508304
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.507677
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.505443
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.506130
Train Epoch: 78 - Loss: 0.5074, Accuracy: 99.96%

Test set: Average loss: 0.5174, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.505930
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.505063
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.504645
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.505627
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.504690
Train Epoch: 79 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.506841
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.506268
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.504542
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.504541
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.503607
Train Epoch: 80 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5176, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.504773
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.505737
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.507397
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.510809
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.510543
Train Epoch: 81 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.503893
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.505110
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.506994
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.505467
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.506266
Train Epoch: 82 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.507496
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.509077
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.504427
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.503518
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.504586
Train Epoch: 83 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.505559
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.504965
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.506835
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.504607
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.507317
Train Epoch: 84 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510170
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.505705
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.504147
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.507136
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509049
Train Epoch: 85 - Loss: 0.5062, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510791
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.503495
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.507479
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.505064
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.505852
Train Epoch: 86 - Loss: 0.5061, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.504915
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.505270
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505268
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.503604
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.528465
Train Epoch: 87 - Loss: 0.5059, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.505814
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.506217
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505332
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.504248
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505329
Train Epoch: 88 - Loss: 0.5058, Accuracy: 99.99%

Test set: Average loss: 0.5166, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.503326
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505130
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.504293
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.505378
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.506602
Train Epoch: 89 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.505625
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.504891
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.505680
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.507783
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509422
Train Epoch: 90 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.508237
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.505284
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509497
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.504880
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.509399
Train Epoch: 91 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.507267
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.503926
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.504808
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.505161
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.509921
Train Epoch: 92 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506372
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504727
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.507690
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.505997
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504162
Train Epoch: 93 - Loss: 0.5055, Accuracy: 100.00%

Test set: Average loss: 0.5163, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.504257
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504596
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.506592
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.505772
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.503815
Train Epoch: 94 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504542
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.507603
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.503993
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.504737
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.503256
Train Epoch: 95 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507183
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.504704
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.503268
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.504264
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.505843
Train Epoch: 96 - Loss: 0.5053, Accuracy: 100.00%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.503464
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.504965
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.505689
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.503950
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.505105
Train Epoch: 97 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506182
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.503346
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505618
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.505253
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.504934
Train Epoch: 98 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506897
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504925
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506060
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503582
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503656
Train Epoch: 99 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.504816
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.505387
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.504202
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504343
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505294
Train Epoch: 100 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9952/10000 (99.52%)


Training completed. Best Accuracy: 99.54%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.584444
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.771998
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.702558
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.627629
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.592964
Train Epoch: 1 - Loss: 0.7525, Accuracy: 93.30%

Test set: Average loss: 0.5852, Accuracy: 9808/10000 (98.08%)

New best model saved with accuracy: 98.08%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.575383
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.573909
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.561199
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.566494
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.543221
Train Epoch: 2 - Loss: 0.5773, Accuracy: 98.29%

Test set: Average loss: 0.5576, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.569552
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.587060
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.539895
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.538071
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.548465
Train Epoch: 3 - Loss: 0.5547, Accuracy: 98.85%

Test set: Average loss: 0.5454, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.601896
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.548310
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.552263
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.551867
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.566875
Train Epoch: 4 - Loss: 0.5445, Accuracy: 99.08%

Test set: Average loss: 0.5406, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.524311
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.574910
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.521606
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.532569
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.553071
Train Epoch: 5 - Loss: 0.5376, Accuracy: 99.28%

Test set: Average loss: 0.5359, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.525764
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.518355
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.527714
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.525477
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.528803
Train Epoch: 6 - Loss: 0.5326, Accuracy: 99.38%

Test set: Average loss: 0.5335, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.527373
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.549532
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.520299
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.512533
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.548390
Train Epoch: 7 - Loss: 0.5292, Accuracy: 99.50%

Test set: Average loss: 0.5302, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.523753
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.543874
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.531204
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.519413
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.518269
Train Epoch: 8 - Loss: 0.5266, Accuracy: 99.54%

Test set: Average loss: 0.5289, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.516207
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.540353
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.522877
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.526241
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.525934
Train Epoch: 9 - Loss: 0.5247, Accuracy: 99.58%

Test set: Average loss: 0.5280, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.515965
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.518344
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.536928
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.526346
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.511118
Train Epoch: 10 - Loss: 0.5228, Accuracy: 99.63%

Test set: Average loss: 0.5276, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.520608
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.514159
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.522434
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.524608
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.515454
Train Epoch: 11 - Loss: 0.5218, Accuracy: 99.67%

Test set: Average loss: 0.5262, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.516839
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.510973
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.533787
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.518328
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.517168
Train Epoch: 12 - Loss: 0.5200, Accuracy: 99.73%

Test set: Average loss: 0.5257, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.516493
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.512667
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.528489
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.511907
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.524496
Train Epoch: 13 - Loss: 0.5192, Accuracy: 99.75%

Test set: Average loss: 0.5250, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.516834
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.522285
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.518764
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.510813
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.542430
Train Epoch: 14 - Loss: 0.5183, Accuracy: 99.77%

Test set: Average loss: 0.5242, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.509237
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.531709
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.519862
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.522627
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.514921
Train Epoch: 15 - Loss: 0.5177, Accuracy: 99.78%

Test set: Average loss: 0.5239, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.510870
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.520014
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.516251
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.513793
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.524388
Train Epoch: 16 - Loss: 0.5171, Accuracy: 99.80%

Test set: Average loss: 0.5235, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.515892
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.513814
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.511724
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.514306
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.509865
Train Epoch: 17 - Loss: 0.5165, Accuracy: 99.80%

Test set: Average loss: 0.5237, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.509747
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.526524
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.510881
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.522878
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.509826
Train Epoch: 18 - Loss: 0.5163, Accuracy: 99.81%

Test set: Average loss: 0.5230, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.510162
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.510918
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.511737
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.518734
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515231
Train Epoch: 19 - Loss: 0.5158, Accuracy: 99.83%

Test set: Average loss: 0.5234, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.508823
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.515691
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.512455
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.509071
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.509980
Train Epoch: 20 - Loss: 0.5158, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.528369
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.510092
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.515055
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.513892
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.512453
Train Epoch: 21 - Loss: 0.5155, Accuracy: 99.83%

Test set: Average loss: 0.5230, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.512164
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.518277
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.512943
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.511722
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.513742
Train Epoch: 22 - Loss: 0.5156, Accuracy: 99.81%

Test set: Average loss: 0.5231, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.511934
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.521818
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.515393
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.534427
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.527472
Train Epoch: 23 - Loss: 0.5153, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.523412
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.509813
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.522000
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.516617
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.515306
Train Epoch: 24 - Loss: 0.5151, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.512121
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.514588
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.525832
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.509552
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.518964
Train Epoch: 25 - Loss: 0.5150, Accuracy: 99.83%

Test set: Average loss: 0.5228, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.527198
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.519024
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.513415
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.512083
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.515857
Train Epoch: 26 - Loss: 0.5172, Accuracy: 99.77%

Test set: Average loss: 0.5234, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.524196
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.521309
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.528282
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.511492
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.513799
Train Epoch: 27 - Loss: 0.5163, Accuracy: 99.80%

Test set: Average loss: 0.5226, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.508544
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.513844
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.512941
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.508950
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.512576
Train Epoch: 28 - Loss: 0.5157, Accuracy: 99.81%

Test set: Average loss: 0.5231, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.514280
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519184
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.512192
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.515053
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.508346
Train Epoch: 29 - Loss: 0.5149, Accuracy: 99.82%

Test set: Average loss: 0.5223, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.510564
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.510909
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.510842
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.509561
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.521627
Train Epoch: 30 - Loss: 0.5143, Accuracy: 99.83%

Test set: Average loss: 0.5223, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.512254
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.511926
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.509723
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.505529
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.541348
Train Epoch: 31 - Loss: 0.5132, Accuracy: 99.86%

Test set: Average loss: 0.5219, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.511982
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.511391
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.509855
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.507819
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.510076
Train Epoch: 32 - Loss: 0.5126, Accuracy: 99.88%

Test set: Average loss: 0.5220, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.517774
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.514626
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.507019
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.508765
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.508101
Train Epoch: 33 - Loss: 0.5123, Accuracy: 99.87%

Test set: Average loss: 0.5205, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.509110
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.510007
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.511518
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.506917
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.512327
Train Epoch: 34 - Loss: 0.5122, Accuracy: 99.86%

Test set: Average loss: 0.5215, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.509082
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.514491
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.511935
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.507797
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.506708
Train Epoch: 35 - Loss: 0.5116, Accuracy: 99.89%

Test set: Average loss: 0.5207, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.508706
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.507988
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.505024
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.512468
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.517557
Train Epoch: 36 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5205, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.507255
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.510707
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.513981
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.506376
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.508659
Train Epoch: 37 - Loss: 0.5106, Accuracy: 99.91%

Test set: Average loss: 0.5197, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.506297
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.510524
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.512303
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.504843
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.509114
Train Epoch: 38 - Loss: 0.5102, Accuracy: 99.93%

Test set: Average loss: 0.5207, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.510785
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.517718
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.515457
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.505857
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.506259
Train Epoch: 39 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5197, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.507508
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.510178
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.517769
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.510790
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.510334
Train Epoch: 40 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.510676
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.511586
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.506175
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.512506
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.512309
Train Epoch: 41 - Loss: 0.5096, Accuracy: 99.92%

Test set: Average loss: 0.5203, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.507430
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.513211
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.510021
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.505358
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.509003
Train Epoch: 42 - Loss: 0.5096, Accuracy: 99.93%

Test set: Average loss: 0.5198, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.509075
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.506523
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.508052
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.505502
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.510475
Train Epoch: 43 - Loss: 0.5095, Accuracy: 99.94%

Test set: Average loss: 0.5198, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.506712
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.515380
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.509553
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.505000
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.506694
Train Epoch: 44 - Loss: 0.5093, Accuracy: 99.95%

Test set: Average loss: 0.5196, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.508610
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.509345
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.508962
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.504036
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.511650
Train Epoch: 45 - Loss: 0.5092, Accuracy: 99.94%

Test set: Average loss: 0.5195, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.505519
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.506444
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.508883
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.506452
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.510543
Train Epoch: 46 - Loss: 0.5089, Accuracy: 99.95%

Test set: Average loss: 0.5196, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.510868
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.508092
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.512363
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.508030
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.508083
Train Epoch: 47 - Loss: 0.5089, Accuracy: 99.95%

Test set: Average loss: 0.5197, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.511327
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.506930
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.504971
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.511196
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.505939
Train Epoch: 48 - Loss: 0.5088, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.507890
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.507286
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.506908
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.506919
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.506504
Train Epoch: 49 - Loss: 0.5088, Accuracy: 99.95%

Test set: Average loss: 0.5197, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.509993
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.507781
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.512780
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512431
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.520093
Train Epoch: 50 - Loss: 0.5089, Accuracy: 99.94%

Test set: Average loss: 0.5194, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.505363
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.506699
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.505722
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.515920
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.507589
Train Epoch: 51 - Loss: 0.5100, Accuracy: 99.92%

Test set: Average loss: 0.5199, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.506226
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.508378
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.508657
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.508994
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.506477
Train Epoch: 52 - Loss: 0.5099, Accuracy: 99.91%

Test set: Average loss: 0.5215, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.510125
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.507591
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.507293
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.508776
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.532087
Train Epoch: 53 - Loss: 0.5098, Accuracy: 99.92%

Test set: Average loss: 0.5209, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.508938
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.511153
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.504319
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.507244
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.512352
Train Epoch: 54 - Loss: 0.5094, Accuracy: 99.92%

Test set: Average loss: 0.5198, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.515585
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.512173
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.509278
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.509149
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.506414
Train Epoch: 55 - Loss: 0.5091, Accuracy: 99.93%

Test set: Average loss: 0.5203, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.507333
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.509892
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.506557
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.512841
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.512478
Train Epoch: 56 - Loss: 0.5086, Accuracy: 99.95%

Test set: Average loss: 0.5195, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.506911
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.507389
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.509088
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.511879
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.504942
Train Epoch: 57 - Loss: 0.5085, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.510715
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.508845
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.509052
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.509674
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506700
Train Epoch: 58 - Loss: 0.5080, Accuracy: 99.97%

Test set: Average loss: 0.5193, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.504638
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.515068
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.511471
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.506557
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.507137
Train Epoch: 59 - Loss: 0.5079, Accuracy: 99.95%

Test set: Average loss: 0.5187, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.505611
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.504562
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.510728
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.507527
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.504346
Train Epoch: 60 - Loss: 0.5080, Accuracy: 99.96%

Test set: Average loss: 0.5188, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.505633
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.505665
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.506242
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.503577
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.512048
Train Epoch: 61 - Loss: 0.5074, Accuracy: 99.96%

Test set: Average loss: 0.5185, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.503224
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.505639
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.503932
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.505194
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.508059
Train Epoch: 62 - Loss: 0.5074, Accuracy: 99.95%

Test set: Average loss: 0.5189, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.504371
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506022
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.505243
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510259
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.503709
Train Epoch: 63 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.509962
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.505052
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.508685
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.506447
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.506644
Train Epoch: 64 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.508751
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.514261
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.509768
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.507409
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.510166
Train Epoch: 65 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5188, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.509060
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.505187
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.504699
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505633
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506319
Train Epoch: 66 - Loss: 0.5069, Accuracy: 99.97%

Test set: Average loss: 0.5184, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.505563
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.506076
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.507750
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.510877
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506332
Train Epoch: 67 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5184, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.509196
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.506065
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.504752
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.503769
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.508260
Train Epoch: 68 - Loss: 0.5065, Accuracy: 99.97%

Test set: Average loss: 0.5182, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.506361
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.505329
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.504706
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.504834
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.504619
Train Epoch: 69 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5180, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.506908
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.504613
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.506316
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.506048
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.505622
Train Epoch: 70 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5182, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.505575
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.508752
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.505695
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.507632
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.503635
Train Epoch: 71 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5182, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.505592
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.508836
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.505474
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507670
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.506021
Train Epoch: 72 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5181, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.504598
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.505759
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508087
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505398
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.528369
Train Epoch: 73 - Loss: 0.5061, Accuracy: 99.99%

Test set: Average loss: 0.5180, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.503966
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.505539
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.507095
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.505145
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.505053
Train Epoch: 74 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5184, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.505635
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.528253
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.505324
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506098
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.504902
Train Epoch: 75 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5185, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.506007
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.508487
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.509004
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.505404
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.504765
Train Epoch: 76 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5186, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511590
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.506456
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.504892
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511839
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.505294
Train Epoch: 77 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5189, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505692
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.505465
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.505632
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.507224
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.504836
Train Epoch: 78 - Loss: 0.5068, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.510229
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.504771
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.507025
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507239
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.507029
Train Epoch: 79 - Loss: 0.5067, Accuracy: 99.97%

Test set: Average loss: 0.5189, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504420
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.504826
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.507327
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.506368
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.504994
Train Epoch: 80 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5178, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.506803
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.505634
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.503454
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.503849
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.507377
Train Epoch: 81 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5180, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.507260
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.505680
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.504590
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.506632
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.506494
Train Epoch: 82 - Loss: 0.5063, Accuracy: 99.97%

Test set: Average loss: 0.5186, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.506538
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.507521
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506586
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.504192
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.505783
Train Epoch: 83 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5178, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.508336
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.505317
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.505073
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.504330
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.505949
Train Epoch: 84 - Loss: 0.5060, Accuracy: 99.97%

Test set: Average loss: 0.5180, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.504971
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.505694
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.505583
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.505746
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.506244
Train Epoch: 85 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5183, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.505455
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.504133
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.504625
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.507715
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.503525
Train Epoch: 86 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5180, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.503633
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.505157
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.507308
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.510365
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.505321
Train Epoch: 87 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5177, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.503408
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507379
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505120
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.503793
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.506403
Train Epoch: 88 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5178, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.504754
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.511953
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.504720
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.504316
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.507357
Train Epoch: 89 - Loss: 0.5054, Accuracy: 99.98%

Test set: Average loss: 0.5178, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.505113
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.504078
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.505889
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.506778
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.506293
Train Epoch: 90 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5176, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.503672
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.505533
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.506396
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.505637
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506818
Train Epoch: 91 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5175, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504878
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.503437
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.503557
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.504701
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.506928
Train Epoch: 92 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5176, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.508225
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.503760
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.510819
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.504236
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504097
Train Epoch: 93 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5177, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.503803
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504539
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.504039
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.506225
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.506018
Train Epoch: 94 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5177, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506507
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.503549
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.504628
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505956
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.505543
Train Epoch: 95 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5179, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.504951
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.503795
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507820
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.503895
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.504494
Train Epoch: 96 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5174, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.503437
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.504983
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.503725
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.507596
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504318
Train Epoch: 97 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5176, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.503129
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.505269
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.504285
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.502923
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.504798
Train Epoch: 98 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5177, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506923
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.503464
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.505388
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503651
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503483
Train Epoch: 99 - Loss: 0.5048, Accuracy: 100.00%

Test set: Average loss: 0.5175, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.503050
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.504593
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.503991
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.503956
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.506618
Train Epoch: 100 - Loss: 0.5047, Accuracy: 100.00%

Test set: Average loss: 0.5174, Accuracy: 9937/10000 (99.37%)


Training completed. Best Accuracy: 99.40%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.462586
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.742315
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.667745
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.661510
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.615398
Train Epoch: 1 - Loss: 0.7479, Accuracy: 93.30%

Test set: Average loss: 0.5871, Accuracy: 9810/10000 (98.10%)

New best model saved with accuracy: 98.10%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.621070
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.569953
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.558898
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.535886
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.545850
Train Epoch: 2 - Loss: 0.5775, Accuracy: 98.30%

Test set: Average loss: 0.5577, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.543869
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.547400
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.584587
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.537838
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.537848
Train Epoch: 3 - Loss: 0.5559, Accuracy: 98.83%

Test set: Average loss: 0.5460, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.549604
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.567138
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.527628
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.564194
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.539067
Train Epoch: 4 - Loss: 0.5448, Accuracy: 99.08%

Test set: Average loss: 0.5398, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.549872
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.527708
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.535917
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.529405
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.526667
Train Epoch: 5 - Loss: 0.5378, Accuracy: 99.23%

Test set: Average loss: 0.5344, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.525034
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.521640
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.525362
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.525869
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.534453
Train Epoch: 6 - Loss: 0.5332, Accuracy: 99.37%

Test set: Average loss: 0.5320, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.521259
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.544748
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.517843
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.535696
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.550191
Train Epoch: 7 - Loss: 0.5298, Accuracy: 99.45%

Test set: Average loss: 0.5308, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.517178
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.532274
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.515755
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.519288
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.517587
Train Epoch: 8 - Loss: 0.5269, Accuracy: 99.52%

Test set: Average loss: 0.5285, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.523975
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.521352
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.525872
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.512263
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.515426
Train Epoch: 9 - Loss: 0.5249, Accuracy: 99.58%

Test set: Average loss: 0.5270, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.519218
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.525306
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.518453
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.523863
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.537708
Train Epoch: 10 - Loss: 0.5230, Accuracy: 99.65%

Test set: Average loss: 0.5252, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.518882
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.512922
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.523184
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.514627
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.530809
Train Epoch: 11 - Loss: 0.5217, Accuracy: 99.68%

Test set: Average loss: 0.5256, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.514997
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.512997
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.510959
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.519557
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.530091
Train Epoch: 12 - Loss: 0.5202, Accuracy: 99.74%

Test set: Average loss: 0.5239, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.513589
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.515854
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.513324
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.516427
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.519313
Train Epoch: 13 - Loss: 0.5192, Accuracy: 99.76%

Test set: Average loss: 0.5233, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.510361
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.510085
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.511457
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.511316
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.509468
Train Epoch: 14 - Loss: 0.5183, Accuracy: 99.78%

Test set: Average loss: 0.5232, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.515896
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.508679
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517802
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.513938
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.513970
Train Epoch: 15 - Loss: 0.5177, Accuracy: 99.77%

Test set: Average loss: 0.5220, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.517123
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.539611
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.514664
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.516955
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.513711
Train Epoch: 16 - Loss: 0.5171, Accuracy: 99.80%

Test set: Average loss: 0.5224, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.525083
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.512364
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.511059
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.521238
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.512293
Train Epoch: 17 - Loss: 0.5164, Accuracy: 99.81%

Test set: Average loss: 0.5223, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.511355
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.508050
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.511353
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.516543
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.512469
Train Epoch: 18 - Loss: 0.5161, Accuracy: 99.83%

Test set: Average loss: 0.5220, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.508916
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.511546
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.515358
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.513980
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.518698
Train Epoch: 19 - Loss: 0.5158, Accuracy: 99.85%

Test set: Average loss: 0.5217, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.516085
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.513043
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.512526
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.512880
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.516514
Train Epoch: 20 - Loss: 0.5155, Accuracy: 99.84%

Test set: Average loss: 0.5217, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.511842
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.518624
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.514423
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.520150
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.513887
Train Epoch: 21 - Loss: 0.5152, Accuracy: 99.86%

Test set: Average loss: 0.5216, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.512923
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.514562
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.512536
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.508593
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.514869
Train Epoch: 22 - Loss: 0.5151, Accuracy: 99.86%

Test set: Average loss: 0.5214, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.512525
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.513147
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.513169
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.512615
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.509342
Train Epoch: 23 - Loss: 0.5148, Accuracy: 99.86%

Test set: Average loss: 0.5214, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.512592
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.514306
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.514400
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.509297
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.511021
Train Epoch: 24 - Loss: 0.5148, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.512193
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.519115
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.544324
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.511070
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.511350
Train Epoch: 25 - Loss: 0.5149, Accuracy: 99.87%

Test set: Average loss: 0.5215, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.514419
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.513080
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.511411
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.512536
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.514868
Train Epoch: 26 - Loss: 0.5172, Accuracy: 99.79%

Test set: Average loss: 0.5227, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.514146
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.517068
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.513796
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.511168
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.517794
Train Epoch: 27 - Loss: 0.5161, Accuracy: 99.82%

Test set: Average loss: 0.5235, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.510479
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.511949
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.512029
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.517250
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.514081
Train Epoch: 28 - Loss: 0.5153, Accuracy: 99.82%

Test set: Average loss: 0.5227, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.509932
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.540775
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.517240
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.526933
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.524745
Train Epoch: 29 - Loss: 0.5146, Accuracy: 99.83%

Test set: Average loss: 0.5219, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.508016
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.514540
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.511259
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.527793
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.513524
Train Epoch: 30 - Loss: 0.5140, Accuracy: 99.83%

Test set: Average loss: 0.5212, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.512497
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.508837
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.526306
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.508053
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.522279
Train Epoch: 31 - Loss: 0.5131, Accuracy: 99.87%

Test set: Average loss: 0.5233, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.509106
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.510663
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.517334
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.517218
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.507136
Train Epoch: 32 - Loss: 0.5125, Accuracy: 99.87%

Test set: Average loss: 0.5214, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.515611
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.515999
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.515969
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.510146
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.511449
Train Epoch: 33 - Loss: 0.5122, Accuracy: 99.89%

Test set: Average loss: 0.5194, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.509480
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.510561
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.509191
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.506005
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.508949
Train Epoch: 34 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5192, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.509739
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.508902
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.524808
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.508071
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.508464
Train Epoch: 35 - Loss: 0.5112, Accuracy: 99.91%

Test set: Average loss: 0.5189, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.511442
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509415
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.509408
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.514046
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.508116
Train Epoch: 36 - Loss: 0.5109, Accuracy: 99.90%

Test set: Average loss: 0.5196, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.506016
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.505799
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.504739
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.504921
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.512674
Train Epoch: 37 - Loss: 0.5103, Accuracy: 99.91%

Test set: Average loss: 0.5188, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.507914
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.507416
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.509877
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.513793
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.509748
Train Epoch: 38 - Loss: 0.5101, Accuracy: 99.94%

Test set: Average loss: 0.5184, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.510580
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.508056
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.508341
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.522425
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.506173
Train Epoch: 39 - Loss: 0.5097, Accuracy: 99.95%

Test set: Average loss: 0.5180, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.504851
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.505416
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.506903
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.505346
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.505369
Train Epoch: 40 - Loss: 0.5095, Accuracy: 99.94%

Test set: Average loss: 0.5180, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.506531
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.505741
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.510175
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.507497
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.508304
Train Epoch: 41 - Loss: 0.5091, Accuracy: 99.95%

Test set: Average loss: 0.5180, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.509978
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.506992
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.505975
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.510886
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.508644
Train Epoch: 42 - Loss: 0.5091, Accuracy: 99.94%

Test set: Average loss: 0.5177, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.509789
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.506912
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.511434
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.504330
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.518359
Train Epoch: 43 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5184, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.509067
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.519233
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.512281
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.505692
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.507036
Train Epoch: 44 - Loss: 0.5088, Accuracy: 99.95%

Test set: Average loss: 0.5178, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.508033
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.507458
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.506836
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.506117
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.508267
Train Epoch: 45 - Loss: 0.5088, Accuracy: 99.95%

Test set: Average loss: 0.5177, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.504896
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.507291
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.506769
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.504803
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.513830
Train Epoch: 46 - Loss: 0.5086, Accuracy: 99.96%

Test set: Average loss: 0.5178, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508579
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.505358
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.508155
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512868
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.504837
Train Epoch: 47 - Loss: 0.5085, Accuracy: 99.96%

Test set: Average loss: 0.5177, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.506558
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.511952
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513472
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.507627
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.508168
Train Epoch: 48 - Loss: 0.5085, Accuracy: 99.97%

Test set: Average loss: 0.5176, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.505944
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.505097
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.510427
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.508885
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.512209
Train Epoch: 49 - Loss: 0.5086, Accuracy: 99.96%

Test set: Average loss: 0.5178, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.504963
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.506839
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.508066
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.505260
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.506944
Train Epoch: 50 - Loss: 0.5085, Accuracy: 99.96%

Test set: Average loss: 0.5176, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.506589
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.506325
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.505164
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.505771
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.507763
Train Epoch: 51 - Loss: 0.5097, Accuracy: 99.93%

Test set: Average loss: 0.5182, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.508235
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.506184
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.511991
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.510736
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.505608
Train Epoch: 52 - Loss: 0.5096, Accuracy: 99.93%

Test set: Average loss: 0.5184, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.506518
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.507525
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.505374
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509189
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.517198
Train Epoch: 53 - Loss: 0.5092, Accuracy: 99.93%

Test set: Average loss: 0.5187, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.509053
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.515627
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.511204
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.509772
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.521240
Train Epoch: 54 - Loss: 0.5089, Accuracy: 99.95%

Test set: Average loss: 0.5181, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.507440
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.508865
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.508247
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.506986
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.510401
Train Epoch: 55 - Loss: 0.5087, Accuracy: 99.94%

Test set: Average loss: 0.5180, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.507571
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.507734
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.509406
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.512744
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.511743
Train Epoch: 56 - Loss: 0.5085, Accuracy: 99.94%

Test set: Average loss: 0.5172, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.506533
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.510695
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.518292
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.506656
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.506829
Train Epoch: 57 - Loss: 0.5083, Accuracy: 99.95%

Test set: Average loss: 0.5178, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.508757
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.506605
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.506217
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.505575
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.510085
Train Epoch: 58 - Loss: 0.5079, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.508419
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.506869
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.507172
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.508752
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.505104
Train Epoch: 59 - Loss: 0.5077, Accuracy: 99.96%

Test set: Average loss: 0.5177, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.506248
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.505332
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.505638
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.504859
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515314
Train Epoch: 60 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5168, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.510608
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.505992
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.506435
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.507082
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.510608
Train Epoch: 61 - Loss: 0.5072, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.505590
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.504609
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.506835
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.508154
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.510531
Train Epoch: 62 - Loss: 0.5069, Accuracy: 99.97%

Test set: Average loss: 0.5167, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.506812
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506073
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.508000
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.504628
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.505047
Train Epoch: 63 - Loss: 0.5068, Accuracy: 99.97%

Test set: Average loss: 0.5166, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.506652
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.505124
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.505757
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.504974
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.505131
Train Epoch: 64 - Loss: 0.5067, Accuracy: 99.97%

Test set: Average loss: 0.5166, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.507860
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.508795
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.504575
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.506980
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.504025
Train Epoch: 65 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5163, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.505938
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.506072
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.505087
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.506296
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.505157
Train Epoch: 66 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5164, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.505432
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.504694
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.504930
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.503594
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.504135
Train Epoch: 67 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.504559
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.506225
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.506898
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.506562
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.505218
Train Epoch: 68 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5160, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.504408
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.515030
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.505015
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.507367
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.504656
Train Epoch: 69 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5161, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.504906
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.504492
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.504763
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.506588
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.504000
Train Epoch: 70 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5161, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.503732
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.505681
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.506685
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.504535
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.506524
Train Epoch: 71 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5161, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.505169
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.504997
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.503559
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507045
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.505508
Train Epoch: 72 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5161, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505812
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.504342
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.511837
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505264
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.508348
Train Epoch: 73 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5159, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.506178
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508682
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.505828
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.503782
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.504444
Train Epoch: 74 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.504615
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.503937
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.504223
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.505215
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506788
Train Epoch: 75 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5160, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.504256
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.504559
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.506278
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.505842
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.504157
Train Epoch: 76 - Loss: 0.5069, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.505597
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.506375
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.528270
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.506418
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.508023
Train Epoch: 77 - Loss: 0.5068, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505719
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.506320
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.505822
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.504783
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.507140
Train Epoch: 78 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.504840
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.505948
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.506419
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.505956
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.503154
Train Epoch: 79 - Loss: 0.5064, Accuracy: 99.97%

Test set: Average loss: 0.5165, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504952
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.503938
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.528980
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.503763
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.507225
Train Epoch: 80 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.510345
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.504392
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.506378
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.506404
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.505032
Train Epoch: 81 - Loss: 0.5063, Accuracy: 99.97%

Test set: Average loss: 0.5158, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.503658
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.505834
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.503952
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.505926
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.505716
Train Epoch: 82 - Loss: 0.5060, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.503323
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.504318
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.505904
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506048
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.504125
Train Epoch: 83 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.504461
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.507294
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.504565
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.503725
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.503599
Train Epoch: 84 - Loss: 0.5056, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.503781
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.504147
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.506036
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.513167
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.505203
Train Epoch: 85 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.504473
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.504352
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.503240
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.505076
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.504891
Train Epoch: 86 - Loss: 0.5052, Accuracy: 99.98%

Test set: Average loss: 0.5158, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.504948
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.503642
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.506931
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509371
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.506015
Train Epoch: 87 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.503400
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.504848
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.503708
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.508484
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505867
Train Epoch: 88 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.503422
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505296
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507848
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.503160
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.506009
Train Epoch: 89 - Loss: 0.5051, Accuracy: 99.98%

Test set: Average loss: 0.5157, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.503960
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.504692
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.504612
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.504812
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.505170
Train Epoch: 90 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5153, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504607
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.503381
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.504510
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.505160
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.503391
Train Epoch: 91 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.502912
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504658
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.502896
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507798
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.503342
Train Epoch: 92 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5155, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.503982
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.503297
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.504327
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.503384
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505500
Train Epoch: 93 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.504818
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.503007
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.503953
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.504733
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.503937
Train Epoch: 94 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5153, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.507328
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.502889
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.505223
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.504300
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.503811
Train Epoch: 95 - Loss: 0.5047, Accuracy: 99.99%

Test set: Average loss: 0.5157, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.503927
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.504215
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.504306
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.503723
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.503177
Train Epoch: 96 - Loss: 0.5047, Accuracy: 99.99%

Test set: Average loss: 0.5155, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.502902
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.503917
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507575
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.504778
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504144
Train Epoch: 97 - Loss: 0.5047, Accuracy: 99.99%

Test set: Average loss: 0.5153, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.504099
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.504377
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505172
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.504409
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.503664
Train Epoch: 98 - Loss: 0.5046, Accuracy: 99.99%

Test set: Average loss: 0.5155, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.504768
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.503934
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.503982
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.502916
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503812
Train Epoch: 99 - Loss: 0.5046, Accuracy: 99.99%

Test set: Average loss: 0.5154, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.503696
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.504851
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.503812
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504286
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.506261
Train Epoch: 100 - Loss: 0.5046, Accuracy: 99.99%

Test set: Average loss: 0.5153, Accuracy: 9943/10000 (99.43%)


Training completed. Best Accuracy: 99.48%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.454933
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.785361
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.685565
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.657657
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.626029
Train Epoch: 1 - Loss: 0.7853, Accuracy: 92.65%

Test set: Average loss: 0.5957, Accuracy: 9803/10000 (98.03%)

New best model saved with accuracy: 98.03%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.595727
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.597125
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.570049
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.600745
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.574869
Train Epoch: 2 - Loss: 0.5825, Accuracy: 98.28%

Test set: Average loss: 0.5661, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.551295
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.581363
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.552197
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.541064
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.537706
Train Epoch: 3 - Loss: 0.5565, Accuracy: 98.94%

Test set: Average loss: 0.5470, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.545295
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.550646
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.550177
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.526914
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.531642
Train Epoch: 4 - Loss: 0.5451, Accuracy: 99.18%

Test set: Average loss: 0.5406, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.533709
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.550749
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.529099
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.541161
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.527889
Train Epoch: 5 - Loss: 0.5378, Accuracy: 99.33%

Test set: Average loss: 0.5366, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.532942
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.528568
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.555032
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.557375
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.524716
Train Epoch: 6 - Loss: 0.5325, Accuracy: 99.46%

Test set: Average loss: 0.5314, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.525185
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.540188
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.553248
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.518180
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.527637
Train Epoch: 7 - Loss: 0.5294, Accuracy: 99.50%

Test set: Average loss: 0.5319, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.529246
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.524388
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.519287
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.525173
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.524050
Train Epoch: 8 - Loss: 0.5264, Accuracy: 99.59%

Test set: Average loss: 0.5293, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.541809
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.515128
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.518981
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.515781
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.515483
Train Epoch: 9 - Loss: 0.5244, Accuracy: 99.61%

Test set: Average loss: 0.5271, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.541164
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.517304
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.536916
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.515095
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.518461
Train Epoch: 10 - Loss: 0.5226, Accuracy: 99.69%

Test set: Average loss: 0.5271, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.545430
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.516774
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.526781
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.517650
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.521975
Train Epoch: 11 - Loss: 0.5216, Accuracy: 99.70%

Test set: Average loss: 0.5260, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.515313
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.522522
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.529997
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.516981
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.509413
Train Epoch: 12 - Loss: 0.5201, Accuracy: 99.74%

Test set: Average loss: 0.5240, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.518514
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.520910
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.512787
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.516594
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.513378
Train Epoch: 13 - Loss: 0.5190, Accuracy: 99.77%

Test set: Average loss: 0.5243, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.523106
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.512822
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.528152
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.518065
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.522517
Train Epoch: 14 - Loss: 0.5184, Accuracy: 99.77%

Test set: Average loss: 0.5234, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.524746
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.523845
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.524402
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.516955
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.525838
Train Epoch: 15 - Loss: 0.5176, Accuracy: 99.83%

Test set: Average loss: 0.5232, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.516811
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.511989
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.530366
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.519666
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.519117
Train Epoch: 16 - Loss: 0.5168, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.514015
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.515510
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.519137
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.516592
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.519824
Train Epoch: 17 - Loss: 0.5166, Accuracy: 99.84%

Test set: Average loss: 0.5227, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.509952
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.512985
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.512017
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.511387
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.514615
Train Epoch: 18 - Loss: 0.5159, Accuracy: 99.84%

Test set: Average loss: 0.5228, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.515266
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.511459
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.530458
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.508891
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.510851
Train Epoch: 19 - Loss: 0.5160, Accuracy: 99.84%

Test set: Average loss: 0.5223, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.517485
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.509086
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.513781
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.533037
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.515296
Train Epoch: 20 - Loss: 0.5154, Accuracy: 99.87%

Test set: Average loss: 0.5226, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.510777
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.514511
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.513286
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.514340
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.508859
Train Epoch: 21 - Loss: 0.5153, Accuracy: 99.88%

Test set: Average loss: 0.5221, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.513352
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.518559
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.516481
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.514559
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.511968
Train Epoch: 22 - Loss: 0.5151, Accuracy: 99.90%

Test set: Average loss: 0.5224, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.516052
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.508810
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.526289
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.510488
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.523472
Train Epoch: 23 - Loss: 0.5151, Accuracy: 99.89%

Test set: Average loss: 0.5222, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.525376
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.514165
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.507775
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.510571
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.513915
Train Epoch: 24 - Loss: 0.5148, Accuracy: 99.89%

Test set: Average loss: 0.5225, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.513366
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.526807
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.522790
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.511434
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.519458
Train Epoch: 25 - Loss: 0.5149, Accuracy: 99.88%

Test set: Average loss: 0.5223, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.510814
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.519934
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.514729
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.517292
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.518947
Train Epoch: 26 - Loss: 0.5171, Accuracy: 99.80%

Test set: Average loss: 0.5232, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.520940
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.510483
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.516861
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.522969
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.517084
Train Epoch: 27 - Loss: 0.5161, Accuracy: 99.82%

Test set: Average loss: 0.5227, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.510172
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.511789
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.517160
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.510215
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.519044
Train Epoch: 28 - Loss: 0.5157, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.511165
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.542182
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.522695
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.525618
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.511714
Train Epoch: 29 - Loss: 0.5145, Accuracy: 99.86%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.516069
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.509186
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.512275
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.512464
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.514602
Train Epoch: 30 - Loss: 0.5139, Accuracy: 99.87%

Test set: Average loss: 0.5218, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.515396
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.508892
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.511245
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.519072
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.508364
Train Epoch: 31 - Loss: 0.5133, Accuracy: 99.88%

Test set: Average loss: 0.5206, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.534791
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.510383
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.515285
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.509585
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.509536
Train Epoch: 32 - Loss: 0.5128, Accuracy: 99.89%

Test set: Average loss: 0.5208, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.519533
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.516064
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.511608
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.514523
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.518692
Train Epoch: 33 - Loss: 0.5121, Accuracy: 99.92%

Test set: Average loss: 0.5199, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.514099
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.512533
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.509299
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.523046
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.510890
Train Epoch: 34 - Loss: 0.5118, Accuracy: 99.91%

Test set: Average loss: 0.5199, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.508497
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.517893
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.512661
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.510281
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.509793
Train Epoch: 35 - Loss: 0.5112, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.508084
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.510736
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.505249
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.514410
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.510360
Train Epoch: 36 - Loss: 0.5109, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.509073
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.514084
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.512782
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.508100
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.514089
Train Epoch: 37 - Loss: 0.5106, Accuracy: 99.94%

Test set: Average loss: 0.5193, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.508755
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.507840
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.511758
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.517706
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.507235
Train Epoch: 38 - Loss: 0.5103, Accuracy: 99.94%

Test set: Average loss: 0.5193, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.514289
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.505564
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.507667
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.506280
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.505390
Train Epoch: 39 - Loss: 0.5101, Accuracy: 99.94%

Test set: Average loss: 0.5186, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.515007
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.510868
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.512294
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.506366
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.516386
Train Epoch: 40 - Loss: 0.5101, Accuracy: 99.95%

Test set: Average loss: 0.5191, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.510763
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.508228
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.508256
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.512878
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.509239
Train Epoch: 41 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5188, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.506956
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.511304
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.530874
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.508872
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.515940
Train Epoch: 42 - Loss: 0.5093, Accuracy: 99.95%

Test set: Average loss: 0.5186, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.506157
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.507444
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.508143
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.508223
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.510626
Train Epoch: 43 - Loss: 0.5092, Accuracy: 99.97%

Test set: Average loss: 0.5185, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.508142
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.506683
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.506242
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.506676
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.508175
Train Epoch: 44 - Loss: 0.5091, Accuracy: 99.96%

Test set: Average loss: 0.5184, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.507039
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.506920
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.508937
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.509789
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.506944
Train Epoch: 45 - Loss: 0.5091, Accuracy: 99.95%

Test set: Average loss: 0.5184, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.509405
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.506577
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.508246
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.508210
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.507915
Train Epoch: 46 - Loss: 0.5090, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.505769
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.510120
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.508751
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.506416
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.508842
Train Epoch: 47 - Loss: 0.5090, Accuracy: 99.96%

Test set: Average loss: 0.5184, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.511954
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.505905
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.506518
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.507072
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.506447
Train Epoch: 48 - Loss: 0.5088, Accuracy: 99.97%

Test set: Average loss: 0.5184, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.510163
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.508568
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.508596
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.508047
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.506681
Train Epoch: 49 - Loss: 0.5088, Accuracy: 99.96%

Test set: Average loss: 0.5182, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.508714
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.506775
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.506923
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.507032
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.506918
Train Epoch: 50 - Loss: 0.5087, Accuracy: 99.97%

Test set: Average loss: 0.5184, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.507715
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.512702
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.505724
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.510163
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.505381
Train Epoch: 51 - Loss: 0.5099, Accuracy: 99.94%

Test set: Average loss: 0.5186, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.511376
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.510774
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.508231
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.512755
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.514191
Train Epoch: 52 - Loss: 0.5095, Accuracy: 99.94%

Test set: Average loss: 0.5188, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.506895
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.511059
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.518307
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.504977
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.505658
Train Epoch: 53 - Loss: 0.5092, Accuracy: 99.96%

Test set: Average loss: 0.5185, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.510010
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.512966
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.506590
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.508072
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.512953
Train Epoch: 54 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5183, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.507162
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.507274
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.509360
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.513015
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.506670
Train Epoch: 55 - Loss: 0.5090, Accuracy: 99.96%

Test set: Average loss: 0.5187, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.505894
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.505126
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.508709
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.507566
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.508098
Train Epoch: 56 - Loss: 0.5085, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.506652
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.508419
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.509260
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.504993
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.507997
Train Epoch: 57 - Loss: 0.5084, Accuracy: 99.96%

Test set: Average loss: 0.5182, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.509467
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.508814
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.509561
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.507388
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506476
Train Epoch: 58 - Loss: 0.5081, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.508245
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.508406
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.505478
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.505435
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.506515
Train Epoch: 59 - Loss: 0.5078, Accuracy: 99.97%

Test set: Average loss: 0.5176, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.504269
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.510238
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.508987
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.508209
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.506303
Train Epoch: 60 - Loss: 0.5076, Accuracy: 99.97%

Test set: Average loss: 0.5176, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.508803
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.506065
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.504412
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.506832
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.506041
Train Epoch: 61 - Loss: 0.5074, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.504588
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.508593
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.508758
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.505942
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.507534
Train Epoch: 62 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.506528
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506348
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.509372
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.507132
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.507947
Train Epoch: 63 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.507700
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.505299
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.504962
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.506520
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.508383
Train Epoch: 64 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.507733
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.505679
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.504359
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.505576
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508153
Train Epoch: 65 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.507757
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.505279
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.504564
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.511110
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506118
Train Epoch: 66 - Loss: 0.5068, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.513155
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.505894
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.503226
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.507325
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.505034
Train Epoch: 67 - Loss: 0.5066, Accuracy: 99.99%

Test set: Average loss: 0.5171, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.505183
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.504557
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.513521
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.508666
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.505916
Train Epoch: 68 - Loss: 0.5066, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.506243
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.503980
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.506141
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.507328
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.504402
Train Epoch: 69 - Loss: 0.5065, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.507250
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.505303
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.507277
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.505587
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507131
Train Epoch: 70 - Loss: 0.5065, Accuracy: 99.99%

Test set: Average loss: 0.5169, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.505776
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.504393
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.504883
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.526586
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.507489
Train Epoch: 71 - Loss: 0.5064, Accuracy: 99.99%

Test set: Average loss: 0.5171, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.508017
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509392
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.506157
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507110
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.506314
Train Epoch: 72 - Loss: 0.5063, Accuracy: 99.99%

Test set: Average loss: 0.5168, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505432
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.507084
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508063
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505906
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.507979
Train Epoch: 73 - Loss: 0.5062, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.505587
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.505200
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506312
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.506919
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.509127
Train Epoch: 74 - Loss: 0.5063, Accuracy: 99.99%

Test set: Average loss: 0.5169, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.509429
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.507098
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.503521
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506447
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.507335
Train Epoch: 75 - Loss: 0.5064, Accuracy: 99.99%

Test set: Average loss: 0.5169, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.505172
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.504332
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.506212
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.504698
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507317
Train Epoch: 76 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5173, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.505574
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.506285
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.506169
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.507995
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.505866
Train Epoch: 77 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5173, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.506178
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.504674
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.508404
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.507566
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.508007
Train Epoch: 78 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.505025
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.509825
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.510002
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.506133
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.504652
Train Epoch: 79 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504479
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.505961
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.513128
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.507084
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.521764
Train Epoch: 80 - Loss: 0.5067, Accuracy: 99.97%

Test set: Average loss: 0.5168, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.504325
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.506165
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.536750
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.506542
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.506175
Train Epoch: 81 - Loss: 0.5063, Accuracy: 99.99%

Test set: Average loss: 0.5171, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.505800
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.504185
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.505813
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.506264
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.508622
Train Epoch: 82 - Loss: 0.5060, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.503072
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.504595
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.504499
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.504441
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.506178
Train Epoch: 83 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5164, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.506730
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.504761
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508443
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.503701
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.515434
Train Epoch: 84 - Loss: 0.5059, Accuracy: 99.98%

Test set: Average loss: 0.5165, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.504363
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.504518
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.503925
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.507182
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.505055
Train Epoch: 85 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.503426
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.503530
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.506516
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506840
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.511689
Train Epoch: 86 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5167, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.506507
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506796
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505090
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.503969
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.504496
Train Epoch: 87 - Loss: 0.5056, Accuracy: 99.98%

Test set: Average loss: 0.5160, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.505708
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.503599
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505446
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.503225
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505875
Train Epoch: 88 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.503852
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505844
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.504515
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.504208
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.505648
Train Epoch: 89 - Loss: 0.5055, Accuracy: 99.98%

Test set: Average loss: 0.5164, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.503752
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.504418
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.506445
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.506657
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.504687
Train Epoch: 90 - Loss: 0.5053, Accuracy: 100.00%

Test set: Average loss: 0.5159, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504592
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.503590
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.505700
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.503618
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508245
Train Epoch: 91 - Loss: 0.5051, Accuracy: 100.00%

Test set: Average loss: 0.5161, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504604
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504233
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507419
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.502616
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.505186
Train Epoch: 92 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.505035
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504483
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.505683
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.506755
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504793
Train Epoch: 93 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.503098
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.506159
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.502955
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.504525
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.508511
Train Epoch: 94 - Loss: 0.5050, Accuracy: 100.00%

Test set: Average loss: 0.5161, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506109
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.503471
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.505488
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505393
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.504331
Train Epoch: 95 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.505138
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.507105
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.503879
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.503147
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.503321
Train Epoch: 96 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505155
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.505653
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.505938
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.503403
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504611
Train Epoch: 97 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.505478
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.507910
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505001
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.505085
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.503616
Train Epoch: 98 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506282
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504625
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506183
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503851
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.504638
Train Epoch: 99 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.505458
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.503373
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.503830
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.503103
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.502650
Train Epoch: 100 - Loss: 0.5049, Accuracy: 100.00%

Test set: Average loss: 0.5161, Accuracy: 9946/10000 (99.46%)


Training completed. Best Accuracy: 99.52%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.729763
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.799721
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.677777
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.614738
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.580631
Train Epoch: 1 - Loss: 0.7590, Accuracy: 93.24%

Test set: Average loss: 0.5881, Accuracy: 9824/10000 (98.24%)

New best model saved with accuracy: 98.24%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.582665
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.574353
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.639696
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.562962
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.573542
Train Epoch: 2 - Loss: 0.5773, Accuracy: 98.31%

Test set: Average loss: 0.5591, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.581661
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.559450
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.584879
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.540006
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.535158
Train Epoch: 3 - Loss: 0.5556, Accuracy: 98.88%

Test set: Average loss: 0.5474, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.539972
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.544600
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.556845
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.559034
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.521791
Train Epoch: 4 - Loss: 0.5440, Accuracy: 99.13%

Test set: Average loss: 0.5394, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.532405
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.542563
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.551059
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.543171
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.532047
Train Epoch: 5 - Loss: 0.5371, Accuracy: 99.33%

Test set: Average loss: 0.5353, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.531383
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.534942
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.540783
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.551087
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.538697
Train Epoch: 6 - Loss: 0.5324, Accuracy: 99.42%

Test set: Average loss: 0.5322, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.534924
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.546456
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.531620
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.526929
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.541239
Train Epoch: 7 - Loss: 0.5292, Accuracy: 99.48%

Test set: Average loss: 0.5294, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.523763
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.541769
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.513772
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.530008
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.517086
Train Epoch: 8 - Loss: 0.5268, Accuracy: 99.55%

Test set: Average loss: 0.5276, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.525423
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.540919
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.522807
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.523120
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.531092
Train Epoch: 9 - Loss: 0.5246, Accuracy: 99.61%

Test set: Average loss: 0.5268, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.520173
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.541180
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.516520
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.514134
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.528264
Train Epoch: 10 - Loss: 0.5227, Accuracy: 99.67%

Test set: Average loss: 0.5256, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.543026
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.513693
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.525545
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.520252
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.519442
Train Epoch: 11 - Loss: 0.5214, Accuracy: 99.70%

Test set: Average loss: 0.5254, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.513412
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.519148
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.514331
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.530246
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.509025
Train Epoch: 12 - Loss: 0.5200, Accuracy: 99.73%

Test set: Average loss: 0.5242, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.513878
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.512116
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.511177
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.518458
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.513314
Train Epoch: 13 - Loss: 0.5192, Accuracy: 99.75%

Test set: Average loss: 0.5247, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.519916
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.514653
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.509630
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.520303
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.515257
Train Epoch: 14 - Loss: 0.5181, Accuracy: 99.78%

Test set: Average loss: 0.5231, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.510175
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.516476
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517914
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.518782
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.514893
Train Epoch: 15 - Loss: 0.5176, Accuracy: 99.80%

Test set: Average loss: 0.5233, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.515350
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.527721
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.521349
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.511832
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.512851
Train Epoch: 16 - Loss: 0.5169, Accuracy: 99.81%

Test set: Average loss: 0.5225, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.513402
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.519014
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.522922
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.511854
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.513970
Train Epoch: 17 - Loss: 0.5166, Accuracy: 99.83%

Test set: Average loss: 0.5223, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.514418
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.513163
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.510164
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.509471
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.514997
Train Epoch: 18 - Loss: 0.5162, Accuracy: 99.83%

Test set: Average loss: 0.5225, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.512592
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.515840
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.521546
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.512867
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.509393
Train Epoch: 19 - Loss: 0.5158, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.510978
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.509470
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.511709
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.512160
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.512096
Train Epoch: 20 - Loss: 0.5156, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.511455
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.514054
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.510380
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.508047
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.512598
Train Epoch: 21 - Loss: 0.5154, Accuracy: 99.85%

Test set: Average loss: 0.5218, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.522026
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.516758
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.513172
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.511996
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.511761
Train Epoch: 22 - Loss: 0.5151, Accuracy: 99.87%

Test set: Average loss: 0.5220, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.514166
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.513182
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.511904
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.512456
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.508493
Train Epoch: 23 - Loss: 0.5152, Accuracy: 99.86%

Test set: Average loss: 0.5219, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.516385
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.513918
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.519848
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.511906
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.513379
Train Epoch: 24 - Loss: 0.5149, Accuracy: 99.87%

Test set: Average loss: 0.5219, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.511197
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.515705
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.521141
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.518695
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.513697
Train Epoch: 25 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5217, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.514399
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.523712
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.520368
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.523180
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.527481
Train Epoch: 26 - Loss: 0.5172, Accuracy: 99.79%

Test set: Average loss: 0.5239, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.533077
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.514505
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.511954
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.513430
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.512274
Train Epoch: 27 - Loss: 0.5165, Accuracy: 99.81%

Test set: Average loss: 0.5220, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.528685
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.520629
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.514668
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.530307
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.510016
Train Epoch: 28 - Loss: 0.5158, Accuracy: 99.82%

Test set: Average loss: 0.5222, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.523473
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.511604
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.527427
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.517359
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.516651
Train Epoch: 29 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5220, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.507617
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.523685
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.512908
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.510693
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.508739
Train Epoch: 30 - Loss: 0.5141, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.511839
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.507396
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.516292
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.508750
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.512022
Train Epoch: 31 - Loss: 0.5135, Accuracy: 99.88%

Test set: Average loss: 0.5214, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.511635
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.515520
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.510526
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.508095
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.512111
Train Epoch: 32 - Loss: 0.5127, Accuracy: 99.90%

Test set: Average loss: 0.5197, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.511670
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.512312
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.521966
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.508933
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.509991
Train Epoch: 33 - Loss: 0.5124, Accuracy: 99.90%

Test set: Average loss: 0.5211, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.511639
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.522809
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.510258
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.512520
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.509738
Train Epoch: 34 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5199, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.515024
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.511040
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.513621
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.510285
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.508362
Train Epoch: 35 - Loss: 0.5115, Accuracy: 99.90%

Test set: Average loss: 0.5198, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.514820
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.507426
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.509191
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.508628
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.510790
Train Epoch: 36 - Loss: 0.5109, Accuracy: 99.92%

Test set: Average loss: 0.5192, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.510415
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.509618
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.508115
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.514009
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.511000
Train Epoch: 37 - Loss: 0.5108, Accuracy: 99.92%

Test set: Average loss: 0.5194, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.518264
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.509615
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.515758
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.510270
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.507596
Train Epoch: 38 - Loss: 0.5104, Accuracy: 99.94%

Test set: Average loss: 0.5189, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.507044
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.509413
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.509156
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513641
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516814
Train Epoch: 39 - Loss: 0.5101, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.507434
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.510705
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.509170
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.509963
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.508745
Train Epoch: 40 - Loss: 0.5097, Accuracy: 99.96%

Test set: Average loss: 0.5187, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.505736
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.509507
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.510615
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.505609
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.507377
Train Epoch: 41 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5184, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.510053
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.510815
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.506424
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.506673
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.509749
Train Epoch: 42 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5185, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.510932
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.513737
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.507735
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.510734
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.505230
Train Epoch: 43 - Loss: 0.5092, Accuracy: 99.94%

Test set: Average loss: 0.5185, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.510054
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.508198
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.507820
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.506312
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.520162
Train Epoch: 44 - Loss: 0.5092, Accuracy: 99.96%

Test set: Average loss: 0.5183, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.514314
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.507536
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.507562
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.505371
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.531168
Train Epoch: 45 - Loss: 0.5091, Accuracy: 99.95%

Test set: Average loss: 0.5183, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.508849
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.511854
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.508788
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.510320
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.508974
Train Epoch: 46 - Loss: 0.5090, Accuracy: 99.97%

Test set: Average loss: 0.5185, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.509246
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.506771
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.506654
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512235
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.506266
Train Epoch: 47 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5182, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.514400
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.506320
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.514229
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.511878
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.510666
Train Epoch: 48 - Loss: 0.5088, Accuracy: 99.97%

Test set: Average loss: 0.5184, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.511501
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.510152
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.506114
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.505396
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.517284
Train Epoch: 49 - Loss: 0.5088, Accuracy: 99.97%

Test set: Average loss: 0.5182, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.506725
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.507149
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.510807
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.506682
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.509497
Train Epoch: 50 - Loss: 0.5087, Accuracy: 99.97%

Test set: Average loss: 0.5182, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.506924
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.509369
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.509065
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.522090
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.529725
Train Epoch: 51 - Loss: 0.5101, Accuracy: 99.93%

Test set: Average loss: 0.5188, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.516483
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.512582
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.510779
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.509975
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.507265
Train Epoch: 52 - Loss: 0.5097, Accuracy: 99.94%

Test set: Average loss: 0.5195, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.508803
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.521727
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.509387
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.508925
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.507061
Train Epoch: 53 - Loss: 0.5096, Accuracy: 99.94%

Test set: Average loss: 0.5186, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.505454
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.509112
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.509353
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.506361
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.507078
Train Epoch: 54 - Loss: 0.5093, Accuracy: 99.94%

Test set: Average loss: 0.5187, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.506364
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.505735
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.510464
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.505654
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.507279
Train Epoch: 55 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5177, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.506813
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.508085
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.505322
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.507041
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.504338
Train Epoch: 56 - Loss: 0.5085, Accuracy: 99.97%

Test set: Average loss: 0.5183, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.520449
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.510030
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.507020
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.514182
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.507346
Train Epoch: 57 - Loss: 0.5083, Accuracy: 99.95%

Test set: Average loss: 0.5185, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.511038
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.507558
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.508946
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.510084
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.505955
Train Epoch: 58 - Loss: 0.5081, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.505728
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.507197
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.507900
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.506529
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.506717
Train Epoch: 59 - Loss: 0.5078, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.504324
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.506389
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.525995
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517393
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.512713
Train Epoch: 60 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5176, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.505881
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.506179
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.506983
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.507225
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.507095
Train Epoch: 61 - Loss: 0.5075, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.508328
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506486
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.508290
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.507606
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.504661
Train Epoch: 62 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.505518
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.507902
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.505433
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.505873
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.505082
Train Epoch: 63 - Loss: 0.5072, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.504659
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.504314
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.508076
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.507156
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.505608
Train Epoch: 64 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.506854
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.506804
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.503875
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.506284
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.511313
Train Epoch: 65 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.504842
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.508162
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.505109
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505334
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.505444
Train Epoch: 66 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.506151
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.505064
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.504531
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.505411
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506407
Train Epoch: 67 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.506685
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.506076
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.505127
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.504269
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.506102
Train Epoch: 68 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.505722
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.505861
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.505120
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.504628
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.509485
Train Epoch: 69 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.505369
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.503537
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510670
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.509331
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507093
Train Epoch: 70 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.513080
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.505013
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.510472
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.503976
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.505931
Train Epoch: 71 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.508108
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.508968
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509340
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.505679
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.506002
Train Epoch: 72 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.506579
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.505001
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.505607
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.508805
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.504520
Train Epoch: 73 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.505381
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.514774
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.508140
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.504412
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.507427
Train Epoch: 74 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.504882
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.503861
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.504775
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.505075
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506148
Train Epoch: 75 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.505031
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.504411
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.503847
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.510261
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.511660
Train Epoch: 76 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5181, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.507659
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.506872
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.506153
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.507441
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.508209
Train Epoch: 77 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5181, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.506302
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.505172
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.505832
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.506299
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.510283
Train Epoch: 78 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.503580
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.505702
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.504235
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.504418
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.507767
Train Epoch: 79 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9957/10000 (99.57%)

New best model saved with accuracy: 99.57%
Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504673
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.505420
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.505426
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.504725
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.512302
Train Epoch: 80 - Loss: 0.5065, Accuracy: 99.97%

Test set: Average loss: 0.5169, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.505830
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.507945
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.504069
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.504848
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.507475
Train Epoch: 81 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.506091
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.507572
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.507747
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.504842
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.503753
Train Epoch: 82 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.505945
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.504555
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.504859
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.505127
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.507579
Train Epoch: 83 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.505481
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.504168
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.517004
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.505087
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.504387
Train Epoch: 84 - Loss: 0.5059, Accuracy: 99.98%

Test set: Average loss: 0.5163, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.504506
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.505257
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.505584
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.503353
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.510726
Train Epoch: 85 - Loss: 0.5057, Accuracy: 99.98%

Test set: Average loss: 0.5161, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.503926
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.506709
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.505737
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.502820
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.505161
Train Epoch: 86 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.505852
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.504781
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.504173
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505037
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.508060
Train Epoch: 87 - Loss: 0.5056, Accuracy: 99.98%

Test set: Average loss: 0.5163, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504256
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.504829
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.504313
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.505147
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.503834
Train Epoch: 88 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.504186
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505923
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.504901
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.508142
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.505116
Train Epoch: 89 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.504098
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.503941
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.503149
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.504575
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.504996
Train Epoch: 90 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.503611
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.505912
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.503511
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.505925
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.504501
Train Epoch: 91 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.505895
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.503931
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.505124
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.506683
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.503782
Train Epoch: 92 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.504130
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.505265
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.503396
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.503294
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.503562
Train Epoch: 93 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505460
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.502602
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.504526
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.505229
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.504769
Train Epoch: 94 - Loss: 0.5050, Accuracy: 99.99%

Test set: Average loss: 0.5158, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504706
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.504001
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506051
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505186
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.503513
Train Epoch: 95 - Loss: 0.5048, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.503479
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.503000
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.503825
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.505078
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.502862
Train Epoch: 96 - Loss: 0.5048, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.504002
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.503816
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.503589
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.504419
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.505903
Train Epoch: 97 - Loss: 0.5048, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.503591
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.503420
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.503580
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.504007
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.504390
Train Epoch: 98 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5158, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.502988
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.503544
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.503814
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503279
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.504391
Train Epoch: 99 - Loss: 0.5048, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.504178
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.503661
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.505232
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.503366
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.502905
Train Epoch: 100 - Loss: 0.5049, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9948/10000 (99.48%)


Training completed. Best Accuracy: 99.57%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.167605
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.854826
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.673025
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.614805
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.613786
Train Epoch: 1 - Loss: 0.8025, Accuracy: 92.39%

Test set: Average loss: 0.5984, Accuracy: 9789/10000 (97.89%)

New best model saved with accuracy: 97.89%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.579990
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.601639
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.567228
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.590289
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.581569
Train Epoch: 2 - Loss: 0.5849, Accuracy: 98.19%

Test set: Average loss: 0.5667, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.575185
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.543330
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.543179
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.560673
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.590678
Train Epoch: 3 - Loss: 0.5595, Accuracy: 98.83%

Test set: Average loss: 0.5492, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.537907
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.547561
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.538153
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.544783
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.543504
Train Epoch: 4 - Loss: 0.5471, Accuracy: 99.08%

Test set: Average loss: 0.5454, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.540547
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.534998
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.549525
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.558546
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.538119
Train Epoch: 5 - Loss: 0.5404, Accuracy: 99.20%

Test set: Average loss: 0.5379, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.533205
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.531104
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.544946
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.531349
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.526758
Train Epoch: 6 - Loss: 0.5358, Accuracy: 99.28%

Test set: Average loss: 0.5347, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.519324
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.522156
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.521630
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.530565
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.526705
Train Epoch: 7 - Loss: 0.5313, Accuracy: 99.46%

Test set: Average loss: 0.5330, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.525526
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.534886
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.526227
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.518745
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.517826
Train Epoch: 8 - Loss: 0.5285, Accuracy: 99.51%

Test set: Average loss: 0.5309, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.533333
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.530606
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.519523
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.514260
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.533013
Train Epoch: 9 - Loss: 0.5260, Accuracy: 99.58%

Test set: Average loss: 0.5291, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.528173
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.521815
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.516467
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.521664
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.525900
Train Epoch: 10 - Loss: 0.5244, Accuracy: 99.61%

Test set: Average loss: 0.5280, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.516427
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.520168
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.518155
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.518517
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.532709
Train Epoch: 11 - Loss: 0.5226, Accuracy: 99.69%

Test set: Average loss: 0.5279, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.520957
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.525001
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.516480
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.515659
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.512336
Train Epoch: 12 - Loss: 0.5215, Accuracy: 99.70%

Test set: Average loss: 0.5260, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.523872
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.520131
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.525415
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.517176
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.514241
Train Epoch: 13 - Loss: 0.5205, Accuracy: 99.74%

Test set: Average loss: 0.5264, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.515453
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.512089
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.521106
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.519166
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.517466
Train Epoch: 14 - Loss: 0.5197, Accuracy: 99.75%

Test set: Average loss: 0.5253, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.522325
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.514965
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.515780
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.512939
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.518829
Train Epoch: 15 - Loss: 0.5187, Accuracy: 99.78%

Test set: Average loss: 0.5250, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.510215
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.521080
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.516545
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.511289
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.522241
Train Epoch: 16 - Loss: 0.5182, Accuracy: 99.78%

Test set: Average loss: 0.5245, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.514228
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.517003
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.507767
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.509662
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.518703
Train Epoch: 17 - Loss: 0.5177, Accuracy: 99.81%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.515986
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.513155
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.512258
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.514997
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.508685
Train Epoch: 18 - Loss: 0.5171, Accuracy: 99.82%

Test set: Average loss: 0.5238, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.520850
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.513588
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.514823
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.509761
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.532348
Train Epoch: 19 - Loss: 0.5170, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.517622
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.512731
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.525253
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.511790
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.510379
Train Epoch: 20 - Loss: 0.5167, Accuracy: 99.83%

Test set: Average loss: 0.5236, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.509659
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.509155
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.516140
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.521594
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.516314
Train Epoch: 21 - Loss: 0.5164, Accuracy: 99.83%

Test set: Average loss: 0.5235, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.525116
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.510277
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.521563
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.513989
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.514530
Train Epoch: 22 - Loss: 0.5162, Accuracy: 99.85%

Test set: Average loss: 0.5235, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.528795
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.519896
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.518985
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.521110
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.516086
Train Epoch: 23 - Loss: 0.5159, Accuracy: 99.85%

Test set: Average loss: 0.5234, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.517653
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.511146
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.516533
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.510896
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.510676
Train Epoch: 24 - Loss: 0.5161, Accuracy: 99.83%

Test set: Average loss: 0.5236, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.513832
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.518669
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.518056
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.521312
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.510180
Train Epoch: 25 - Loss: 0.5160, Accuracy: 99.85%

Test set: Average loss: 0.5234, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.513305
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.529625
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.525383
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.514479
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.509777
Train Epoch: 26 - Loss: 0.5182, Accuracy: 99.78%

Test set: Average loss: 0.5232, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.525314
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.515360
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.518050
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.515023
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.520494
Train Epoch: 27 - Loss: 0.5173, Accuracy: 99.78%

Test set: Average loss: 0.5234, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.515303
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.516955
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.542648
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.513903
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.510112
Train Epoch: 28 - Loss: 0.5165, Accuracy: 99.80%

Test set: Average loss: 0.5237, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.531932
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.510384
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.512436
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522529
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.516188
Train Epoch: 29 - Loss: 0.5154, Accuracy: 99.85%

Test set: Average loss: 0.5221, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.523403
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.509340
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.513278
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.509221
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.509594
Train Epoch: 30 - Loss: 0.5151, Accuracy: 99.83%

Test set: Average loss: 0.5216, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.515944
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.510702
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.510476
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.510953
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.508098
Train Epoch: 31 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5218, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.509831
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.507336
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.508613
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514989
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.515359
Train Epoch: 32 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5216, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.513058
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.513817
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.513777
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.513021
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.509199
Train Epoch: 33 - Loss: 0.5131, Accuracy: 99.88%

Test set: Average loss: 0.5208, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.508948
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.511209
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.510848
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.511640
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.509519
Train Epoch: 34 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5208, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.515207
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.524847
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.515669
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.510703
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.508290
Train Epoch: 35 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5200, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.509254
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509244
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.509788
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.507934
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.506041
Train Epoch: 36 - Loss: 0.5115, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.511972
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.506868
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.513230
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.509658
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.510541
Train Epoch: 37 - Loss: 0.5113, Accuracy: 99.91%

Test set: Average loss: 0.5200, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.516697
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.508773
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.513814
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.510540
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.508420
Train Epoch: 38 - Loss: 0.5110, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.509696
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.511012
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.511558
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.512066
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.506202
Train Epoch: 39 - Loss: 0.5106, Accuracy: 99.94%

Test set: Average loss: 0.5200, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.508256
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.511052
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.507541
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.512731
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.509730
Train Epoch: 40 - Loss: 0.5103, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.508165
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.504656
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.509352
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.506210
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.510639
Train Epoch: 41 - Loss: 0.5102, Accuracy: 99.95%

Test set: Average loss: 0.5195, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.517300
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.510728
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.511774
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.506142
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.506663
Train Epoch: 42 - Loss: 0.5099, Accuracy: 99.94%

Test set: Average loss: 0.5191, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.509423
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.505824
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.508006
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.506566
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.508075
Train Epoch: 43 - Loss: 0.5097, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.510680
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.506513
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.508615
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.509710
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.508645
Train Epoch: 44 - Loss: 0.5096, Accuracy: 99.96%

Test set: Average loss: 0.5192, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.509089
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.510303
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.508749
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.508891
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.509259
Train Epoch: 45 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.507143
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.505540
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.507255
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.509332
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.506684
Train Epoch: 46 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5189, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508207
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.507619
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.514981
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.507274
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.508637
Train Epoch: 47 - Loss: 0.5093, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.511062
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.507241
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.510959
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.507606
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.516474
Train Epoch: 48 - Loss: 0.5093, Accuracy: 99.97%

Test set: Average loss: 0.5190, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.509022
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.509692
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.505813
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.508560
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.509326
Train Epoch: 49 - Loss: 0.5094, Accuracy: 99.97%

Test set: Average loss: 0.5189, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.509928
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.507212
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.513151
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.510543
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.507329
Train Epoch: 50 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5189, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.508656
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.512047
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.510458
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.510744
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.507252
Train Epoch: 51 - Loss: 0.5105, Accuracy: 99.94%

Test set: Average loss: 0.5202, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.515063
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.509097
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.514309
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.520899
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.509415
Train Epoch: 52 - Loss: 0.5103, Accuracy: 99.93%

Test set: Average loss: 0.5191, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.510875
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.507249
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.508522
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.507681
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.507931
Train Epoch: 53 - Loss: 0.5100, Accuracy: 99.95%

Test set: Average loss: 0.5189, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.506137
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.506458
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.514039
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.508799
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.509159
Train Epoch: 54 - Loss: 0.5098, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.537353
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.520468
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.507509
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.513604
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.509176
Train Epoch: 55 - Loss: 0.5094, Accuracy: 99.94%

Test set: Average loss: 0.5186, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.516416
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.507929
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.515512
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.506215
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.511694
Train Epoch: 56 - Loss: 0.5091, Accuracy: 99.94%

Test set: Average loss: 0.5182, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.507544
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.508121
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.508144
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.508724
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.510391
Train Epoch: 57 - Loss: 0.5089, Accuracy: 99.95%

Test set: Average loss: 0.5182, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.507336
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.505581
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.509435
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.506961
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.518410
Train Epoch: 58 - Loss: 0.5086, Accuracy: 99.96%

Test set: Average loss: 0.5185, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.507065
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.506490
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.513464
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.508529
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.505218
Train Epoch: 59 - Loss: 0.5083, Accuracy: 99.96%

Test set: Average loss: 0.5180, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.505190
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.508694
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.505823
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.505910
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.507310
Train Epoch: 60 - Loss: 0.5084, Accuracy: 99.97%

Test set: Average loss: 0.5179, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.507452
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.509213
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.509101
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.506714
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.508437
Train Epoch: 61 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5179, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.506364
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.507707
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.509122
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.505662
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.504464
Train Epoch: 62 - Loss: 0.5078, Accuracy: 99.97%

Test set: Average loss: 0.5173, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.506591
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.504991
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.505040
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.506323
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.508876
Train Epoch: 63 - Loss: 0.5075, Accuracy: 99.97%

Test set: Average loss: 0.5179, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.505731
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508884
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.510065
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.505476
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.506348
Train Epoch: 64 - Loss: 0.5074, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.505916
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.506518
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.506623
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.508440
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508904
Train Epoch: 65 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.505477
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.507179
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.506166
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.506403
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.505193
Train Epoch: 66 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.507045
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.503803
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.509270
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.508480
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.504267
Train Epoch: 67 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.509468
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.507782
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.504937
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.509438
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.506438
Train Epoch: 68 - Loss: 0.5069, Accuracy: 99.99%

Test set: Average loss: 0.5172, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.507976
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.503524
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.505602
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.505227
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506206
Train Epoch: 69 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.506144
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.507311
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.504470
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.509360
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.511200
Train Epoch: 70 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.504219
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.505400
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.503868
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510010
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.506405
Train Epoch: 71 - Loss: 0.5068, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.505024
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.504259
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508029
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507091
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508589
Train Epoch: 72 - Loss: 0.5066, Accuracy: 99.99%

Test set: Average loss: 0.5169, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505263
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508789
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.505429
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.506965
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.505287
Train Epoch: 73 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.505806
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.504404
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506503
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.506621
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.507465
Train Epoch: 74 - Loss: 0.5067, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.509414
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.507083
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.504644
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506716
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.508073
Train Epoch: 75 - Loss: 0.5065, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.505183
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510597
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.504946
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.506062
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.504463
Train Epoch: 76 - Loss: 0.5075, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.507246
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.504818
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.507032
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.505820
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.509797
Train Epoch: 77 - Loss: 0.5073, Accuracy: 99.97%

Test set: Average loss: 0.5189, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505482
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.504170
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.514553
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.503862
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.506567
Train Epoch: 78 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.503921
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.505277
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.504369
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507438
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.504956
Train Epoch: 79 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.508044
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.506018
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.506288
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.505446
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.507142
Train Epoch: 80 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5165, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.506452
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.504965
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.507524
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.505124
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.504397
Train Epoch: 81 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5182, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.504683
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.509612
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.504923
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.503755
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.505374
Train Epoch: 82 - Loss: 0.5065, Accuracy: 99.99%

Test set: Average loss: 0.5169, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.504577
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.506912
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.505834
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506347
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.504893
Train Epoch: 83 - Loss: 0.5065, Accuracy: 99.97%

Test set: Average loss: 0.5173, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.506086
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.505846
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.509703
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.508376
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.507798
Train Epoch: 84 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.505960
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.511403
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.505367
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.505595
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.504353
Train Epoch: 85 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.503562
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.504469
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.507523
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.505056
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.510215
Train Epoch: 86 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5163, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.506088
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.507318
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505011
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.507415
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509636
Train Epoch: 87 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.506803
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507004
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505545
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.503933
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.504530
Train Epoch: 88 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.506768
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.504445
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.503990
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.504209
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.503239
Train Epoch: 89 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.505710
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.505585
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507115
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.505235
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.503472
Train Epoch: 90 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504372
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.503467
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.504649
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.506379
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.505714
Train Epoch: 91 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504208
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504191
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.509319
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.508650
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.504858
Train Epoch: 92 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.504957
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.505170
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.506753
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.504515
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504185
Train Epoch: 93 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505570
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507658
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.505790
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.503341
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.505267
Train Epoch: 94 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504020
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.504262
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.504272
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.503143
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.504327
Train Epoch: 95 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9953/10000 (99.53%)

New best model saved with accuracy: 99.53%
Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.504777
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.504184
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.503591
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.507362
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.505715
Train Epoch: 96 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.503867
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.504553
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.504209
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.506053
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.506192
Train Epoch: 97 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.502779
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.504761
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.504316
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.503470
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.506781
Train Epoch: 98 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.505795
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.503803
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.503503
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503311
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503926
Train Epoch: 99 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.503938
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.504831
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.505166
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504691
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.504871
Train Epoch: 100 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9948/10000 (99.48%)


Training completed. Best Accuracy: 99.53%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.807985
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.757081
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.708426
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.654685
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.601247
Train Epoch: 1 - Loss: 0.7870, Accuracy: 92.68%

Test set: Average loss: 0.5929, Accuracy: 9790/10000 (97.90%)

New best model saved with accuracy: 97.90%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.569332
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.582285
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.598348
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.563129
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.565472
Train Epoch: 2 - Loss: 0.5823, Accuracy: 98.20%

Test set: Average loss: 0.5636, Accuracy: 9854/10000 (98.54%)

New best model saved with accuracy: 98.54%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.551222
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.556449
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.540367
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.541954
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.532919
Train Epoch: 3 - Loss: 0.5582, Accuracy: 98.78%

Test set: Average loss: 0.5479, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.554719
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.551705
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.542230
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.554580
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.558536
Train Epoch: 4 - Loss: 0.5472, Accuracy: 99.03%

Test set: Average loss: 0.5421, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.558083
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.558747
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.536458
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.537022
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.534425
Train Epoch: 5 - Loss: 0.5398, Accuracy: 99.25%

Test set: Average loss: 0.5367, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.517760
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.519980
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.532792
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.522036
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.541899
Train Epoch: 6 - Loss: 0.5349, Accuracy: 99.40%

Test set: Average loss: 0.5336, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.528036
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.528883
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.526433
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.525420
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.533806
Train Epoch: 7 - Loss: 0.5313, Accuracy: 99.48%

Test set: Average loss: 0.5319, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.525994
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.523124
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.528122
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.539805
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.523005
Train Epoch: 8 - Loss: 0.5283, Accuracy: 99.54%

Test set: Average loss: 0.5310, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.518472
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.526183
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.526489
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.517662
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.532676
Train Epoch: 9 - Loss: 0.5265, Accuracy: 99.58%

Test set: Average loss: 0.5287, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.520525
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.517355
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.520253
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.518688
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.535638
Train Epoch: 10 - Loss: 0.5242, Accuracy: 99.64%

Test set: Average loss: 0.5282, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.522507
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.519347
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.524346
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.513016
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.524201
Train Epoch: 11 - Loss: 0.5228, Accuracy: 99.68%

Test set: Average loss: 0.5269, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.524946
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.525576
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.512888
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.519453
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.519155
Train Epoch: 12 - Loss: 0.5214, Accuracy: 99.73%

Test set: Average loss: 0.5262, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.527126
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.512609
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.534746
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.523497
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.510427
Train Epoch: 13 - Loss: 0.5205, Accuracy: 99.74%

Test set: Average loss: 0.5255, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.516400
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.511982
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.514755
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.517374
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.518263
Train Epoch: 14 - Loss: 0.5197, Accuracy: 99.75%

Test set: Average loss: 0.5251, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.510052
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.519774
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.525305
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.518144
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.515359
Train Epoch: 15 - Loss: 0.5188, Accuracy: 99.78%

Test set: Average loss: 0.5245, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.525995
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.509508
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.517950
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.521184
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.524451
Train Epoch: 16 - Loss: 0.5183, Accuracy: 99.79%

Test set: Average loss: 0.5244, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.512964
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.509558
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.520324
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.530281
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.517532
Train Epoch: 17 - Loss: 0.5178, Accuracy: 99.80%

Test set: Average loss: 0.5239, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.515900
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.516908
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.545030
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.511832
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.511489
Train Epoch: 18 - Loss: 0.5175, Accuracy: 99.82%

Test set: Average loss: 0.5239, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.513805
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.516160
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.523883
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.530558
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515193
Train Epoch: 19 - Loss: 0.5171, Accuracy: 99.81%

Test set: Average loss: 0.5237, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.516999
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.513845
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.512119
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.512758
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.512469
Train Epoch: 20 - Loss: 0.5165, Accuracy: 99.84%

Test set: Average loss: 0.5237, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.513138
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.513981
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.516980
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.515742
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.513685
Train Epoch: 21 - Loss: 0.5166, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.511499
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.510374
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.528862
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.519348
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.512400
Train Epoch: 22 - Loss: 0.5164, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.513770
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.518924
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.509540
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.511632
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.511864
Train Epoch: 23 - Loss: 0.5162, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.522567
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.524511
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.515089
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.513695
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.521046
Train Epoch: 24 - Loss: 0.5161, Accuracy: 99.84%

Test set: Average loss: 0.5233, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.513845
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.512628
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.515442
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.516217
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.515256
Train Epoch: 25 - Loss: 0.5162, Accuracy: 99.83%

Test set: Average loss: 0.5234, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.514367
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.517109
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.518338
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.508642
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.513076
Train Epoch: 26 - Loss: 0.5186, Accuracy: 99.76%

Test set: Average loss: 0.5249, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.509814
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.516342
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.511936
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.528872
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.513238
Train Epoch: 27 - Loss: 0.5175, Accuracy: 99.77%

Test set: Average loss: 0.5238, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.515739
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.511609
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.519721
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.510498
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.510577
Train Epoch: 28 - Loss: 0.5166, Accuracy: 99.79%

Test set: Average loss: 0.5232, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.510545
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.520741
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.516202
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.518307
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.512949
Train Epoch: 29 - Loss: 0.5158, Accuracy: 99.83%

Test set: Average loss: 0.5222, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.521220
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.526381
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.508886
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.514551
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.507937
Train Epoch: 30 - Loss: 0.5152, Accuracy: 99.84%

Test set: Average loss: 0.5229, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.510729
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.510638
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.512416
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.512129
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.512737
Train Epoch: 31 - Loss: 0.5145, Accuracy: 99.85%

Test set: Average loss: 0.5218, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.508591
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.511420
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.508501
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.522591
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.512814
Train Epoch: 32 - Loss: 0.5143, Accuracy: 99.85%

Test set: Average loss: 0.5215, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.510092
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.536123
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.511469
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.507409
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.514089
Train Epoch: 33 - Loss: 0.5134, Accuracy: 99.88%

Test set: Average loss: 0.5210, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.512834
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.505913
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.511002
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.513701
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.507159
Train Epoch: 34 - Loss: 0.5129, Accuracy: 99.88%

Test set: Average loss: 0.5205, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.509822
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.513127
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.520313
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.509542
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.511948
Train Epoch: 35 - Loss: 0.5125, Accuracy: 99.90%

Test set: Average loss: 0.5208, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.511000
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.512242
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.509735
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.509523
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.509065
Train Epoch: 36 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5204, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.508737
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.509009
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.508008
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.515863
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.510190
Train Epoch: 37 - Loss: 0.5119, Accuracy: 99.89%

Test set: Average loss: 0.5198, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.519863
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.508371
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.507479
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.510692
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.512766
Train Epoch: 38 - Loss: 0.5113, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.516675
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.507515
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.508007
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513830
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.506667
Train Epoch: 39 - Loss: 0.5111, Accuracy: 99.93%

Test set: Average loss: 0.5193, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.508593
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.509663
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.512528
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.508679
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.507927
Train Epoch: 40 - Loss: 0.5108, Accuracy: 99.92%

Test set: Average loss: 0.5198, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.511953
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.507294
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.513675
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.506426
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.510201
Train Epoch: 41 - Loss: 0.5106, Accuracy: 99.93%

Test set: Average loss: 0.5195, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.508752
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.507804
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.508084
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.511180
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.507286
Train Epoch: 42 - Loss: 0.5104, Accuracy: 99.93%

Test set: Average loss: 0.5192, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.525539
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.506966
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.506459
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.508805
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.509572
Train Epoch: 43 - Loss: 0.5102, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.507389
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.506021
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.505614
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.508185
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.509824
Train Epoch: 44 - Loss: 0.5102, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.508434
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.510779
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.508055
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.507196
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.510696
Train Epoch: 45 - Loss: 0.5101, Accuracy: 99.95%

Test set: Average loss: 0.5192, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.512384
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.509372
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.516097
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.509173
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.507362
Train Epoch: 46 - Loss: 0.5098, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508601
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.508738
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.508055
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.524978
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.509196
Train Epoch: 47 - Loss: 0.5099, Accuracy: 99.94%

Test set: Average loss: 0.5188, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.507470
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.520519
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.507294
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.510437
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.509322
Train Epoch: 48 - Loss: 0.5098, Accuracy: 99.95%

Test set: Average loss: 0.5189, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.507864
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.508483
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.513514
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.509316
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.509020
Train Epoch: 49 - Loss: 0.5097, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.510080
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.510726
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.507437
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512161
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.510708
Train Epoch: 50 - Loss: 0.5097, Accuracy: 99.94%

Test set: Average loss: 0.5188, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.518145
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.511604
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.510912
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.505219
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.509927
Train Epoch: 51 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5191, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.513476
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.509424
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.510236
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.512539
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.506496
Train Epoch: 52 - Loss: 0.5107, Accuracy: 99.93%

Test set: Average loss: 0.5193, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.508733
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.512749
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.513945
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.508736
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.507224
Train Epoch: 53 - Loss: 0.5105, Accuracy: 99.94%

Test set: Average loss: 0.5198, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.505360
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.507861
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.515761
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.512259
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.510827
Train Epoch: 54 - Loss: 0.5101, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.507169
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.508782
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.506111
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.507629
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.508254
Train Epoch: 55 - Loss: 0.5101, Accuracy: 99.94%

Test set: Average loss: 0.5183, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.505560
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.507389
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.507593
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.508055
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.515096
Train Epoch: 56 - Loss: 0.5095, Accuracy: 99.94%

Test set: Average loss: 0.5182, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.507695
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.508295
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.505876
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.508353
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.510892
Train Epoch: 57 - Loss: 0.5093, Accuracy: 99.95%

Test set: Average loss: 0.5185, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.506149
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.511367
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.511022
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.505466
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506601
Train Epoch: 58 - Loss: 0.5090, Accuracy: 99.95%

Test set: Average loss: 0.5186, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.504753
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.511776
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.506187
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.529557
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.505856
Train Epoch: 59 - Loss: 0.5088, Accuracy: 99.94%

Test set: Average loss: 0.5181, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.509129
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.506520
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.506406
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.506307
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.507369
Train Epoch: 60 - Loss: 0.5086, Accuracy: 99.96%

Test set: Average loss: 0.5182, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.505779
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.505657
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.514528
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.507094
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.507602
Train Epoch: 61 - Loss: 0.5083, Accuracy: 99.96%

Test set: Average loss: 0.5180, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510116
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506669
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.511589
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.508823
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.508005
Train Epoch: 62 - Loss: 0.5082, Accuracy: 99.96%

Test set: Average loss: 0.5176, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.507924
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506152
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.510483
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.507931
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.505473
Train Epoch: 63 - Loss: 0.5079, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.507586
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.511013
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.507164
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.505501
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.509021
Train Epoch: 64 - Loss: 0.5078, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.507357
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509082
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.506919
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.507385
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.507820
Train Epoch: 65 - Loss: 0.5076, Accuracy: 99.98%

Test set: Average loss: 0.5175, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.506379
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.505572
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.506035
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.507661
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506413
Train Epoch: 66 - Loss: 0.5076, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.507650
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.507126
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.508270
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.506728
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509737
Train Epoch: 67 - Loss: 0.5073, Accuracy: 99.98%

Test set: Average loss: 0.5176, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.506654
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.506619
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.510803
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.504119
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.505936
Train Epoch: 68 - Loss: 0.5073, Accuracy: 99.99%

Test set: Average loss: 0.5172, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512269
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.504636
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.505027
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.506347
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506889
Train Epoch: 69 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5174, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.522860
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.519991
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.505471
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.504974
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507569
Train Epoch: 70 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.505472
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.507518
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.511490
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510096
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.506816
Train Epoch: 71 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.506100
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.504815
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.513463
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512608
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.505152
Train Epoch: 72 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505825
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508317
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.506237
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505833
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.509505
Train Epoch: 73 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.504761
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.505372
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506469
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.505017
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.512378
Train Epoch: 74 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.507766
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.508537
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.504682
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508525
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506543
Train Epoch: 75 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.507905
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.506555
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.506046
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.506874
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507307
Train Epoch: 76 - Loss: 0.5081, Accuracy: 99.96%

Test set: Average loss: 0.5184, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.504757
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.508943
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.508225
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.506130
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.506740
Train Epoch: 77 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5186, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505407
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.507325
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.506577
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.507336
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.504231
Train Epoch: 78 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.508651
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.504468
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.505009
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.514253
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.505232
Train Epoch: 79 - Loss: 0.5075, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504698
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507803
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509869
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.505404
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.509917
Train Epoch: 80 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.505187
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.504496
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.505891
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.504647
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.508511
Train Epoch: 81 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.503860
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.506096
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.512137
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.507325
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.505488
Train Epoch: 82 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5173, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.505352
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.505570
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506374
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.508046
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.509177
Train Epoch: 83 - Loss: 0.5069, Accuracy: 99.97%

Test set: Average loss: 0.5173, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.506321
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.504892
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.505738
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.507477
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508342
Train Epoch: 84 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510411
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.507807
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.508210
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.505932
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.510295
Train Epoch: 85 - Loss: 0.5065, Accuracy: 99.99%

Test set: Average loss: 0.5167, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.505177
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.504708
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.507152
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.504754
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.508232
Train Epoch: 86 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.504463
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.504958
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505497
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.504769
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.503482
Train Epoch: 87 - Loss: 0.5063, Accuracy: 99.98%

Test set: Average loss: 0.5175, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504670
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507207
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509366
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512511
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505877
Train Epoch: 88 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.504677
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507171
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.506611
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.503499
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.507009
Train Epoch: 89 - Loss: 0.5060, Accuracy: 99.99%

Test set: Average loss: 0.5166, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.503814
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.526886
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.506758
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508129
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.504535
Train Epoch: 90 - Loss: 0.5059, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.506155
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510169
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.503522
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.504424
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508079
Train Epoch: 91 - Loss: 0.5058, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504956
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.509766
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507201
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507664
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.511110
Train Epoch: 92 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506489
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507680
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.505075
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.504957
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504654
Train Epoch: 93 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5164, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.504534
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.505599
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507420
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.505140
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.504929
Train Epoch: 94 - Loss: 0.5057, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.503629
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.504540
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.504085
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505005
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.507176
Train Epoch: 95 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.504503
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.506579
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.504929
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.505296
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.505039
Train Epoch: 96 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506752
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508318
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.505320
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.503955
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.503958
Train Epoch: 97 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.505098
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.505443
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.504628
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.504792
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.506498
Train Epoch: 98 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506372
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.505265
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506746
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503672
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.507023
Train Epoch: 99 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.504958
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.504860
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.506760
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.508699
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.504374
Train Epoch: 100 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9946/10000 (99.46%)


Training completed. Best Accuracy: 99.49%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 6.005089
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.770782
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.715440
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.738521
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.619965
Train Epoch: 1 - Loss: 0.8090, Accuracy: 91.89%

Test set: Average loss: 0.6057, Accuracy: 9771/10000 (97.71%)

New best model saved with accuracy: 97.71%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.594341
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.613433
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.669713
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.613677
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.594234
Train Epoch: 2 - Loss: 0.5872, Accuracy: 98.05%

Test set: Average loss: 0.5663, Accuracy: 9854/10000 (98.54%)

New best model saved with accuracy: 98.54%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.551568
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.559206
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.591646
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.549601
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.559871
Train Epoch: 3 - Loss: 0.5606, Accuracy: 98.77%

Test set: Average loss: 0.5505, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.574030
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.547738
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.543104
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.551720
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.552130
Train Epoch: 4 - Loss: 0.5478, Accuracy: 99.11%

Test set: Average loss: 0.5435, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.524650
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.538063
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.527022
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.531105
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.533742
Train Epoch: 5 - Loss: 0.5407, Accuracy: 99.26%

Test set: Average loss: 0.5396, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.529744
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.519802
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.551982
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.541042
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.533187
Train Epoch: 6 - Loss: 0.5353, Accuracy: 99.37%

Test set: Average loss: 0.5350, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.535582
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.547841
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.518546
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.522107
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.527284
Train Epoch: 7 - Loss: 0.5314, Accuracy: 99.49%

Test set: Average loss: 0.5327, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.533025
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.523930
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.521936
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.517819
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.520051
Train Epoch: 8 - Loss: 0.5285, Accuracy: 99.56%

Test set: Average loss: 0.5305, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.516940
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.521290
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.534581
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.518824
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.522606
Train Epoch: 9 - Loss: 0.5264, Accuracy: 99.59%

Test set: Average loss: 0.5296, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.519642
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.516288
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.526061
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.519084
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.531353
Train Epoch: 10 - Loss: 0.5245, Accuracy: 99.63%

Test set: Average loss: 0.5277, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.516230
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.529558
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.530642
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.513478
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.538358
Train Epoch: 11 - Loss: 0.5231, Accuracy: 99.67%

Test set: Average loss: 0.5275, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.521588
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.511425
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.514289
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.514886
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.523419
Train Epoch: 12 - Loss: 0.5217, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.515196
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.511951
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.535250
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.516730
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.516068
Train Epoch: 13 - Loss: 0.5206, Accuracy: 99.74%

Test set: Average loss: 0.5255, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.517754
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.514793
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.524368
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.527081
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.518430
Train Epoch: 14 - Loss: 0.5196, Accuracy: 99.74%

Test set: Average loss: 0.5252, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.513585
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.512166
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.536942
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.537323
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.522150
Train Epoch: 15 - Loss: 0.5190, Accuracy: 99.78%

Test set: Average loss: 0.5248, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.515900
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.513387
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.535421
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.511646
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.524453
Train Epoch: 16 - Loss: 0.5183, Accuracy: 99.82%

Test set: Average loss: 0.5245, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.527761
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.514519
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.509590
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.516327
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.537319
Train Epoch: 17 - Loss: 0.5177, Accuracy: 99.82%

Test set: Average loss: 0.5243, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.523960
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.510650
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.510914
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.545203
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.524886
Train Epoch: 18 - Loss: 0.5175, Accuracy: 99.81%

Test set: Average loss: 0.5242, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.509739
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.513759
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.545662
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.515637
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515889
Train Epoch: 19 - Loss: 0.5170, Accuracy: 99.85%

Test set: Average loss: 0.5241, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.513024
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.515507
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.513981
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.512838
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.513730
Train Epoch: 20 - Loss: 0.5169, Accuracy: 99.83%

Test set: Average loss: 0.5237, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.521626
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.513104
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.518482
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.516426
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.513493
Train Epoch: 21 - Loss: 0.5167, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.510161
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.511921
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.527916
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.518906
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.514233
Train Epoch: 22 - Loss: 0.5164, Accuracy: 99.84%

Test set: Average loss: 0.5237, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.511271
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.510009
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.513679
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.526036
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.516586
Train Epoch: 23 - Loss: 0.5163, Accuracy: 99.85%

Test set: Average loss: 0.5235, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.525952
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.522377
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.511096
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.529903
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.521712
Train Epoch: 24 - Loss: 0.5162, Accuracy: 99.83%

Test set: Average loss: 0.5236, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.518574
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.525642
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.511170
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.514102
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.516366
Train Epoch: 25 - Loss: 0.5163, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.510997
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.524702
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.534151
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.511087
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.510405
Train Epoch: 26 - Loss: 0.5179, Accuracy: 99.79%

Test set: Average loss: 0.5252, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.523361
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.519030
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.512363
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.525984
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.511999
Train Epoch: 27 - Loss: 0.5172, Accuracy: 99.80%

Test set: Average loss: 0.5240, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.510411
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.516861
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.508433
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.519972
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.512537
Train Epoch: 28 - Loss: 0.5165, Accuracy: 99.81%

Test set: Average loss: 0.5226, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.523620
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.512653
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.518819
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.509999
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.508969
Train Epoch: 29 - Loss: 0.5156, Accuracy: 99.84%

Test set: Average loss: 0.5234, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.511467
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.515131
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.511889
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.517537
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.516061
Train Epoch: 30 - Loss: 0.5150, Accuracy: 99.85%

Test set: Average loss: 0.5224, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.517171
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.509801
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517461
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.518224
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.515625
Train Epoch: 31 - Loss: 0.5146, Accuracy: 99.84%

Test set: Average loss: 0.5210, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.509038
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.509548
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.514492
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.509139
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.511770
Train Epoch: 32 - Loss: 0.5135, Accuracy: 99.90%

Test set: Average loss: 0.5212, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.524575
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.515538
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.514167
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.512841
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.511543
Train Epoch: 33 - Loss: 0.5134, Accuracy: 99.88%

Test set: Average loss: 0.5217, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.513089
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.515226
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.510531
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.512366
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.517550
Train Epoch: 34 - Loss: 0.5128, Accuracy: 99.87%

Test set: Average loss: 0.5209, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.523064
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.515679
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.513429
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.522027
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.507115
Train Epoch: 35 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5200, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.515758
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509563
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.508924
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.508637
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.514033
Train Epoch: 36 - Loss: 0.5118, Accuracy: 99.92%

Test set: Average loss: 0.5206, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.507452
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516054
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.520343
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.509654
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.510267
Train Epoch: 37 - Loss: 0.5116, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.509837
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.512321
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.516901
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.514663
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.511599
Train Epoch: 38 - Loss: 0.5112, Accuracy: 99.93%

Test set: Average loss: 0.5196, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.513303
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.508192
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.509072
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513554
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.509291
Train Epoch: 39 - Loss: 0.5109, Accuracy: 99.93%

Test set: Average loss: 0.5193, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.509756
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.507950
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.510675
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.507045
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.510812
Train Epoch: 40 - Loss: 0.5107, Accuracy: 99.93%

Test set: Average loss: 0.5195, Accuracy: 9953/10000 (99.53%)

New best model saved with accuracy: 99.53%
Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.509964
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.507407
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.511135
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.506445
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.511707
Train Epoch: 41 - Loss: 0.5105, Accuracy: 99.93%

Test set: Average loss: 0.5192, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.506562
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.507583
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.511444
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.511895
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.508617
Train Epoch: 42 - Loss: 0.5103, Accuracy: 99.94%

Test set: Average loss: 0.5192, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.506842
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.514973
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.507228
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.507741
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.509917
Train Epoch: 43 - Loss: 0.5101, Accuracy: 99.96%

Test set: Average loss: 0.5192, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.507767
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.506366
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.511553
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.506450
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.514137
Train Epoch: 44 - Loss: 0.5099, Accuracy: 99.95%

Test set: Average loss: 0.5191, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.506730
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.507571
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.510477
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.507256
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.517611
Train Epoch: 45 - Loss: 0.5100, Accuracy: 99.95%

Test set: Average loss: 0.5189, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.516690
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.509792
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.508096
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.508528
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.507791
Train Epoch: 46 - Loss: 0.5096, Accuracy: 99.97%

Test set: Average loss: 0.5188, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.516812
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.509638
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.512882
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.515387
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.509702
Train Epoch: 47 - Loss: 0.5097, Accuracy: 99.96%

Test set: Average loss: 0.5190, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.507216
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.517817
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.507006
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.508690
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.509988
Train Epoch: 48 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5187, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.508999
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.507442
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.508070
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.508779
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.509660
Train Epoch: 49 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5189, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.509234
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.509803
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.508692
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.511245
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.511775
Train Epoch: 50 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5190, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.514811
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.515404
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.508347
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.511568
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.513359
Train Epoch: 51 - Loss: 0.5108, Accuracy: 99.92%

Test set: Average loss: 0.5212, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.514835
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.511090
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.510281
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.513076
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.505443
Train Epoch: 52 - Loss: 0.5104, Accuracy: 99.94%

Test set: Average loss: 0.5193, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.516286
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.512128
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.509517
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.505911
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.507812
Train Epoch: 53 - Loss: 0.5103, Accuracy: 99.94%

Test set: Average loss: 0.5189, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.508795
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.512393
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.516644
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.510319
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.507952
Train Epoch: 54 - Loss: 0.5098, Accuracy: 99.93%

Test set: Average loss: 0.5188, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.509392
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.513397
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.508618
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.512739
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.508668
Train Epoch: 55 - Loss: 0.5097, Accuracy: 99.94%

Test set: Average loss: 0.5198, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.510232
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.514942
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.511206
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.508587
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.512256
Train Epoch: 56 - Loss: 0.5095, Accuracy: 99.94%

Test set: Average loss: 0.5188, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.513961
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.505990
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.513413
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.510849
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.508203
Train Epoch: 57 - Loss: 0.5091, Accuracy: 99.94%

Test set: Average loss: 0.5181, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.506535
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.505231
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.508918
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.512963
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506277
Train Epoch: 58 - Loss: 0.5089, Accuracy: 99.96%

Test set: Average loss: 0.5184, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.507239
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.514010
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.508040
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.518816
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.508400
Train Epoch: 59 - Loss: 0.5085, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.508872
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.506628
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.508444
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.510617
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.503960
Train Epoch: 60 - Loss: 0.5084, Accuracy: 99.97%

Test set: Average loss: 0.5179, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.504861
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.512780
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.508738
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.509197
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.507503
Train Epoch: 61 - Loss: 0.5081, Accuracy: 99.98%

Test set: Average loss: 0.5179, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.504542
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506392
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.507044
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.510856
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509617
Train Epoch: 62 - Loss: 0.5080, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.510075
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506065
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.506104
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.505592
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.509789
Train Epoch: 63 - Loss: 0.5078, Accuracy: 99.97%

Test set: Average loss: 0.5178, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.511262
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508048
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.505415
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.507432
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.505498
Train Epoch: 64 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.513483
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509330
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.508090
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.505599
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.513003
Train Epoch: 65 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.507356
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.508984
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.506395
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505507
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.507484
Train Epoch: 66 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5174, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.505605
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.504721
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.508673
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.505608
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506515
Train Epoch: 67 - Loss: 0.5073, Accuracy: 99.98%

Test set: Average loss: 0.5172, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.505390
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.507851
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.508754
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.505990
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.508234
Train Epoch: 68 - Loss: 0.5073, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512579
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.505787
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507134
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.505061
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506381
Train Epoch: 69 - Loss: 0.5073, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.506148
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.506586
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.505649
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.504965
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.511153
Train Epoch: 70 - Loss: 0.5071, Accuracy: 99.98%

Test set: Average loss: 0.5171, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508857
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.506112
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.504493
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.506099
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.505775
Train Epoch: 71 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.506738
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.505419
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508613
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507722
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.505552
Train Epoch: 72 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505449
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.509925
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508111
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.507763
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.505200
Train Epoch: 73 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.507144
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.504947
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506229
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.506153
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.506277
Train Epoch: 74 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5170, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.504540
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.508335
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.505160
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506634
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.507350
Train Epoch: 75 - Loss: 0.5069, Accuracy: 99.99%

Test set: Average loss: 0.5170, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.508371
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510587
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510064
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.505173
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507845
Train Epoch: 76 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5175, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.508243
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.505414
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.508395
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.504009
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.506294
Train Epoch: 77 - Loss: 0.5076, Accuracy: 99.98%

Test set: Average loss: 0.5178, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.504470
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.507042
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.507059
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.506745
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.508629
Train Epoch: 78 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5177, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.508844
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.504904
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.508268
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.509034
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.505072
Train Epoch: 79 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5172, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.508173
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.506181
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.505405
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.505159
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.506678
Train Epoch: 80 - Loss: 0.5073, Accuracy: 99.97%

Test set: Average loss: 0.5170, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.504332
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.507167
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.505308
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.507555
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.503634
Train Epoch: 81 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5173, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.507990
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.508790
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.507762
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.505696
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.506759
Train Epoch: 82 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5176, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.506201
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.504612
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506490
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506263
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.505166
Train Epoch: 83 - Loss: 0.5067, Accuracy: 99.97%

Test set: Average loss: 0.5169, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.505719
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.507797
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508039
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509329
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.507798
Train Epoch: 84 - Loss: 0.5066, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.505659
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.505055
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.504848
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.504559
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.505953
Train Epoch: 85 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.504175
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.503979
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.509414
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.507987
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.504631
Train Epoch: 86 - Loss: 0.5062, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.504226
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.503314
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505234
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505939
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.504382
Train Epoch: 87 - Loss: 0.5060, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504130
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.504476
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.504633
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.503258
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.503756
Train Epoch: 88 - Loss: 0.5060, Accuracy: 99.99%

Test set: Average loss: 0.5168, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.505852
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505365
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.506818
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.510843
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.508232
Train Epoch: 89 - Loss: 0.5059, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.505579
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.505575
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.505088
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.506197
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.505769
Train Epoch: 90 - Loss: 0.5058, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.503826
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.505475
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.505048
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.509302
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508546
Train Epoch: 91 - Loss: 0.5058, Accuracy: 99.99%

Test set: Average loss: 0.5163, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504546
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.506623
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.505181
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.504917
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.505286
Train Epoch: 92 - Loss: 0.5057, Accuracy: 99.98%

Test set: Average loss: 0.5163, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.505224
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504217
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.505143
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.504171
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505124
Train Epoch: 93 - Loss: 0.5056, Accuracy: 100.00%

Test set: Average loss: 0.5164, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.504536
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.503192
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.504783
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.503632
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.505887
Train Epoch: 94 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.505433
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508339
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.504981
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.503048
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.506873
Train Epoch: 95 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.504345
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.503629
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.503756
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.504756
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.508027
Train Epoch: 96 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506221
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508197
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507938
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.504396
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504302
Train Epoch: 97 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5162, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.503649
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509458
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.504869
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.504451
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.505432
Train Epoch: 98 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5161, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506548
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504240
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506854
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.503427
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.505100
Train Epoch: 99 - Loss: 0.5055, Accuracy: 100.00%

Test set: Average loss: 0.5163, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.505215
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.507003
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.505745
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504602
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.503251
Train Epoch: 100 - Loss: 0.5055, Accuracy: 100.00%

Test set: Average loss: 0.5159, Accuracy: 9950/10000 (99.50%)


Training completed. Best Accuracy: 99.56%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.519605
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.757679
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.757038
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.662654
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.610109
Train Epoch: 1 - Loss: 0.7813, Accuracy: 92.48%

Test set: Average loss: 0.5977, Accuracy: 9772/10000 (97.72%)

New best model saved with accuracy: 97.72%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.597998
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.589137
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.578280
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.581494
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.577126
Train Epoch: 2 - Loss: 0.5828, Accuracy: 98.11%

Test set: Average loss: 0.5647, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.560630
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.581039
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.550672
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.589800
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.574966
Train Epoch: 3 - Loss: 0.5596, Accuracy: 98.72%

Test set: Average loss: 0.5498, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.561024
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.553965
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.545246
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.533378
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.525841
Train Epoch: 4 - Loss: 0.5470, Accuracy: 99.09%

Test set: Average loss: 0.5429, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.529955
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.527663
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.528667
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.551221
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.532277
Train Epoch: 5 - Loss: 0.5399, Accuracy: 99.25%

Test set: Average loss: 0.5374, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.546462
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.530966
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.555270
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.538244
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.542574
Train Epoch: 6 - Loss: 0.5355, Accuracy: 99.31%

Test set: Average loss: 0.5341, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.522731
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.516096
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.517926
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.543941
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.521664
Train Epoch: 7 - Loss: 0.5312, Accuracy: 99.48%

Test set: Average loss: 0.5327, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.520022
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.515709
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.518347
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.526034
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.548830
Train Epoch: 8 - Loss: 0.5284, Accuracy: 99.52%

Test set: Average loss: 0.5307, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.517338
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.550044
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.548238
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.538702
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.521670
Train Epoch: 9 - Loss: 0.5265, Accuracy: 99.57%

Test set: Average loss: 0.5291, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.525462
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.518029
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.525551
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.524422
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.526686
Train Epoch: 10 - Loss: 0.5244, Accuracy: 99.61%

Test set: Average loss: 0.5279, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.515413
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.528231
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.531121
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.528043
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.527428
Train Epoch: 11 - Loss: 0.5228, Accuracy: 99.65%

Test set: Average loss: 0.5269, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.514879
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.531886
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.521208
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.520547
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.518706
Train Epoch: 12 - Loss: 0.5217, Accuracy: 99.69%

Test set: Average loss: 0.5267, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.521317
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.513385
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.521231
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.524220
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.540442
Train Epoch: 13 - Loss: 0.5209, Accuracy: 99.73%

Test set: Average loss: 0.5252, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.518848
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.521740
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.519608
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.518946
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.523564
Train Epoch: 14 - Loss: 0.5198, Accuracy: 99.73%

Test set: Average loss: 0.5260, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.530711
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.518962
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517846
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.512859
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.512720
Train Epoch: 15 - Loss: 0.5191, Accuracy: 99.75%

Test set: Average loss: 0.5250, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.509393
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.521452
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.540046
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.522202
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.510429
Train Epoch: 16 - Loss: 0.5185, Accuracy: 99.79%

Test set: Average loss: 0.5248, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.531663
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.507410
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.526837
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.515828
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.512273
Train Epoch: 17 - Loss: 0.5180, Accuracy: 99.80%

Test set: Average loss: 0.5247, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.515342
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.517142
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.513272
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.520853
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.515221
Train Epoch: 18 - Loss: 0.5174, Accuracy: 99.82%

Test set: Average loss: 0.5243, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.516295
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.508804
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.513122
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.515018
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.518142
Train Epoch: 19 - Loss: 0.5169, Accuracy: 99.82%

Test set: Average loss: 0.5242, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.513376
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.517883
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.523601
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.510633
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.511093
Train Epoch: 20 - Loss: 0.5169, Accuracy: 99.81%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.514350
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.510785
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.517181
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.518232
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.512524
Train Epoch: 21 - Loss: 0.5169, Accuracy: 99.81%

Test set: Average loss: 0.5237, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.518194
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.510918
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.516936
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.513066
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.512824
Train Epoch: 22 - Loss: 0.5164, Accuracy: 99.84%

Test set: Average loss: 0.5238, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.514598
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.512876
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.511345
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.516372
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.523818
Train Epoch: 23 - Loss: 0.5164, Accuracy: 99.83%

Test set: Average loss: 0.5239, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.510465
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.508859
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.537411
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.515873
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.520232
Train Epoch: 24 - Loss: 0.5164, Accuracy: 99.84%

Test set: Average loss: 0.5239, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.509240
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.510804
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.518369
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.517516
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.526795
Train Epoch: 25 - Loss: 0.5163, Accuracy: 99.84%

Test set: Average loss: 0.5240, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.516231
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.531904
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.516803
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.516018
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.509603
Train Epoch: 26 - Loss: 0.5185, Accuracy: 99.75%

Test set: Average loss: 0.5242, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.512539
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.513831
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.512383
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.511597
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.516956
Train Epoch: 27 - Loss: 0.5174, Accuracy: 99.78%

Test set: Average loss: 0.5270, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.513934
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.513872
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.514053
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.520451
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.514144
Train Epoch: 28 - Loss: 0.5169, Accuracy: 99.80%

Test set: Average loss: 0.5242, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.518598
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.522825
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.519566
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.517295
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.510875
Train Epoch: 29 - Loss: 0.5160, Accuracy: 99.82%

Test set: Average loss: 0.5233, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.509466
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.512625
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.515322
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.514355
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.508536
Train Epoch: 30 - Loss: 0.5152, Accuracy: 99.83%

Test set: Average loss: 0.5234, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.509405
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.510765
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.510014
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.514300
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.522147
Train Epoch: 31 - Loss: 0.5144, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.508547
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.516387
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.513952
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514850
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.509970
Train Epoch: 32 - Loss: 0.5136, Accuracy: 99.88%

Test set: Average loss: 0.5219, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.510923
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.523970
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.514017
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.509745
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.513233
Train Epoch: 33 - Loss: 0.5133, Accuracy: 99.89%

Test set: Average loss: 0.5216, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.514340
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.514630
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.514205
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.514628
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.518183
Train Epoch: 34 - Loss: 0.5132, Accuracy: 99.87%

Test set: Average loss: 0.5211, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.508424
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.516128
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.508103
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.517712
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.508379
Train Epoch: 35 - Loss: 0.5121, Accuracy: 99.92%

Test set: Average loss: 0.5210, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.532555
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.511348
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.512850
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.507841
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.510125
Train Epoch: 36 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5211, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.508563
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.507091
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.526564
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.512927
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.508963
Train Epoch: 37 - Loss: 0.5115, Accuracy: 99.90%

Test set: Average loss: 0.5208, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.506710
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.509804
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.507087
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.510290
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.506274
Train Epoch: 38 - Loss: 0.5110, Accuracy: 99.92%

Test set: Average loss: 0.5205, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.516267
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.506519
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.508655
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.514065
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.509122
Train Epoch: 39 - Loss: 0.5109, Accuracy: 99.92%

Test set: Average loss: 0.5205, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.510612
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.509634
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.513933
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.511817
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.510690
Train Epoch: 40 - Loss: 0.5109, Accuracy: 99.92%

Test set: Average loss: 0.5205, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.516926
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.510807
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.507588
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.506708
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.513249
Train Epoch: 41 - Loss: 0.5105, Accuracy: 99.94%

Test set: Average loss: 0.5204, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.526825
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.506312
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.507147
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.507526
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.521867
Train Epoch: 42 - Loss: 0.5101, Accuracy: 99.93%

Test set: Average loss: 0.5205, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.507887
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.509081
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.507201
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.507344
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.523306
Train Epoch: 43 - Loss: 0.5101, Accuracy: 99.96%

Test set: Average loss: 0.5201, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.518456
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.511953
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.507932
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.514876
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.509352
Train Epoch: 44 - Loss: 0.5099, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.506284
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.509100
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.509322
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.505939
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.507245
Train Epoch: 45 - Loss: 0.5099, Accuracy: 99.94%

Test set: Average loss: 0.5199, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.505700
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.507417
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.512725
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.506055
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.505809
Train Epoch: 46 - Loss: 0.5098, Accuracy: 99.94%

Test set: Average loss: 0.5200, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.510823
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.507023
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.507078
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.506246
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.509672
Train Epoch: 47 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5198, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.506717
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.515650
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.508996
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.510380
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.510703
Train Epoch: 48 - Loss: 0.5096, Accuracy: 99.96%

Test set: Average loss: 0.5201, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.510059
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.508188
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.510717
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.510676
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.513222
Train Epoch: 49 - Loss: 0.5096, Accuracy: 99.96%

Test set: Average loss: 0.5203, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.508208
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.507370
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.505353
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.508922
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.513642
Train Epoch: 50 - Loss: 0.5096, Accuracy: 99.96%

Test set: Average loss: 0.5197, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.508205
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.509574
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.512217
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.510251
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.507398
Train Epoch: 51 - Loss: 0.5108, Accuracy: 99.92%

Test set: Average loss: 0.5199, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.507977
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.509563
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.507436
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.520468
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.506595
Train Epoch: 52 - Loss: 0.5105, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.507894
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.507334
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.510117
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509376
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.508846
Train Epoch: 53 - Loss: 0.5104, Accuracy: 99.92%

Test set: Average loss: 0.5207, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.508227
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.513514
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.513247
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513283
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.509469
Train Epoch: 54 - Loss: 0.5098, Accuracy: 99.95%

Test set: Average loss: 0.5207, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.508120
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.506272
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.509288
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.505813
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.511504
Train Epoch: 55 - Loss: 0.5098, Accuracy: 99.94%

Test set: Average loss: 0.5200, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.507467
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.509756
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.507349
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.505708
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.509903
Train Epoch: 56 - Loss: 0.5094, Accuracy: 99.93%

Test set: Average loss: 0.5198, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.505068
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511924
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.511584
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.506961
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.507386
Train Epoch: 57 - Loss: 0.5091, Accuracy: 99.94%

Test set: Average loss: 0.5194, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.507030
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.510782
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.505527
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.509872
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.508691
Train Epoch: 58 - Loss: 0.5089, Accuracy: 99.96%

Test set: Average loss: 0.5198, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.506351
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.505467
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.512713
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.506636
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.514177
Train Epoch: 59 - Loss: 0.5085, Accuracy: 99.96%

Test set: Average loss: 0.5198, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.506869
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.505889
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.507210
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.508619
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.507268
Train Epoch: 60 - Loss: 0.5083, Accuracy: 99.95%

Test set: Average loss: 0.5192, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.505587
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.509015
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.510112
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.506041
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.505138
Train Epoch: 61 - Loss: 0.5081, Accuracy: 99.96%

Test set: Average loss: 0.5193, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.505633
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506477
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.506316
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.505600
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.507448
Train Epoch: 62 - Loss: 0.5078, Accuracy: 99.97%

Test set: Average loss: 0.5188, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.506382
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506156
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.504337
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.504890
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.509644
Train Epoch: 63 - Loss: 0.5077, Accuracy: 99.97%

Test set: Average loss: 0.5190, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.509437
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508394
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506830
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.507407
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.509396
Train Epoch: 64 - Loss: 0.5075, Accuracy: 99.97%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.505893
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.506452
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.505314
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.507218
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506171
Train Epoch: 65 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5189, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.506343
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.509063
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.505957
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.507335
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506253
Train Epoch: 66 - Loss: 0.5074, Accuracy: 99.98%

Test set: Average loss: 0.5189, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.511315
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.505973
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.506391
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.504707
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506774
Train Epoch: 67 - Loss: 0.5072, Accuracy: 99.97%

Test set: Average loss: 0.5186, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510694
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.505579
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.507917
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515025
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.507044
Train Epoch: 68 - Loss: 0.5072, Accuracy: 99.98%

Test set: Average loss: 0.5185, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.504371
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.515753
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.506819
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.505034
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.507019
Train Epoch: 69 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5184, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.506020
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.504410
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.511701
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.507459
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.512434
Train Epoch: 70 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5184, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.504827
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.507048
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.507429
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.506821
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.504521
Train Epoch: 71 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5187, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511045
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509024
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.504959
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.507890
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508805
Train Epoch: 72 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5185, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.513127
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508488
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508056
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505600
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.506779
Train Epoch: 73 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5186, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511581
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.505236
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506327
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.507972
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.504427
Train Epoch: 74 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5189, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.508250
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.505709
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.503859
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.504858
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.513414
Train Epoch: 75 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5187, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.506722
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.509130
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.506279
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.507892
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.505299
Train Epoch: 76 - Loss: 0.5076, Accuracy: 99.97%

Test set: Average loss: 0.5195, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.506458
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.506650
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.513054
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.504828
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.504945
Train Epoch: 77 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5188, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.506264
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.504519
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.506979
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.511769
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.504748
Train Epoch: 78 - Loss: 0.5074, Accuracy: 99.97%

Test set: Average loss: 0.5180, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.504527
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.510169
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.507736
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.506194
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.505596
Train Epoch: 79 - Loss: 0.5072, Accuracy: 99.97%

Test set: Average loss: 0.5187, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.505527
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507386
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.505218
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.509849
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.518627
Train Epoch: 80 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5198, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.505556
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.504866
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.503835
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.505325
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.508312
Train Epoch: 81 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5184, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.504275
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.504154
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.505000
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.505366
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.504709
Train Epoch: 82 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5191, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.506385
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.505096
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506222
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.503312
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.507380
Train Epoch: 83 - Loss: 0.5066, Accuracy: 99.98%

Test set: Average loss: 0.5189, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.505265
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509448
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.505265
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.504803
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.506167
Train Epoch: 84 - Loss: 0.5065, Accuracy: 99.98%

Test set: Average loss: 0.5192, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.511784
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.506067
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.506089
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.506432
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.508360
Train Epoch: 85 - Loss: 0.5063, Accuracy: 99.99%

Test set: Average loss: 0.5180, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.505855
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.507298
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.507120
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506669
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.504639
Train Epoch: 86 - Loss: 0.5062, Accuracy: 99.99%

Test set: Average loss: 0.5187, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.505013
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506097
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.503307
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.503473
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.506561
Train Epoch: 87 - Loss: 0.5060, Accuracy: 99.98%

Test set: Average loss: 0.5181, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504530
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.514039
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505165
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.508551
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.503783
Train Epoch: 88 - Loss: 0.5058, Accuracy: 99.99%

Test set: Average loss: 0.5182, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.508566
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505992
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.505740
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.503416
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.509160
Train Epoch: 89 - Loss: 0.5059, Accuracy: 99.99%

Test set: Average loss: 0.5183, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.504753
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.506635
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.503953
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.505937
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.504439
Train Epoch: 90 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5178, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.502943
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.507146
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.504437
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.504475
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.513537
Train Epoch: 91 - Loss: 0.5057, Accuracy: 99.99%

Test set: Average loss: 0.5179, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.512148
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504623
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.503870
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.504107
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.506468
Train Epoch: 92 - Loss: 0.5055, Accuracy: 100.00%

Test set: Average loss: 0.5179, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.507581
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504899
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.504207
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.504256
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505280
Train Epoch: 93 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5179, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.504962
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504825
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.504233
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.503657
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.503068
Train Epoch: 94 - Loss: 0.5054, Accuracy: 99.99%

Test set: Average loss: 0.5182, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506118
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.507245
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.503631
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.503982
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509679
Train Epoch: 95 - Loss: 0.5054, Accuracy: 100.00%

Test set: Average loss: 0.5179, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.505171
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.503945
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.506268
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.506135
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.505577
Train Epoch: 96 - Loss: 0.5053, Accuracy: 100.00%

Test set: Average loss: 0.5180, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506521
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508408
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.503439
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.504253
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504402
Train Epoch: 97 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5182, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.505847
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.503637
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.511245
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.504402
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.506177
Train Epoch: 98 - Loss: 0.5052, Accuracy: 100.00%

Test set: Average loss: 0.5177, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506772
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504534
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.505537
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.504831
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503742
Train Epoch: 99 - Loss: 0.5053, Accuracy: 100.00%

Test set: Average loss: 0.5182, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.505034
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.503150
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.504348
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504541
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.504664
Train Epoch: 100 - Loss: 0.5052, Accuracy: 100.00%

Test set: Average loss: 0.5177, Accuracy: 9938/10000 (99.38%)


Training completed. Best Accuracy: 99.47%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 6.252336
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.895338
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.729025
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.648872
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.627979
Train Epoch: 1 - Loss: 0.8360, Accuracy: 92.28%

Test set: Average loss: 0.5972, Accuracy: 9794/10000 (97.94%)

New best model saved with accuracy: 97.94%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.567868
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.619441
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.594865
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.562687
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.557002
Train Epoch: 2 - Loss: 0.5827, Accuracy: 98.31%

Test set: Average loss: 0.5621, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.572763
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.579397
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.546895
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.535154
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.547765
Train Epoch: 3 - Loss: 0.5591, Accuracy: 98.85%

Test set: Average loss: 0.5487, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.541024
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.530894
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.539586
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.534703
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.576478
Train Epoch: 4 - Loss: 0.5472, Accuracy: 99.10%

Test set: Average loss: 0.5415, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.554201
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.526079
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.528143
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.546114
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.533203
Train Epoch: 5 - Loss: 0.5401, Accuracy: 99.26%

Test set: Average loss: 0.5382, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.538124
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.536816
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.562218
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.534420
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.575852
Train Epoch: 6 - Loss: 0.5344, Accuracy: 99.39%

Test set: Average loss: 0.5340, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.526426
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.531173
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.524799
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.574228
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.523424
Train Epoch: 7 - Loss: 0.5308, Accuracy: 99.47%

Test set: Average loss: 0.5319, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.531890
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.526110
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.523973
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.527935
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.522335
Train Epoch: 8 - Loss: 0.5286, Accuracy: 99.53%

Test set: Average loss: 0.5302, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.522111
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.523852
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.530051
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.523891
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.533108
Train Epoch: 9 - Loss: 0.5261, Accuracy: 99.56%

Test set: Average loss: 0.5283, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.522170
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.529513
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.524478
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.518943
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.524257
Train Epoch: 10 - Loss: 0.5244, Accuracy: 99.60%

Test set: Average loss: 0.5283, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.522096
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.512191
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.542493
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.523633
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.529944
Train Epoch: 11 - Loss: 0.5227, Accuracy: 99.66%

Test set: Average loss: 0.5261, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.524611
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.513478
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.519102
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.531906
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.527493
Train Epoch: 12 - Loss: 0.5217, Accuracy: 99.67%

Test set: Average loss: 0.5258, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.511367
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.511054
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.509873
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.512524
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.534086
Train Epoch: 13 - Loss: 0.5201, Accuracy: 99.72%

Test set: Average loss: 0.5254, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.512151
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.519315
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.522112
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.516191
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.515255
Train Epoch: 14 - Loss: 0.5195, Accuracy: 99.76%

Test set: Average loss: 0.5243, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.527519
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.516507
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.513055
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.512360
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.512109
Train Epoch: 15 - Loss: 0.5189, Accuracy: 99.75%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.514536
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.519015
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.515922
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.512317
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.511084
Train Epoch: 16 - Loss: 0.5181, Accuracy: 99.78%

Test set: Average loss: 0.5236, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.519166
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.517233
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.514816
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.523675
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.509590
Train Epoch: 17 - Loss: 0.5180, Accuracy: 99.77%

Test set: Average loss: 0.5236, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.515813
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.514365
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.520120
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.511473
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.523039
Train Epoch: 18 - Loss: 0.5176, Accuracy: 99.78%

Test set: Average loss: 0.5237, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.514309
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.516882
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.522313
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.513762
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.516867
Train Epoch: 19 - Loss: 0.5169, Accuracy: 99.79%

Test set: Average loss: 0.5230, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.525701
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.515756
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.516143
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.511070
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.520965
Train Epoch: 20 - Loss: 0.5168, Accuracy: 99.82%

Test set: Average loss: 0.5230, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.513174
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.509703
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.515615
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.509350
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.521827
Train Epoch: 21 - Loss: 0.5163, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.519126
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.512391
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.512248
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.510973
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.517926
Train Epoch: 22 - Loss: 0.5165, Accuracy: 99.80%

Test set: Average loss: 0.5230, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.510990
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.513159
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.515805
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.512660
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.511458
Train Epoch: 23 - Loss: 0.5161, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.512157
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.526262
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.512930
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.521247
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.521934
Train Epoch: 24 - Loss: 0.5161, Accuracy: 99.81%

Test set: Average loss: 0.5227, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.509721
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.511429
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.509223
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.523461
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.519267
Train Epoch: 25 - Loss: 0.5160, Accuracy: 99.84%

Test set: Average loss: 0.5229, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.520833
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.517541
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.515789
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.510434
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.510682
Train Epoch: 26 - Loss: 0.5184, Accuracy: 99.74%

Test set: Average loss: 0.5235, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.541834
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.525129
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.509764
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.517137
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.523611
Train Epoch: 27 - Loss: 0.5173, Accuracy: 99.77%

Test set: Average loss: 0.5233, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.516768
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.530971
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.509895
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.532961
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.519242
Train Epoch: 28 - Loss: 0.5164, Accuracy: 99.80%

Test set: Average loss: 0.5225, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.509557
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.511423
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.512123
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.512132
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.511391
Train Epoch: 29 - Loss: 0.5157, Accuracy: 99.82%

Test set: Average loss: 0.5219, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.509833
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.511837
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.510985
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.514751
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.525724
Train Epoch: 30 - Loss: 0.5146, Accuracy: 99.83%

Test set: Average loss: 0.5224, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.509126
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.517708
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.515136
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.509864
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.528959
Train Epoch: 31 - Loss: 0.5139, Accuracy: 99.85%

Test set: Average loss: 0.5217, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.509233
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.511555
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.514735
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.519476
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.513357
Train Epoch: 32 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5204, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.510925
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.515021
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.508141
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.508295
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.529297
Train Epoch: 33 - Loss: 0.5129, Accuracy: 99.89%

Test set: Average loss: 0.5200, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.518428
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.515144
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.514026
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.521568
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.513754
Train Epoch: 34 - Loss: 0.5123, Accuracy: 99.90%

Test set: Average loss: 0.5203, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.509127
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.510410
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.508047
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.509659
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.511069
Train Epoch: 35 - Loss: 0.5119, Accuracy: 99.90%

Test set: Average loss: 0.5195, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.512325
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509890
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.508841
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.509026
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.516607
Train Epoch: 36 - Loss: 0.5116, Accuracy: 99.90%

Test set: Average loss: 0.5193, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.514950
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.514476
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.517230
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.513039
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.528940
Train Epoch: 37 - Loss: 0.5112, Accuracy: 99.92%

Test set: Average loss: 0.5191, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.510174
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.511598
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.510330
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.510839
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.509428
Train Epoch: 38 - Loss: 0.5107, Accuracy: 99.92%

Test set: Average loss: 0.5190, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.507535
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.507966
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.508636
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.508311
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516505
Train Epoch: 39 - Loss: 0.5107, Accuracy: 99.92%

Test set: Average loss: 0.5192, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.509964
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.510778
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.507663
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.510175
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.510646
Train Epoch: 40 - Loss: 0.5102, Accuracy: 99.93%

Test set: Average loss: 0.5186, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.507493
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.513330
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.508368
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.505962
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.505946
Train Epoch: 41 - Loss: 0.5103, Accuracy: 99.92%

Test set: Average loss: 0.5186, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.505131
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.507434
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.511680
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.506524
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.512008
Train Epoch: 42 - Loss: 0.5100, Accuracy: 99.94%

Test set: Average loss: 0.5184, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.507813
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.511083
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.505293
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.507590
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.518818
Train Epoch: 43 - Loss: 0.5098, Accuracy: 99.94%

Test set: Average loss: 0.5186, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.506541
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.507786
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.511162
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.506103
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.510197
Train Epoch: 44 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5183, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.509175
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.506147
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.507748
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.506461
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.511585
Train Epoch: 45 - Loss: 0.5096, Accuracy: 99.95%

Test set: Average loss: 0.5186, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.507034
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.507497
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.513751
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.506342
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.508980
Train Epoch: 46 - Loss: 0.5095, Accuracy: 99.96%

Test set: Average loss: 0.5183, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508130
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.514601
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.506191
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.509605
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.506921
Train Epoch: 47 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5183, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.507879
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.507934
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.507397
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.508615
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.515848
Train Epoch: 48 - Loss: 0.5094, Accuracy: 99.95%

Test set: Average loss: 0.5181, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.506799
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.508425
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.508408
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.509868
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.517299
Train Epoch: 49 - Loss: 0.5094, Accuracy: 99.96%

Test set: Average loss: 0.5182, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.508183
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.509580
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.508610
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.510353
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.506836
Train Epoch: 50 - Loss: 0.5094, Accuracy: 99.95%

Test set: Average loss: 0.5184, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.505813
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.509202
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.507582
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.507137
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.515369
Train Epoch: 51 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5195, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.507859
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.510419
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.506755
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.511437
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513730
Train Epoch: 52 - Loss: 0.5102, Accuracy: 99.94%

Test set: Average loss: 0.5190, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.508788
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.508399
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.506265
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.518418
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.507195
Train Epoch: 53 - Loss: 0.5100, Accuracy: 99.93%

Test set: Average loss: 0.5186, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.508161
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.540464
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.507663
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.511067
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.509852
Train Epoch: 54 - Loss: 0.5096, Accuracy: 99.93%

Test set: Average loss: 0.5193, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.508932
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.509928
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.518140
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.509962
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.506817
Train Epoch: 55 - Loss: 0.5092, Accuracy: 99.95%

Test set: Average loss: 0.5179, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.507584
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.505941
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514409
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.509851
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.508048
Train Epoch: 56 - Loss: 0.5091, Accuracy: 99.95%

Test set: Average loss: 0.5181, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.510118
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511508
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.507027
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.511013
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.509103
Train Epoch: 57 - Loss: 0.5087, Accuracy: 99.96%

Test set: Average loss: 0.5178, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.511856
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.505531
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.507290
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.505723
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.505667
Train Epoch: 58 - Loss: 0.5084, Accuracy: 99.96%

Test set: Average loss: 0.5178, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.512738
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.508817
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.511046
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.511753
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.505256
Train Epoch: 59 - Loss: 0.5083, Accuracy: 99.96%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.504155
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.506463
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.505930
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.510733
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.508974
Train Epoch: 60 - Loss: 0.5081, Accuracy: 99.96%

Test set: Average loss: 0.5171, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.511195
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.505391
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.507133
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.511864
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.506895
Train Epoch: 61 - Loss: 0.5080, Accuracy: 99.97%

Test set: Average loss: 0.5171, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510051
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.505828
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.506375
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.507229
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509632
Train Epoch: 62 - Loss: 0.5075, Accuracy: 99.98%

Test set: Average loss: 0.5168, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.507246
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.518658
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.505705
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.505607
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.520301
Train Epoch: 63 - Loss: 0.5075, Accuracy: 99.97%

Test set: Average loss: 0.5176, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.504350
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508934
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506807
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.511553
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.507635
Train Epoch: 64 - Loss: 0.5073, Accuracy: 99.97%

Test set: Average loss: 0.5167, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.504316
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509868
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.507168
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.507062
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506562
Train Epoch: 65 - Loss: 0.5073, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.506817
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.505759
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.506895
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.507382
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506614
Train Epoch: 66 - Loss: 0.5071, Accuracy: 99.97%

Test set: Average loss: 0.5168, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.513537
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.506297
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.504391
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.504722
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506910
Train Epoch: 67 - Loss: 0.5069, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.505270
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.507520
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.505948
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.505621
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.505070
Train Epoch: 68 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5169, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.504877
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.506652
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.505433
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.506079
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506372
Train Epoch: 69 - Loss: 0.5068, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.507359
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.505325
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.507455
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.515795
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.509174
Train Epoch: 70 - Loss: 0.5067, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.504789
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.508582
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.506002
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.513315
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.506029
Train Epoch: 71 - Loss: 0.5068, Accuracy: 99.99%

Test set: Average loss: 0.5166, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.503840
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.505609
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.528053
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.505444
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511969
Train Epoch: 72 - Loss: 0.5066, Accuracy: 99.98%

Test set: Average loss: 0.5165, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.506417
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.505019
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.505994
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.505603
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.508995
Train Epoch: 73 - Loss: 0.5066, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.506186
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.506601
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.504437
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.504738
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.504740
Train Epoch: 74 - Loss: 0.5066, Accuracy: 99.99%

Test set: Average loss: 0.5165, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.507933
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.508657
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.504965
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506108
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.511059
Train Epoch: 75 - Loss: 0.5066, Accuracy: 99.98%

Test set: Average loss: 0.5165, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.505568
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.534776
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.513172
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.504499
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507307
Train Epoch: 76 - Loss: 0.5075, Accuracy: 99.98%

Test set: Average loss: 0.5164, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.508098
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.504418
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.505172
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.504538
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.507675
Train Epoch: 77 - Loss: 0.5074, Accuracy: 99.96%

Test set: Average loss: 0.5169, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505023
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.503279
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.510685
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.508997
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.510130
Train Epoch: 78 - Loss: 0.5070, Accuracy: 99.97%

Test set: Average loss: 0.5165, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.503872
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.504507
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.506589
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.504246
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.506331
Train Epoch: 79 - Loss: 0.5070, Accuracy: 99.98%

Test set: Average loss: 0.5167, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.506036
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507934
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.504000
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.512144
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.504217
Train Epoch: 80 - Loss: 0.5066, Accuracy: 99.97%

Test set: Average loss: 0.5168, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.506599
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.508864
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.507498
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.508592
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.505827
Train Epoch: 81 - Loss: 0.5067, Accuracy: 99.97%

Test set: Average loss: 0.5163, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.507352
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.512845
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.510373
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.506205
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.506947
Train Epoch: 82 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5165, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.505406
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.508987
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.504049
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.504636
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.506993
Train Epoch: 83 - Loss: 0.5064, Accuracy: 99.98%

Test set: Average loss: 0.5162, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509286
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.504806
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.506098
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509959
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.505631
Train Epoch: 84 - Loss: 0.5061, Accuracy: 99.98%

Test set: Average loss: 0.5160, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.502726
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.503782
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.509072
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.504148
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.506437
Train Epoch: 85 - Loss: 0.5059, Accuracy: 99.99%

Test set: Average loss: 0.5160, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.503852
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.502696
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.509307
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.502920
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.505511
Train Epoch: 86 - Loss: 0.5059, Accuracy: 99.99%

Test set: Average loss: 0.5158, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.504403
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.505463
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.504923
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.504780
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.504283
Train Epoch: 87 - Loss: 0.5058, Accuracy: 99.98%

Test set: Average loss: 0.5166, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504354
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.505420
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.506373
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.504706
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505754
Train Epoch: 88 - Loss: 0.5056, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.504634
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.504734
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.504882
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.505544
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.503377
Train Epoch: 89 - Loss: 0.5056, Accuracy: 99.98%

Test set: Average loss: 0.5157, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.505053
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.505286
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.505572
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.505418
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.503671
Train Epoch: 90 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5158, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504499
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.505508
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.505667
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.503271
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.507086
Train Epoch: 91 - Loss: 0.5055, Accuracy: 99.99%

Test set: Average loss: 0.5159, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504516
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.503367
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.503958
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.506293
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.503001
Train Epoch: 92 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.503930
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504796
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.504427
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.507865
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.504072
Train Epoch: 93 - Loss: 0.5053, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505859
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.502861
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.505290
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.504847
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.504379
Train Epoch: 94 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5155, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504608
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.505430
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.504355
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.509853
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.504783
Train Epoch: 95 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5158, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.504853
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508264
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.504481
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.505203
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.503528
Train Epoch: 96 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5154, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505154
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.502962
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.503790
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.503410
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.505353
Train Epoch: 97 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5156, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.503152
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.503785
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508063
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.506931
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.505262
Train Epoch: 98 - Loss: 0.5052, Accuracy: 100.00%

Test set: Average loss: 0.5155, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.503322
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.503551
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506929
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.507280
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.507254
Train Epoch: 99 - Loss: 0.5052, Accuracy: 99.99%

Test set: Average loss: 0.5155, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506450
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.510542
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.503839
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.505404
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505313
Train Epoch: 100 - Loss: 0.5051, Accuracy: 99.99%

Test set: Average loss: 0.5154, Accuracy: 9945/10000 (99.45%)


Training completed. Best Accuracy: 99.49%
