Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.310777
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.885016
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.332534
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.304247
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.276962
Train Epoch: 1 - Loss: 0.7211, Accuracy: 79.61%

Test set: Average loss: 0.2485, Accuracy: 9279/10000 (92.79%)

New best model saved with accuracy: 92.79%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.254804
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.211932
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.177921
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.165479
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.125537
Train Epoch: 2 - Loss: 0.2007, Accuracy: 93.97%

Test set: Average loss: 0.1377, Accuracy: 9587/10000 (95.87%)

New best model saved with accuracy: 95.87%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.132353
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.180946
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.067077
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.100945
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.103950
Train Epoch: 3 - Loss: 0.1279, Accuracy: 96.13%

Test set: Average loss: 0.0998, Accuracy: 9683/10000 (96.83%)

New best model saved with accuracy: 96.83%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.096769
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.070481
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.073802
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.073784
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.081524
Train Epoch: 4 - Loss: 0.0953, Accuracy: 97.13%

Test set: Average loss: 0.0772, Accuracy: 9749/10000 (97.49%)

New best model saved with accuracy: 97.49%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.049494
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.067812
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.056152
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.085851
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.038520
Train Epoch: 5 - Loss: 0.0771, Accuracy: 97.68%

Test set: Average loss: 0.0717, Accuracy: 9769/10000 (97.69%)

New best model saved with accuracy: 97.69%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.075722
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.093708
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.173426
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.033587
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.073625
Train Epoch: 6 - Loss: 0.0658, Accuracy: 98.05%

Test set: Average loss: 0.0573, Accuracy: 9814/10000 (98.14%)

New best model saved with accuracy: 98.14%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.029959
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.041779
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.056614
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.021820
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.065067
Train Epoch: 7 - Loss: 0.0574, Accuracy: 98.28%

Test set: Average loss: 0.0522, Accuracy: 9837/10000 (98.37%)

New best model saved with accuracy: 98.37%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.030573
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.116478
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.035449
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.069210
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.037048
Train Epoch: 8 - Loss: 0.0507, Accuracy: 98.50%

Test set: Average loss: 0.0512, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.065077
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.098700
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.031984
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.040937
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.007685
Train Epoch: 9 - Loss: 0.0463, Accuracy: 98.57%

Test set: Average loss: 0.0484, Accuracy: 9833/10000 (98.33%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.007823
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.027360
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.057856
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.066947
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.017514
Train Epoch: 10 - Loss: 0.0420, Accuracy: 98.78%

Test set: Average loss: 0.0466, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.067097
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.028503
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.031150
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.017985
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.079737
Train Epoch: 11 - Loss: 0.0386, Accuracy: 98.89%

Test set: Average loss: 0.0424, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.028246
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.006061
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.075883
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.042387
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.018064
Train Epoch: 12 - Loss: 0.0363, Accuracy: 98.94%

Test set: Average loss: 0.0402, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.025128
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.027384
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.008057
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.040744
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.044268
Train Epoch: 13 - Loss: 0.0337, Accuracy: 99.03%

Test set: Average loss: 0.0399, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.020322
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.046706
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.025043
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.032929
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.044498
Train Epoch: 14 - Loss: 0.0317, Accuracy: 99.11%

Test set: Average loss: 0.0375, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.032744
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.007190
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.017586
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.006321
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.043309
Train Epoch: 15 - Loss: 0.0302, Accuracy: 99.14%

Test set: Average loss: 0.0375, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.045234
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.058024
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.008576
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.023401
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.026215
Train Epoch: 16 - Loss: 0.0287, Accuracy: 99.17%

Test set: Average loss: 0.0371, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.031780
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.020883
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.029500
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.015199
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.029480
Train Epoch: 17 - Loss: 0.0273, Accuracy: 99.25%

Test set: Average loss: 0.0352, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.014831
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.008331
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.008588
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.051779
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.073031
Train Epoch: 18 - Loss: 0.0264, Accuracy: 99.27%

Test set: Average loss: 0.0359, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.006384
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.015321
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.004426
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.039738
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.014850
Train Epoch: 19 - Loss: 0.0256, Accuracy: 99.30%

Test set: Average loss: 0.0352, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.025534
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.062287
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.043392
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.018102
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.047247
Train Epoch: 20 - Loss: 0.0249, Accuracy: 99.33%

Test set: Average loss: 0.0348, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.024894
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.048566
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.014525
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.053174
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.050993
Train Epoch: 21 - Loss: 0.0244, Accuracy: 99.33%

Test set: Average loss: 0.0346, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.009845
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.003717
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.024207
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.008684
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.022247
Train Epoch: 22 - Loss: 0.0240, Accuracy: 99.34%

Test set: Average loss: 0.0347, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.026430
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.012001
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.010244
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.014323
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.021791
Train Epoch: 23 - Loss: 0.0237, Accuracy: 99.35%

Test set: Average loss: 0.0346, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.010241
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.069796
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.032526
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.063682
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.025432
Train Epoch: 24 - Loss: 0.0235, Accuracy: 99.37%

Test set: Average loss: 0.0346, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.040320
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.024288
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.013211
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.010209
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.037971
Train Epoch: 25 - Loss: 0.0234, Accuracy: 99.37%

Test set: Average loss: 0.0346, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.103040
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.024928
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.010559
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.040186
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.046098
Train Epoch: 26 - Loss: 0.0295, Accuracy: 99.12%

Test set: Average loss: 0.0356, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.012160
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.006642
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.025828
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.003588
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.010235
Train Epoch: 27 - Loss: 0.0278, Accuracy: 99.16%

Test set: Average loss: 0.0368, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.019689
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.025227
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.013637
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.013112
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.007512
Train Epoch: 28 - Loss: 0.0260, Accuracy: 99.21%

Test set: Average loss: 0.0343, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.010614
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.010640
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.061739
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.072102
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.039644
Train Epoch: 29 - Loss: 0.0234, Accuracy: 99.31%

Test set: Average loss: 0.0400, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.013492
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.007921
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.021648
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.031917
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.020899
Train Epoch: 30 - Loss: 0.0220, Accuracy: 99.38%

Test set: Average loss: 0.0359, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.024895
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.030034
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.018037
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.004727
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.015569
Train Epoch: 31 - Loss: 0.0202, Accuracy: 99.42%

Test set: Average loss: 0.0386, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.022340
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.010767
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.018220
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.024808
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.005893
Train Epoch: 32 - Loss: 0.0189, Accuracy: 99.47%

Test set: Average loss: 0.0338, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.007986
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.004865
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.005910
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.011891
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.022011
Train Epoch: 33 - Loss: 0.0172, Accuracy: 99.53%

Test set: Average loss: 0.0346, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.006879
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.013838
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.006318
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.014955
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.009248
Train Epoch: 34 - Loss: 0.0163, Accuracy: 99.56%

Test set: Average loss: 0.0362, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.019989
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.004622
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.008531
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.003453
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.013906
Train Epoch: 35 - Loss: 0.0149, Accuracy: 99.59%

Test set: Average loss: 0.0349, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.005358
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.013195
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.007360
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.003327
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.010046
Train Epoch: 36 - Loss: 0.0140, Accuracy: 99.62%

Test set: Average loss: 0.0313, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.005580
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.004442
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.036797
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.011227
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.024022
Train Epoch: 37 - Loss: 0.0129, Accuracy: 99.69%

Test set: Average loss: 0.0352, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.015385
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.004285
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.008600
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.004528
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.015350
Train Epoch: 38 - Loss: 0.0122, Accuracy: 99.70%

Test set: Average loss: 0.0317, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.009639
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.007191
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.006069
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.010941
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.003177
Train Epoch: 39 - Loss: 0.0115, Accuracy: 99.73%

Test set: Average loss: 0.0313, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.007374
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.002822
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.007438
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.004805
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.003425
Train Epoch: 40 - Loss: 0.0109, Accuracy: 99.75%

Test set: Average loss: 0.0311, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.013751
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.005782
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.011685
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.011117
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.025814
Train Epoch: 41 - Loss: 0.0103, Accuracy: 99.79%

Test set: Average loss: 0.0326, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.030491
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.007709
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.010073
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.018926
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.008977
Train Epoch: 42 - Loss: 0.0099, Accuracy: 99.80%

Test set: Average loss: 0.0311, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.006567
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.004732
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.007653
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.005743
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.003283
Train Epoch: 43 - Loss: 0.0095, Accuracy: 99.84%

Test set: Average loss: 0.0305, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.006219
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.017789
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.006457
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.008846
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.006060
Train Epoch: 44 - Loss: 0.0093, Accuracy: 99.82%

Test set: Average loss: 0.0315, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.025814
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.007743
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.004824
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.014747
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.007829
Train Epoch: 45 - Loss: 0.0090, Accuracy: 99.84%

Test set: Average loss: 0.0306, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.006235
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.002811
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.002235
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.005637
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.002580
Train Epoch: 46 - Loss: 0.0088, Accuracy: 99.83%

Test set: Average loss: 0.0303, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.019983
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.003373
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.003688
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.006530
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.002116
Train Epoch: 47 - Loss: 0.0087, Accuracy: 99.84%

Test set: Average loss: 0.0305, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.009026
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.002722
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.005737
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.014035
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.013848
Train Epoch: 48 - Loss: 0.0086, Accuracy: 99.85%

Test set: Average loss: 0.0303, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.003534
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.005263
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.004131
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.007895
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.003297
Train Epoch: 49 - Loss: 0.0085, Accuracy: 99.85%

Test set: Average loss: 0.0304, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.004776
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.001005
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.010907
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.006476
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.008688
Train Epoch: 50 - Loss: 0.0084, Accuracy: 99.85%

Test set: Average loss: 0.0304, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.000913
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.007910
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.007454
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.011758
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.006109
Train Epoch: 51 - Loss: 0.0117, Accuracy: 99.70%

Test set: Average loss: 0.0318, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.005570
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.012240
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.005929
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.012394
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.007499
Train Epoch: 52 - Loss: 0.0116, Accuracy: 99.70%

Test set: Average loss: 0.0326, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.014543
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.013891
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.014274
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.009656
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.039407
Train Epoch: 53 - Loss: 0.0105, Accuracy: 99.75%

Test set: Average loss: 0.0330, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.010312
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.003186
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.010963
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.006008
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.005172
Train Epoch: 54 - Loss: 0.0097, Accuracy: 99.77%

Test set: Average loss: 0.0319, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.008096
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.004570
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.008986
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.009848
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.010330
Train Epoch: 55 - Loss: 0.0091, Accuracy: 99.80%

Test set: Average loss: 0.0304, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.003208
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.006238
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.001056
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.025541
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.005337
Train Epoch: 56 - Loss: 0.0086, Accuracy: 99.81%

Test set: Average loss: 0.0318, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.007299
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.006963
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.002881
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.001148
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.010079
Train Epoch: 57 - Loss: 0.0080, Accuracy: 99.83%

Test set: Average loss: 0.0318, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.004138
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.024402
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.003327
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.004130
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.004260
Train Epoch: 58 - Loss: 0.0072, Accuracy: 99.84%

Test set: Average loss: 0.0330, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.020850
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.009810
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.001119
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.002341
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.006155
Train Epoch: 59 - Loss: 0.0066, Accuracy: 99.89%

Test set: Average loss: 0.0318, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.005730
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.003294
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.014244
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.002519
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.002807
Train Epoch: 60 - Loss: 0.0062, Accuracy: 99.89%

Test set: Average loss: 0.0328, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.004849
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.006447
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.001401
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.002915
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.006483
Train Epoch: 61 - Loss: 0.0058, Accuracy: 99.91%

Test set: Average loss: 0.0317, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.003486
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.017488
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.001698
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.003908
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.007244
Train Epoch: 62 - Loss: 0.0056, Accuracy: 99.91%

Test set: Average loss: 0.0309, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.001979
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.003543
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.002904
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.002914
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.004003
Train Epoch: 63 - Loss: 0.0052, Accuracy: 99.92%

Test set: Average loss: 0.0329, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.003242
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.003900
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.001713
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.002163
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.003053
Train Epoch: 64 - Loss: 0.0050, Accuracy: 99.94%

Test set: Average loss: 0.0308, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.002726
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.002511
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.003106
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.006090
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.020225
Train Epoch: 65 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0318, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.004143
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.008647
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.001059
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.001793
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.007147
Train Epoch: 66 - Loss: 0.0046, Accuracy: 99.94%

Test set: Average loss: 0.0310, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.006420
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.001478
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.005327
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.009946
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.001124
Train Epoch: 67 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0308, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.003402
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.007028
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.012137
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.000484
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.005091
Train Epoch: 68 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0311, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.003659
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.002271
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.004790
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.003277
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.003956
Train Epoch: 69 - Loss: 0.0041, Accuracy: 99.95%

Test set: Average loss: 0.0311, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.001377
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.005427
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.002598
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.003308
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.001842
Train Epoch: 70 - Loss: 0.0040, Accuracy: 99.95%

Test set: Average loss: 0.0314, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.001319
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.004714
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.002034
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.001628
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.001463
Train Epoch: 71 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0316, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.002725
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.001868
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.001865
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.001695
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.000877
Train Epoch: 72 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0312, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.002458
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.004242
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.002254
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.006100
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001242
Train Epoch: 73 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0314, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.006149
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.003920
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.002695
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.002037
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.006488
Train Epoch: 74 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.004916
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.003840
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.001979
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.001430
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.002338
Train Epoch: 75 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.004205
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.001555
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.003166
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.004065
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.001661
Train Epoch: 76 - Loss: 0.0053, Accuracy: 99.90%

Test set: Average loss: 0.0333, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.003166
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.010490
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.001924
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.002991
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.014173
Train Epoch: 77 - Loss: 0.0055, Accuracy: 99.90%

Test set: Average loss: 0.0332, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.003655
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.013882
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.011611
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.001696
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.000862
Train Epoch: 78 - Loss: 0.0049, Accuracy: 99.92%

Test set: Average loss: 0.0337, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.006276
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.002885
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.008442
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.001997
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.001641
Train Epoch: 79 - Loss: 0.0048, Accuracy: 99.92%

Test set: Average loss: 0.0320, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.002076
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.000457
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.002083
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.002999
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.003520
Train Epoch: 80 - Loss: 0.0043, Accuracy: 99.94%

Test set: Average loss: 0.0321, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.001618
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.021550
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.000609
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.002055
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.002605
Train Epoch: 81 - Loss: 0.0040, Accuracy: 99.95%

Test set: Average loss: 0.0318, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.005495
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002236
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.001740
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.005397
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.001105
Train Epoch: 82 - Loss: 0.0039, Accuracy: 99.94%

Test set: Average loss: 0.0321, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.005968
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.007388
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.002701
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.003434
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.001422
Train Epoch: 83 - Loss: 0.0035, Accuracy: 99.95%

Test set: Average loss: 0.0318, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.000896
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.001964
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.000824
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.002378
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.002829
Train Epoch: 84 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0322, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.002565
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.002503
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.002108
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.001959
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.002568
Train Epoch: 85 - Loss: 0.0030, Accuracy: 99.97%

Test set: Average loss: 0.0332, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.003645
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.002268
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.004155
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.001608
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.000652
Train Epoch: 86 - Loss: 0.0030, Accuracy: 99.97%

Test set: Average loss: 0.0325, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.002004
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.004734
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.006997
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.007393
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.000616
Train Epoch: 87 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0325, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.002240
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001419
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.005460
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.003783
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.002804
Train Epoch: 88 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0324, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.002921
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.001624
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.004647
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.002936
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.000855
Train Epoch: 89 - Loss: 0.0026, Accuracy: 99.98%

Test set: Average loss: 0.0329, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.001122
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.000680
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.001173
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.000447
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.002559
Train Epoch: 90 - Loss: 0.0025, Accuracy: 99.97%

Test set: Average loss: 0.0332, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.003608
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.001292
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.000447
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.004171
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.000835
Train Epoch: 91 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0334, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.002774
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.001621
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001918
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.001824
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.003738
Train Epoch: 92 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0330, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.005818
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.001275
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.001258
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001854
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.001485
Train Epoch: 93 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0321, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.001282
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.001866
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.002916
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.002261
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.000689
Train Epoch: 94 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.003187
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.010266
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.000912
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.000312
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001213
Train Epoch: 95 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0327, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001539
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.000902
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.002799
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001109
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.000803
Train Epoch: 96 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0327, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.001602
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.000755
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.001054
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.001730
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.001210
Train Epoch: 97 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.001782
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.000893
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.005370
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.002917
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.003466
Train Epoch: 98 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.003748
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.000500
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.000946
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.006299
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.002089
Train Epoch: 99 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0325, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.001613
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.000820
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.003201
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001870
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.001266
Train Epoch: 100 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9902/10000 (99.02%)


Training completed. Best Accuracy: 99.07%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.305207
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.821646
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.500433
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.251351
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.163000
Train Epoch: 1 - Loss: 0.7176, Accuracy: 78.84%

Test set: Average loss: 0.2417, Accuracy: 9294/10000 (92.94%)

New best model saved with accuracy: 92.94%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.345798
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.274182
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.166253
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.138243
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.204901
Train Epoch: 2 - Loss: 0.1987, Accuracy: 94.09%

Test set: Average loss: 0.1427, Accuracy: 9588/10000 (95.88%)

New best model saved with accuracy: 95.88%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.237219
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.080619
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.088548
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.042376
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.115914
Train Epoch: 3 - Loss: 0.1282, Accuracy: 96.21%

Test set: Average loss: 0.1009, Accuracy: 9695/10000 (96.95%)

New best model saved with accuracy: 96.95%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.097891
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.038189
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.134521
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.056422
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.132403
Train Epoch: 4 - Loss: 0.0955, Accuracy: 97.13%

Test set: Average loss: 0.0782, Accuracy: 9755/10000 (97.55%)

New best model saved with accuracy: 97.55%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.065530
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.096932
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.066055
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.084471
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.053713
Train Epoch: 5 - Loss: 0.0769, Accuracy: 97.68%

Test set: Average loss: 0.0671, Accuracy: 9787/10000 (97.87%)

New best model saved with accuracy: 97.87%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.050054
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.052095
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.060906
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.062050
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.055223
Train Epoch: 6 - Loss: 0.0650, Accuracy: 98.04%

Test set: Average loss: 0.0556, Accuracy: 9826/10000 (98.26%)

New best model saved with accuracy: 98.26%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.059213
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.045560
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.062431
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.054142
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.065689
Train Epoch: 7 - Loss: 0.0566, Accuracy: 98.27%

Test set: Average loss: 0.0554, Accuracy: 9817/10000 (98.17%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.050925
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.027213
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.033155
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.029213
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.054467
Train Epoch: 8 - Loss: 0.0503, Accuracy: 98.48%

Test set: Average loss: 0.0495, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.029660
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.082006
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.049802
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.036260
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.022501
Train Epoch: 9 - Loss: 0.0452, Accuracy: 98.66%

Test set: Average loss: 0.0443, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.029315
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.027055
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.108172
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.035932
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.026629
Train Epoch: 10 - Loss: 0.0412, Accuracy: 98.75%

Test set: Average loss: 0.0435, Accuracy: 9860/10000 (98.60%)

New best model saved with accuracy: 98.60%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.037223
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.068856
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.037909
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.052326
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.022986
Train Epoch: 11 - Loss: 0.0380, Accuracy: 98.88%

Test set: Average loss: 0.0421, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.023809
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.008504
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.019087
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.015034
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.024659
Train Epoch: 12 - Loss: 0.0353, Accuracy: 98.99%

Test set: Average loss: 0.0400, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.022973
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.022391
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.013434
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.016153
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.019653
Train Epoch: 13 - Loss: 0.0332, Accuracy: 99.00%

Test set: Average loss: 0.0450, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.014558
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.012884
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.037894
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.009854
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.021289
Train Epoch: 14 - Loss: 0.0312, Accuracy: 99.09%

Test set: Average loss: 0.0383, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.043497
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.007710
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.044043
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.053809
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.087137
Train Epoch: 15 - Loss: 0.0296, Accuracy: 99.16%

Test set: Average loss: 0.0374, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.013698
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.012025
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.012995
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.011260
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.030570
Train Epoch: 16 - Loss: 0.0282, Accuracy: 99.19%

Test set: Average loss: 0.0378, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.015610
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.039027
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.013082
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.011081
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.003012
Train Epoch: 17 - Loss: 0.0271, Accuracy: 99.25%

Test set: Average loss: 0.0369, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.052135
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.046861
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.046184
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.062205
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.023022
Train Epoch: 18 - Loss: 0.0260, Accuracy: 99.27%

Test set: Average loss: 0.0359, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.048180
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.028141
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.017073
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.004357
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.026697
Train Epoch: 19 - Loss: 0.0252, Accuracy: 99.28%

Test set: Average loss: 0.0354, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.027321
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.010353
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.029052
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.040278
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.010279
Train Epoch: 20 - Loss: 0.0246, Accuracy: 99.32%

Test set: Average loss: 0.0352, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.014466
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.018650
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.011526
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.018744
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.009372
Train Epoch: 21 - Loss: 0.0240, Accuracy: 99.33%

Test set: Average loss: 0.0352, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.011238
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.043984
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.004712
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.028010
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.037565
Train Epoch: 22 - Loss: 0.0236, Accuracy: 99.33%

Test set: Average loss: 0.0348, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.029285
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.021957
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.011502
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.011472
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.014570
Train Epoch: 23 - Loss: 0.0233, Accuracy: 99.36%

Test set: Average loss: 0.0346, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.013684
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.019345
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.028036
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.011225
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.021388
Train Epoch: 24 - Loss: 0.0231, Accuracy: 99.37%

Test set: Average loss: 0.0347, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.022448
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.016712
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.012103
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.003688
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.027736
Train Epoch: 25 - Loss: 0.0230, Accuracy: 99.37%

Test set: Average loss: 0.0347, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.005217
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.052153
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.025613
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.032941
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.040861
Train Epoch: 26 - Loss: 0.0297, Accuracy: 99.10%

Test set: Average loss: 0.0513, Accuracy: 9828/10000 (98.28%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.057960
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.009659
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.047703
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.026845
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.019833
Train Epoch: 27 - Loss: 0.0282, Accuracy: 99.12%

Test set: Average loss: 0.0403, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.004931
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.040498
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.016638
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.017483
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.008127
Train Epoch: 28 - Loss: 0.0261, Accuracy: 99.20%

Test set: Average loss: 0.0429, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.045162
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.042649
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.033399
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.007116
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.011430
Train Epoch: 29 - Loss: 0.0240, Accuracy: 99.26%

Test set: Average loss: 0.0368, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.020499
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.033987
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.028913
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.027497
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.025716
Train Epoch: 30 - Loss: 0.0220, Accuracy: 99.33%

Test set: Average loss: 0.0346, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.012370
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.010523
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.001850
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.025009
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.009187
Train Epoch: 31 - Loss: 0.0204, Accuracy: 99.39%

Test set: Average loss: 0.0357, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.045077
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.005577
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.030149
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.002854
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.011290
Train Epoch: 32 - Loss: 0.0189, Accuracy: 99.45%

Test set: Average loss: 0.0348, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.007080
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.018233
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.035988
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.013581
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.038476
Train Epoch: 33 - Loss: 0.0177, Accuracy: 99.52%

Test set: Average loss: 0.0330, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.004220
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.018118
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.058016
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.013738
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.013239
Train Epoch: 34 - Loss: 0.0163, Accuracy: 99.56%

Test set: Average loss: 0.0326, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.025198
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.028243
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.103487
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.011230
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.032656
Train Epoch: 35 - Loss: 0.0153, Accuracy: 99.56%

Test set: Average loss: 0.0324, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.010058
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.009177
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.004156
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.007127
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.013666
Train Epoch: 36 - Loss: 0.0143, Accuracy: 99.62%

Test set: Average loss: 0.0316, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.006681
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.004726
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.009067
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.012111
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.009298
Train Epoch: 37 - Loss: 0.0133, Accuracy: 99.67%

Test set: Average loss: 0.0308, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.002511
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.004337
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.004851
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.013401
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.008331
Train Epoch: 38 - Loss: 0.0125, Accuracy: 99.71%

Test set: Average loss: 0.0317, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.025352
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.033775
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.015124
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.005818
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.004102
Train Epoch: 39 - Loss: 0.0117, Accuracy: 99.73%

Test set: Average loss: 0.0311, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.008999
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.029150
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.003976
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.008078
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.005159
Train Epoch: 40 - Loss: 0.0110, Accuracy: 99.77%

Test set: Average loss: 0.0328, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.006048
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.003137
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.010250
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.010274
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.008565
Train Epoch: 41 - Loss: 0.0106, Accuracy: 99.78%

Test set: Average loss: 0.0327, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.003568
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.005490
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.007577
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.005899
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.007043
Train Epoch: 42 - Loss: 0.0101, Accuracy: 99.79%

Test set: Average loss: 0.0309, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.005892
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.023754
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.003648
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.002481
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.047853
Train Epoch: 43 - Loss: 0.0098, Accuracy: 99.80%

Test set: Average loss: 0.0312, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.010550
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.003923
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.005329
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.006477
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.011994
Train Epoch: 44 - Loss: 0.0095, Accuracy: 99.82%

Test set: Average loss: 0.0306, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.006223
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.004424
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.003171
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.006198
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.023064
Train Epoch: 45 - Loss: 0.0092, Accuracy: 99.83%

Test set: Average loss: 0.0308, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.002830
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.005908
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.021160
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.016021
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.003701
Train Epoch: 46 - Loss: 0.0090, Accuracy: 99.83%

Test set: Average loss: 0.0307, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.004318
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.011292
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.004980
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.001476
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.006619
Train Epoch: 47 - Loss: 0.0089, Accuracy: 99.83%

Test set: Average loss: 0.0310, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.009674
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.001568
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.008229
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.003959
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.008405
Train Epoch: 48 - Loss: 0.0087, Accuracy: 99.85%

Test set: Average loss: 0.0307, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.009704
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.024503
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.004583
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.002296
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.007844
Train Epoch: 49 - Loss: 0.0087, Accuracy: 99.85%

Test set: Average loss: 0.0306, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.009926
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.007712
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.001661
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.006762
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.002610
Train Epoch: 50 - Loss: 0.0086, Accuracy: 99.85%

Test set: Average loss: 0.0307, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.007001
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.012622
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.002946
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.002008
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.018768
Train Epoch: 51 - Loss: 0.0121, Accuracy: 99.69%

Test set: Average loss: 0.0325, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.022230
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.010151
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.005880
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.001918
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.014498
Train Epoch: 52 - Loss: 0.0118, Accuracy: 99.70%

Test set: Average loss: 0.0343, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.002835
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.015011
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.026285
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.015714
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.005874
Train Epoch: 53 - Loss: 0.0111, Accuracy: 99.71%

Test set: Average loss: 0.0320, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.003639
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.014402
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.007405
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.013163
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.003178
Train Epoch: 54 - Loss: 0.0102, Accuracy: 99.75%

Test set: Average loss: 0.0360, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.004460
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.005726
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.012236
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.001805
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.002875
Train Epoch: 55 - Loss: 0.0092, Accuracy: 99.78%

Test set: Average loss: 0.0328, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.001709
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.004473
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.004959
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.004338
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.009359
Train Epoch: 56 - Loss: 0.0087, Accuracy: 99.81%

Test set: Average loss: 0.0323, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.011838
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.003658
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.006944
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.001896
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.006325
Train Epoch: 57 - Loss: 0.0080, Accuracy: 99.82%

Test set: Average loss: 0.0324, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.002052
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.004087
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.015136
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.024007
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.007686
Train Epoch: 58 - Loss: 0.0077, Accuracy: 99.84%

Test set: Average loss: 0.0314, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.005911
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.003124
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.004657
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.001394
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.017311
Train Epoch: 59 - Loss: 0.0070, Accuracy: 99.86%

Test set: Average loss: 0.0314, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.004701
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.007022
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.009191
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.019438
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.001924
Train Epoch: 60 - Loss: 0.0063, Accuracy: 99.89%

Test set: Average loss: 0.0315, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.004115
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.002646
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.005007
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.002783
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.006154
Train Epoch: 61 - Loss: 0.0061, Accuracy: 99.89%

Test set: Average loss: 0.0317, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.009581
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.005457
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.002965
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.002954
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.019940
Train Epoch: 62 - Loss: 0.0056, Accuracy: 99.90%

Test set: Average loss: 0.0320, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.002304
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.003277
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.004276
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.003129
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.001595
Train Epoch: 63 - Loss: 0.0052, Accuracy: 99.93%

Test set: Average loss: 0.0318, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.003488
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.000891
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.004633
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.009885
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.009142
Train Epoch: 64 - Loss: 0.0051, Accuracy: 99.92%

Test set: Average loss: 0.0319, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.001337
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.005359
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.002354
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.002596
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.003699
Train Epoch: 65 - Loss: 0.0048, Accuracy: 99.93%

Test set: Average loss: 0.0321, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.002262
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.005628
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.001266
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.002807
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.002791
Train Epoch: 66 - Loss: 0.0046, Accuracy: 99.94%

Test set: Average loss: 0.0321, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.005521
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.002347
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.009405
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.004118
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.009804
Train Epoch: 67 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0313, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.003521
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.004509
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.003447
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.005334
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.001206
Train Epoch: 68 - Loss: 0.0043, Accuracy: 99.94%

Test set: Average loss: 0.0319, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.004571
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.003226
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.003221
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.001604
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.001847
Train Epoch: 69 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0315, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.002765
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.003208
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.002043
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.003586
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.001500
Train Epoch: 70 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0315, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.000406
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.002950
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.006066
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.004800
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.001494
Train Epoch: 71 - Loss: 0.0040, Accuracy: 99.95%

Test set: Average loss: 0.0314, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.002119
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.002028
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.001942
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.005031
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.002541
Train Epoch: 72 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.001202
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.002363
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.004287
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.001498
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001650
Train Epoch: 73 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.000987
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002290
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.004547
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.001674
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.004058
Train Epoch: 74 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.001580
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.002657
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.010835
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.002384
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.007165
Train Epoch: 75 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0313, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.002435
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.001725
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.002707
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.003487
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.003116
Train Epoch: 76 - Loss: 0.0054, Accuracy: 99.90%

Test set: Average loss: 0.0326, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.001842
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.001217
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.006046
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.003576
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.004759
Train Epoch: 77 - Loss: 0.0054, Accuracy: 99.91%

Test set: Average loss: 0.0323, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.015258
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.001691
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.003083
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.004235
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.001235
Train Epoch: 78 - Loss: 0.0052, Accuracy: 99.89%

Test set: Average loss: 0.0341, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.004609
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.003076
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.005500
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.003437
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.002400
Train Epoch: 79 - Loss: 0.0048, Accuracy: 99.93%

Test set: Average loss: 0.0351, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.003158
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.000951
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.006247
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.000991
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.005303
Train Epoch: 80 - Loss: 0.0043, Accuracy: 99.94%

Test set: Average loss: 0.0324, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.003569
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.001424
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.002523
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.003763
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.001177
Train Epoch: 81 - Loss: 0.0039, Accuracy: 99.95%

Test set: Average loss: 0.0341, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.000241
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002251
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.002011
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.004529
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.004251
Train Epoch: 82 - Loss: 0.0038, Accuracy: 99.95%

Test set: Average loss: 0.0329, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.002181
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.002996
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.002634
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.006735
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.000681
Train Epoch: 83 - Loss: 0.0035, Accuracy: 99.96%

Test set: Average loss: 0.0328, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.000703
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.005217
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.002577
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.000996
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.002036
Train Epoch: 84 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0322, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.001481
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.002498
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.001247
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.013831
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.003309
Train Epoch: 85 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0324, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.002589
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.002693
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.001115
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.001668
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.003144
Train Epoch: 86 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0324, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.009112
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.001635
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.002848
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001618
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.000866
Train Epoch: 87 - Loss: 0.0027, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.004247
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.003122
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.001899
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.000640
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001707
Train Epoch: 88 - Loss: 0.0026, Accuracy: 99.97%

Test set: Average loss: 0.0333, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.002796
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.001694
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.002261
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.002266
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.004557
Train Epoch: 89 - Loss: 0.0025, Accuracy: 99.99%

Test set: Average loss: 0.0321, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.002199
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.001869
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.000809
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.002372
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.004894
Train Epoch: 90 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0323, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.001677
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.001171
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.000798
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.000703
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.005610
Train Epoch: 91 - Loss: 0.0023, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.003430
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.000139
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.003282
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.002031
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.000432
Train Epoch: 92 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0326, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.002787
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.002875
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.001427
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001189
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.001507
Train Epoch: 93 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0327, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.006197
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.000590
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.002661
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.003107
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.001750
Train Epoch: 94 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0325, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.001853
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.004453
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.011529
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001358
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.000961
Train Epoch: 95 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001834
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.001225
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.003071
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001610
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.003266
Train Epoch: 96 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.002934
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.001843
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.001462
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.002904
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.000675
Train Epoch: 97 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.002353
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.001941
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.003695
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.001090
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.002527
Train Epoch: 98 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.001037
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.001267
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.001491
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.001149
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.002090
Train Epoch: 99 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.002638
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.003017
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.002321
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.003568
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.001101
Train Epoch: 100 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0325, Accuracy: 9901/10000 (99.01%)


Training completed. Best Accuracy: 99.05%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.301481
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.027073
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.484959
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.318413
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.251365
Train Epoch: 1 - Loss: 0.7581, Accuracy: 76.78%

Test set: Average loss: 0.2611, Accuracy: 9255/10000 (92.55%)

New best model saved with accuracy: 92.55%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.180919
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.206423
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.367605
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.130845
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.056846
Train Epoch: 2 - Loss: 0.2070, Accuracy: 93.88%

Test set: Average loss: 0.1467, Accuracy: 9554/10000 (95.54%)

New best model saved with accuracy: 95.54%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.113095
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.163490
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.098084
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.164250
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.115401
Train Epoch: 3 - Loss: 0.1276, Accuracy: 96.22%

Test set: Average loss: 0.0958, Accuracy: 9704/10000 (97.04%)

New best model saved with accuracy: 97.04%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.134582
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.132061
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.087931
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.114213
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.051136
Train Epoch: 4 - Loss: 0.0932, Accuracy: 97.25%

Test set: Average loss: 0.0848, Accuracy: 9742/10000 (97.42%)

New best model saved with accuracy: 97.42%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.050051
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.049282
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.090428
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.051653
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.047007
Train Epoch: 5 - Loss: 0.0750, Accuracy: 97.75%

Test set: Average loss: 0.0692, Accuracy: 9776/10000 (97.76%)

New best model saved with accuracy: 97.76%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.027125
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.044004
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.034992
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.036753
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.013994
Train Epoch: 6 - Loss: 0.0636, Accuracy: 98.11%

Test set: Average loss: 0.0574, Accuracy: 9818/10000 (98.18%)

New best model saved with accuracy: 98.18%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.058576
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.038114
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.024931
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.053716
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.040402
Train Epoch: 7 - Loss: 0.0548, Accuracy: 98.34%

Test set: Average loss: 0.0504, Accuracy: 9837/10000 (98.37%)

New best model saved with accuracy: 98.37%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.067725
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.115257
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.026977
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.034782
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.041436
Train Epoch: 8 - Loss: 0.0492, Accuracy: 98.55%

Test set: Average loss: 0.0472, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.086127
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.032903
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.057680
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.020077
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.038494
Train Epoch: 9 - Loss: 0.0444, Accuracy: 98.70%

Test set: Average loss: 0.0457, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.033064
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.085441
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.010046
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.036065
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.013262
Train Epoch: 10 - Loss: 0.0402, Accuracy: 98.87%

Test set: Average loss: 0.0431, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.011378
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.102332
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.039545
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.013005
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.026134
Train Epoch: 11 - Loss: 0.0371, Accuracy: 98.90%

Test set: Average loss: 0.0396, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.020946
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.012662
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.018355
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.028952
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.011733
Train Epoch: 12 - Loss: 0.0348, Accuracy: 99.00%

Test set: Average loss: 0.0387, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.029980
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.005967
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.023224
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.011839
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.027865
Train Epoch: 13 - Loss: 0.0321, Accuracy: 99.09%

Test set: Average loss: 0.0383, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.020453
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.006606
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.012566
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.042709
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.071836
Train Epoch: 14 - Loss: 0.0304, Accuracy: 99.16%

Test set: Average loss: 0.0386, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.022414
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.024385
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.069484
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.049759
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.010487
Train Epoch: 15 - Loss: 0.0287, Accuracy: 99.20%

Test set: Average loss: 0.0371, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.020250
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.064963
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.019354
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.030070
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.024207
Train Epoch: 16 - Loss: 0.0274, Accuracy: 99.25%

Test set: Average loss: 0.0365, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.028927
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.047441
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.032419
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.006997
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.017128
Train Epoch: 17 - Loss: 0.0262, Accuracy: 99.29%

Test set: Average loss: 0.0361, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.059151
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.007934
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.010137
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.012152
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.051483
Train Epoch: 18 - Loss: 0.0254, Accuracy: 99.33%

Test set: Average loss: 0.0355, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.011964
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.075430
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.009055
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.015525
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.012106
Train Epoch: 19 - Loss: 0.0245, Accuracy: 99.35%

Test set: Average loss: 0.0351, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.029969
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.038740
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.019075
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.024536
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.044218
Train Epoch: 20 - Loss: 0.0239, Accuracy: 99.39%

Test set: Average loss: 0.0353, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.010758
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.014281
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.059174
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.031064
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.033039
Train Epoch: 21 - Loss: 0.0234, Accuracy: 99.41%

Test set: Average loss: 0.0347, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.024992
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.011088
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.003538
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.007007
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.017291
Train Epoch: 22 - Loss: 0.0229, Accuracy: 99.41%

Test set: Average loss: 0.0346, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.007441
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.006293
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.034685
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.023446
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.022164
Train Epoch: 23 - Loss: 0.0227, Accuracy: 99.42%

Test set: Average loss: 0.0346, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.021869
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.021232
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.026504
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.036835
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.013177
Train Epoch: 24 - Loss: 0.0225, Accuracy: 99.43%

Test set: Average loss: 0.0346, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.020527
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.014984
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.016619
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.011960
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.013981
Train Epoch: 25 - Loss: 0.0224, Accuracy: 99.44%

Test set: Average loss: 0.0346, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.004449
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.020983
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.075747
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.029021
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.012085
Train Epoch: 26 - Loss: 0.0295, Accuracy: 99.14%

Test set: Average loss: 0.0523, Accuracy: 9840/10000 (98.40%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.024277
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.010859
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.071692
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.010548
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.044774
Train Epoch: 27 - Loss: 0.0265, Accuracy: 99.22%

Test set: Average loss: 0.0413, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.021380
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.021865
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.039047
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.015265
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.010600
Train Epoch: 28 - Loss: 0.0247, Accuracy: 99.27%

Test set: Average loss: 0.0368, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.019109
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.003484
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.069214
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.053015
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.012599
Train Epoch: 29 - Loss: 0.0230, Accuracy: 99.34%

Test set: Average loss: 0.0341, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.021938
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.017755
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.019351
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.011224
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.068868
Train Epoch: 30 - Loss: 0.0211, Accuracy: 99.39%

Test set: Average loss: 0.0331, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.005220
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.016074
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.003954
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.014982
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.022992
Train Epoch: 31 - Loss: 0.0193, Accuracy: 99.42%

Test set: Average loss: 0.0347, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.012391
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.018131
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.006173
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.002209
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.036589
Train Epoch: 32 - Loss: 0.0186, Accuracy: 99.47%

Test set: Average loss: 0.0349, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.005699
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.026564
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.042653
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.008936
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.004672
Train Epoch: 33 - Loss: 0.0167, Accuracy: 99.55%

Test set: Average loss: 0.0371, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.019664
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.007209
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.010834
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.016962
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.014168
Train Epoch: 34 - Loss: 0.0157, Accuracy: 99.58%

Test set: Average loss: 0.0372, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.006298
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.019915
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.020429
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.056306
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.009373
Train Epoch: 35 - Loss: 0.0146, Accuracy: 99.62%

Test set: Average loss: 0.0341, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.003408
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.032415
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.010390
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.007614
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.002111
Train Epoch: 36 - Loss: 0.0135, Accuracy: 99.68%

Test set: Average loss: 0.0345, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.003663
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.024328
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.004657
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.008302
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.049125
Train Epoch: 37 - Loss: 0.0129, Accuracy: 99.67%

Test set: Average loss: 0.0321, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.003588
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.016967
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.023363
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.019767
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.013489
Train Epoch: 38 - Loss: 0.0119, Accuracy: 99.72%

Test set: Average loss: 0.0328, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.010007
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.011075
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.005648
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.004791
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.006135
Train Epoch: 39 - Loss: 0.0115, Accuracy: 99.71%

Test set: Average loss: 0.0324, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.005669
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.001903
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.004179
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.004540
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.018543
Train Epoch: 40 - Loss: 0.0108, Accuracy: 99.73%

Test set: Average loss: 0.0330, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.004992
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.002660
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.020509
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.005046
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.030990
Train Epoch: 41 - Loss: 0.0103, Accuracy: 99.77%

Test set: Average loss: 0.0339, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.001246
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.006532
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.002310
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.008110
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.009231
Train Epoch: 42 - Loss: 0.0099, Accuracy: 99.80%

Test set: Average loss: 0.0323, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.003159
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.018814
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.003250
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.007789
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.005042
Train Epoch: 43 - Loss: 0.0095, Accuracy: 99.80%

Test set: Average loss: 0.0322, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.003127
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.005631
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.008040
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.011861
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.010447
Train Epoch: 44 - Loss: 0.0092, Accuracy: 99.81%

Test set: Average loss: 0.0323, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.038848
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.009137
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.034796
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.011150
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.002723
Train Epoch: 45 - Loss: 0.0091, Accuracy: 99.81%

Test set: Average loss: 0.0325, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.007241
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.016907
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.005418
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.020194
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.075089
Train Epoch: 46 - Loss: 0.0089, Accuracy: 99.82%

Test set: Average loss: 0.0324, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.013088
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.002402
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.003397
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.013948
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.003603
Train Epoch: 47 - Loss: 0.0087, Accuracy: 99.82%

Test set: Average loss: 0.0322, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.015665
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.003576
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.003556
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.011916
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.012023
Train Epoch: 48 - Loss: 0.0086, Accuracy: 99.83%

Test set: Average loss: 0.0323, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.008968
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.012059
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.007004
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.018848
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.003422
Train Epoch: 49 - Loss: 0.0086, Accuracy: 99.84%

Test set: Average loss: 0.0323, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.004689
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.002134
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.011884
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.023882
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.015275
Train Epoch: 50 - Loss: 0.0085, Accuracy: 99.83%

Test set: Average loss: 0.0323, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.002277
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.011948
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.004210
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.013971
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.005805
Train Epoch: 51 - Loss: 0.0123, Accuracy: 99.67%

Test set: Average loss: 0.0347, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.013161
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.015742
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.011937
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.002128
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.012202
Train Epoch: 52 - Loss: 0.0112, Accuracy: 99.74%

Test set: Average loss: 0.0487, Accuracy: 9850/10000 (98.50%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.004924
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.002811
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.070848
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.016875
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.001702
Train Epoch: 53 - Loss: 0.0106, Accuracy: 99.75%

Test set: Average loss: 0.0350, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.002498
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.011579
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.004825
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.004663
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.015997
Train Epoch: 54 - Loss: 0.0102, Accuracy: 99.76%

Test set: Average loss: 0.0383, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.007610
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.004942
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.008145
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.008952
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.012708
Train Epoch: 55 - Loss: 0.0091, Accuracy: 99.80%

Test set: Average loss: 0.0378, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.006468
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.003390
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.009913
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.006006
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.000952
Train Epoch: 56 - Loss: 0.0086, Accuracy: 99.81%

Test set: Average loss: 0.0343, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.035840
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.002849
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.003976
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.041525
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.011151
Train Epoch: 57 - Loss: 0.0079, Accuracy: 99.84%

Test set: Average loss: 0.0343, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.003189
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.002939
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.008147
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.002000
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.004524
Train Epoch: 58 - Loss: 0.0075, Accuracy: 99.85%

Test set: Average loss: 0.0342, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.001695
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.016895
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.007412
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.015074
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.002734
Train Epoch: 59 - Loss: 0.0068, Accuracy: 99.88%

Test set: Average loss: 0.0335, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.002114
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.001188
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.008042
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.008577
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.001297
Train Epoch: 60 - Loss: 0.0064, Accuracy: 99.89%

Test set: Average loss: 0.0331, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.001456
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.010811
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.001605
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.002253
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.010707
Train Epoch: 61 - Loss: 0.0060, Accuracy: 99.89%

Test set: Average loss: 0.0335, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.004115
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.008098
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.003308
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.003464
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.004168
Train Epoch: 62 - Loss: 0.0057, Accuracy: 99.91%

Test set: Average loss: 0.0361, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.003013
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.000764
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.005075
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.004171
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.007659
Train Epoch: 63 - Loss: 0.0054, Accuracy: 99.93%

Test set: Average loss: 0.0341, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.001662
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.006196
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.003226
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.004244
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.007974
Train Epoch: 64 - Loss: 0.0051, Accuracy: 99.94%

Test set: Average loss: 0.0347, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.009124
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.001959
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.005567
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.007761
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.004748
Train Epoch: 65 - Loss: 0.0049, Accuracy: 99.94%

Test set: Average loss: 0.0343, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.003461
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.003239
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.007458
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.002833
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.000820
Train Epoch: 66 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0341, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.004664
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.007851
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.008951
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.000829
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.002568
Train Epoch: 67 - Loss: 0.0046, Accuracy: 99.94%

Test set: Average loss: 0.0349, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.001119
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.006428
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.008095
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.002259
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.003374
Train Epoch: 68 - Loss: 0.0045, Accuracy: 99.94%

Test set: Average loss: 0.0342, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.002577
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.001033
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.006187
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.011495
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.001714
Train Epoch: 69 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0343, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.001660
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.001555
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.001954
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.001189
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.003123
Train Epoch: 70 - Loss: 0.0043, Accuracy: 99.96%

Test set: Average loss: 0.0341, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.003177
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.040017
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.008028
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.003557
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.003363
Train Epoch: 71 - Loss: 0.0042, Accuracy: 99.96%

Test set: Average loss: 0.0340, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.002488
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.002906
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.001980
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.006721
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.001789
Train Epoch: 72 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0341, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.001678
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.002928
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.002412
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.001717
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001739
Train Epoch: 73 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0341, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.004489
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.003162
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.001661
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.002350
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.001621
Train Epoch: 74 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0341, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.002737
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.000581
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.000883
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.004321
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.002687
Train Epoch: 75 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0341, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.003314
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.020906
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.010445
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.010850
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.001424
Train Epoch: 76 - Loss: 0.0055, Accuracy: 99.90%

Test set: Average loss: 0.0343, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.016035
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.016572
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.003961
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.002397
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.007470
Train Epoch: 77 - Loss: 0.0053, Accuracy: 99.91%

Test set: Average loss: 0.0381, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.000663
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.001582
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.002228
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.003792
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.002353
Train Epoch: 78 - Loss: 0.0051, Accuracy: 99.92%

Test set: Average loss: 0.0353, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.004440
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.001867
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.005396
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.002271
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.001646
Train Epoch: 79 - Loss: 0.0049, Accuracy: 99.92%

Test set: Average loss: 0.0398, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.000729
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.000825
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.004323
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.008876
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.001270
Train Epoch: 80 - Loss: 0.0046, Accuracy: 99.92%

Test set: Average loss: 0.0356, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.003134
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.002555
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.002307
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.000845
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.003022
Train Epoch: 81 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0371, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.003744
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.000754
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.002047
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.003525
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.001891
Train Epoch: 82 - Loss: 0.0040, Accuracy: 99.94%

Test set: Average loss: 0.0375, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.005277
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.002870
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.003852
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.002283
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.003149
Train Epoch: 83 - Loss: 0.0036, Accuracy: 99.96%

Test set: Average loss: 0.0365, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.000885
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.003796
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.001372
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.001555
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.010293
Train Epoch: 84 - Loss: 0.0035, Accuracy: 99.97%

Test set: Average loss: 0.0366, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.002226
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.002644
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.004982
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.003961
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.003322
Train Epoch: 85 - Loss: 0.0034, Accuracy: 99.96%

Test set: Average loss: 0.0360, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.000729
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.001365
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.001161
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.002131
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.000850
Train Epoch: 86 - Loss: 0.0032, Accuracy: 99.97%

Test set: Average loss: 0.0361, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.002805
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.001080
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001014
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001422
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.002517
Train Epoch: 87 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0357, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.003572
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.002740
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.001509
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.001563
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001170
Train Epoch: 88 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0360, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.003186
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.000918
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.001669
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.001471
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.008472
Train Epoch: 89 - Loss: 0.0028, Accuracy: 99.97%

Test set: Average loss: 0.0358, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.001057
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.001957
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.001301
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001353
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.001595
Train Epoch: 90 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0362, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.004104
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.003435
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.003368
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.000638
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.000510
Train Epoch: 91 - Loss: 0.0026, Accuracy: 99.98%

Test set: Average loss: 0.0361, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.003368
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.004815
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001148
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.004125
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.003732
Train Epoch: 92 - Loss: 0.0025, Accuracy: 99.99%

Test set: Average loss: 0.0383, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.001122
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.001440
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.000696
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.000959
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.002797
Train Epoch: 93 - Loss: 0.0025, Accuracy: 99.98%

Test set: Average loss: 0.0359, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.000881
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.001768
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.001242
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.001572
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.000981
Train Epoch: 94 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0366, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.001853
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.004009
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.002438
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001403
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.002151
Train Epoch: 95 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0365, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001425
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.002108
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.000873
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001060
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.001445
Train Epoch: 96 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0364, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.001029
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.001578
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.001196
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.001403
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.002984
Train Epoch: 97 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0363, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.001812
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.000883
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.001310
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.005796
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.000660
Train Epoch: 98 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0363, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.001703
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.001184
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.003296
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.009212
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.002301
Train Epoch: 99 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0363, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.001356
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.004429
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.001996
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001225
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.003288
Train Epoch: 100 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0363, Accuracy: 9891/10000 (98.91%)


Training completed. Best Accuracy: 98.94%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.301985
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.963139
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.375680
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.337999
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.347135
Train Epoch: 1 - Loss: 0.7026, Accuracy: 79.87%

Test set: Average loss: 0.2507, Accuracy: 9281/10000 (92.81%)

New best model saved with accuracy: 92.81%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.297696
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.199530
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.141432
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.175615
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.185801
Train Epoch: 2 - Loss: 0.2073, Accuracy: 93.79%

Test set: Average loss: 0.1566, Accuracy: 9532/10000 (95.32%)

New best model saved with accuracy: 95.32%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.125911
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.161138
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.146642
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.172120
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.154624
Train Epoch: 3 - Loss: 0.1374, Accuracy: 95.94%

Test set: Average loss: 0.1047, Accuracy: 9694/10000 (96.94%)

New best model saved with accuracy: 96.94%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.087211
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.045646
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.129530
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.091795
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.051455
Train Epoch: 4 - Loss: 0.1027, Accuracy: 96.97%

Test set: Average loss: 0.0860, Accuracy: 9744/10000 (97.44%)

New best model saved with accuracy: 97.44%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.073163
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.070397
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.129468
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.079542
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.104712
Train Epoch: 5 - Loss: 0.0827, Accuracy: 97.51%

Test set: Average loss: 0.0704, Accuracy: 9787/10000 (97.87%)

New best model saved with accuracy: 97.87%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.082490
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.079946
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.096821
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.049954
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.089957
Train Epoch: 6 - Loss: 0.0694, Accuracy: 97.90%

Test set: Average loss: 0.0608, Accuracy: 9806/10000 (98.06%)

New best model saved with accuracy: 98.06%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.017818
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.054591
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.044952
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.008324
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.085735
Train Epoch: 7 - Loss: 0.0604, Accuracy: 98.19%

Test set: Average loss: 0.0712, Accuracy: 9767/10000 (97.67%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.036697
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.024545
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.088118
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.044035
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.063794
Train Epoch: 8 - Loss: 0.0537, Accuracy: 98.39%

Test set: Average loss: 0.0497, Accuracy: 9840/10000 (98.40%)

New best model saved with accuracy: 98.40%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.032620
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.023699
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.042459
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.093712
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.015477
Train Epoch: 9 - Loss: 0.0482, Accuracy: 98.54%

Test set: Average loss: 0.0470, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.023250
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.097280
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.031585
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.024209
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.040462
Train Epoch: 10 - Loss: 0.0436, Accuracy: 98.70%

Test set: Average loss: 0.0459, Accuracy: 9847/10000 (98.47%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.061041
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.077025
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.014069
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.043220
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.025960
Train Epoch: 11 - Loss: 0.0398, Accuracy: 98.83%

Test set: Average loss: 0.0438, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.044714
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.069060
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.040600
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.033040
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.020661
Train Epoch: 12 - Loss: 0.0368, Accuracy: 98.90%

Test set: Average loss: 0.0407, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.099744
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.008436
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.013554
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.018759
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.027494
Train Epoch: 13 - Loss: 0.0346, Accuracy: 98.96%

Test set: Average loss: 0.0388, Accuracy: 9869/10000 (98.69%)

New best model saved with accuracy: 98.69%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.021579
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.043528
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.008209
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.017437
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.045293
Train Epoch: 14 - Loss: 0.0323, Accuracy: 99.08%

Test set: Average loss: 0.0379, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.023177
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.033254
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.047529
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.029851
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.016438
Train Epoch: 15 - Loss: 0.0305, Accuracy: 99.13%

Test set: Average loss: 0.0390, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.023273
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.007959
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.023927
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.030112
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.032812
Train Epoch: 16 - Loss: 0.0289, Accuracy: 99.18%

Test set: Average loss: 0.0373, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.013579
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.011713
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.045703
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.021414
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.025101
Train Epoch: 17 - Loss: 0.0277, Accuracy: 99.22%

Test set: Average loss: 0.0367, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.043242
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.039058
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.043517
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.019200
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.033599
Train Epoch: 18 - Loss: 0.0266, Accuracy: 99.27%

Test set: Average loss: 0.0362, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.018352
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.016378
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.017151
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.063856
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.005795
Train Epoch: 19 - Loss: 0.0258, Accuracy: 99.28%

Test set: Average loss: 0.0356, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.039046
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.021681
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.017814
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.013030
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.027636
Train Epoch: 20 - Loss: 0.0252, Accuracy: 99.33%

Test set: Average loss: 0.0359, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.008319
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.025650
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.059636
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.006502
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.004908
Train Epoch: 21 - Loss: 0.0247, Accuracy: 99.34%

Test set: Average loss: 0.0353, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.068411
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.020638
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.013821
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.045881
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.012519
Train Epoch: 22 - Loss: 0.0242, Accuracy: 99.36%

Test set: Average loss: 0.0351, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.032929
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.013322
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.045341
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.022166
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.043180
Train Epoch: 23 - Loss: 0.0239, Accuracy: 99.37%

Test set: Average loss: 0.0352, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.040414
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.008097
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.063896
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.029986
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.022272
Train Epoch: 24 - Loss: 0.0237, Accuracy: 99.38%

Test set: Average loss: 0.0352, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.032325
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.027204
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.002603
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.012346
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.020714
Train Epoch: 25 - Loss: 0.0236, Accuracy: 99.39%

Test set: Average loss: 0.0352, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.029503
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.013142
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.075900
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.019822
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.013424
Train Epoch: 26 - Loss: 0.0305, Accuracy: 99.06%

Test set: Average loss: 0.0378, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.038411
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.062863
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.028728
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.037422
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.042410
Train Epoch: 27 - Loss: 0.0279, Accuracy: 99.18%

Test set: Average loss: 0.0351, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.087406
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.020427
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.029662
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.015670
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.024638
Train Epoch: 28 - Loss: 0.0259, Accuracy: 99.24%

Test set: Average loss: 0.0393, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.043941
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.044581
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.014470
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.011896
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.079264
Train Epoch: 29 - Loss: 0.0242, Accuracy: 99.29%

Test set: Average loss: 0.0366, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.012327
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.020614
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.010217
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.021308
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.018609
Train Epoch: 30 - Loss: 0.0213, Accuracy: 99.36%

Test set: Average loss: 0.0358, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.002643
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.026907
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.031523
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.009984
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.009054
Train Epoch: 31 - Loss: 0.0204, Accuracy: 99.44%

Test set: Average loss: 0.0373, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.005272
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.009913
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.005553
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.025158
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.021256
Train Epoch: 32 - Loss: 0.0186, Accuracy: 99.48%

Test set: Average loss: 0.0371, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.043066
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.045326
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.005564
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.006185
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.009584
Train Epoch: 33 - Loss: 0.0171, Accuracy: 99.54%

Test set: Average loss: 0.0355, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.020333
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.022766
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.002612
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.051315
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.010652
Train Epoch: 34 - Loss: 0.0160, Accuracy: 99.58%

Test set: Average loss: 0.0340, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.012002
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.034404
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.078050
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.003085
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.013353
Train Epoch: 35 - Loss: 0.0149, Accuracy: 99.61%

Test set: Average loss: 0.0334, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.007166
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.022677
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.003979
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.027730
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.007907
Train Epoch: 36 - Loss: 0.0138, Accuracy: 99.66%

Test set: Average loss: 0.0370, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.021269
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.015792
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.007253
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.013770
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.020402
Train Epoch: 37 - Loss: 0.0130, Accuracy: 99.66%

Test set: Average loss: 0.0318, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.002416
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.036252
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.012962
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.008793
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.006475
Train Epoch: 38 - Loss: 0.0122, Accuracy: 99.72%

Test set: Average loss: 0.0317, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.003222
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.020922
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.000575
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.015911
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.014274
Train Epoch: 39 - Loss: 0.0114, Accuracy: 99.75%

Test set: Average loss: 0.0328, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.008622
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.018040
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.004351
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.021155
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.011069
Train Epoch: 40 - Loss: 0.0108, Accuracy: 99.75%

Test set: Average loss: 0.0315, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.004676
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.003198
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.007174
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.008842
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.003260
Train Epoch: 41 - Loss: 0.0103, Accuracy: 99.77%

Test set: Average loss: 0.0319, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.010504
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.003245
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.091362
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.004610
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.022251
Train Epoch: 42 - Loss: 0.0098, Accuracy: 99.81%

Test set: Average loss: 0.0319, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.022177
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.008683
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.012419
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.002716
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.003672
Train Epoch: 43 - Loss: 0.0096, Accuracy: 99.81%

Test set: Average loss: 0.0318, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.003156
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.001359
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.003152
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.002977
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.008390
Train Epoch: 44 - Loss: 0.0092, Accuracy: 99.81%

Test set: Average loss: 0.0316, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.014555
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.015419
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.012265
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.002267
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.004751
Train Epoch: 45 - Loss: 0.0089, Accuracy: 99.82%

Test set: Average loss: 0.0316, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.004226
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.007243
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.004119
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.007326
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.021208
Train Epoch: 46 - Loss: 0.0088, Accuracy: 99.84%

Test set: Average loss: 0.0317, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.009165
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.005024
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.007670
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.002989
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.002265
Train Epoch: 47 - Loss: 0.0086, Accuracy: 99.84%

Test set: Average loss: 0.0316, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.008900
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.003089
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.004955
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.009360
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.003978
Train Epoch: 48 - Loss: 0.0085, Accuracy: 99.84%

Test set: Average loss: 0.0316, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.021566
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.012038
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.002915
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.014332
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.003066
Train Epoch: 49 - Loss: 0.0084, Accuracy: 99.85%

Test set: Average loss: 0.0317, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.012308
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.005758
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.005705
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.003763
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.007478
Train Epoch: 50 - Loss: 0.0084, Accuracy: 99.84%

Test set: Average loss: 0.0317, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.019798
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.034515
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.005759
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.124206
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.020642
Train Epoch: 51 - Loss: 0.0121, Accuracy: 99.71%

Test set: Average loss: 0.0328, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.009218
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.006195
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.006165
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.028557
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.025369
Train Epoch: 52 - Loss: 0.0114, Accuracy: 99.70%

Test set: Average loss: 0.0359, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.004066
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.006170
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.009955
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.006070
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.029033
Train Epoch: 53 - Loss: 0.0109, Accuracy: 99.71%

Test set: Average loss: 0.0344, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.004119
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.011932
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.002464
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.017350
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.003210
Train Epoch: 54 - Loss: 0.0099, Accuracy: 99.77%

Test set: Average loss: 0.0347, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.012080
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.010424
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.008091
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.007422
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.007731
Train Epoch: 55 - Loss: 0.0091, Accuracy: 99.77%

Test set: Average loss: 0.0327, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.002448
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.002211
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.002250
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.011635
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.092266
Train Epoch: 56 - Loss: 0.0081, Accuracy: 99.81%

Test set: Average loss: 0.0361, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.002879
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.006223
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.007238
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.006388
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.003435
Train Epoch: 57 - Loss: 0.0078, Accuracy: 99.84%

Test set: Average loss: 0.0346, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.007415
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.002195
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.006827
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.023089
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.005235
Train Epoch: 58 - Loss: 0.0071, Accuracy: 99.85%

Test set: Average loss: 0.0336, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.005641
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.003171
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.005161
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.002646
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.005620
Train Epoch: 59 - Loss: 0.0067, Accuracy: 99.89%

Test set: Average loss: 0.0343, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.006228
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.000825
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.006303
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.000716
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.012520
Train Epoch: 60 - Loss: 0.0061, Accuracy: 99.89%

Test set: Average loss: 0.0336, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.001227
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.010851
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.000461
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.004402
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.002958
Train Epoch: 61 - Loss: 0.0057, Accuracy: 99.90%

Test set: Average loss: 0.0357, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.004686
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.008027
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.019254
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.002448
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.006918
Train Epoch: 62 - Loss: 0.0055, Accuracy: 99.92%

Test set: Average loss: 0.0327, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.002464
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.007314
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.001687
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.004455
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.002218
Train Epoch: 63 - Loss: 0.0051, Accuracy: 99.93%

Test set: Average loss: 0.0335, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.002746
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.016005
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.002992
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.002875
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.002836
Train Epoch: 64 - Loss: 0.0049, Accuracy: 99.94%

Test set: Average loss: 0.0329, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.000894
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.004048
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.003724
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.007454
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.001153
Train Epoch: 65 - Loss: 0.0046, Accuracy: 99.94%

Test set: Average loss: 0.0332, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.002870
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.001253
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.000710
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.009136
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.001083
Train Epoch: 66 - Loss: 0.0045, Accuracy: 99.94%

Test set: Average loss: 0.0331, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.005772
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.010362
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.003453
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.001992
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.001340
Train Epoch: 67 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0331, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.003818
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.007987
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.006449
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.001659
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.001649
Train Epoch: 68 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.001088
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.001697
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.003358
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.001716
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.001664
Train Epoch: 69 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0329, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.001096
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.004128
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.002076
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.001529
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.003050
Train Epoch: 70 - Loss: 0.0040, Accuracy: 99.97%

Test set: Average loss: 0.0329, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.003457
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.001307
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.006352
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.001912
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.003003
Train Epoch: 71 - Loss: 0.0039, Accuracy: 99.97%

Test set: Average loss: 0.0327, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.003325
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.002073
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.001862
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.001948
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.001205
Train Epoch: 72 - Loss: 0.0038, Accuracy: 99.97%

Test set: Average loss: 0.0329, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.001807
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.004292
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.000687
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.001210
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001293
Train Epoch: 73 - Loss: 0.0038, Accuracy: 99.97%

Test set: Average loss: 0.0329, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.002241
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.006163
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.002210
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.007022
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.002295
Train Epoch: 74 - Loss: 0.0037, Accuracy: 99.97%

Test set: Average loss: 0.0329, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.009753
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.004065
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.000808
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.001327
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.002884
Train Epoch: 75 - Loss: 0.0037, Accuracy: 99.97%

Test set: Average loss: 0.0328, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.002962
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.003491
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.005455
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.000762
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.002648
Train Epoch: 76 - Loss: 0.0054, Accuracy: 99.91%

Test set: Average loss: 0.0351, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.002831
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.004656
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.002324
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.003524
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.008643
Train Epoch: 77 - Loss: 0.0053, Accuracy: 99.89%

Test set: Average loss: 0.0336, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.009649
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.004518
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.005843
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.003040
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.003881
Train Epoch: 78 - Loss: 0.0052, Accuracy: 99.91%

Test set: Average loss: 0.0375, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.004477
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.007093
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.001493
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.000944
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.001866
Train Epoch: 79 - Loss: 0.0045, Accuracy: 99.92%

Test set: Average loss: 0.0326, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.001948
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.003198
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.004666
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.015223
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.003950
Train Epoch: 80 - Loss: 0.0044, Accuracy: 99.93%

Test set: Average loss: 0.0332, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.000556
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.001725
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.001597
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.001662
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.001530
Train Epoch: 81 - Loss: 0.0040, Accuracy: 99.94%

Test set: Average loss: 0.0357, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.001632
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002345
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.001632
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.002931
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.004639
Train Epoch: 82 - Loss: 0.0036, Accuracy: 99.96%

Test set: Average loss: 0.0371, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.004563
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.001236
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.003255
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.002135
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.005026
Train Epoch: 83 - Loss: 0.0036, Accuracy: 99.94%

Test set: Average loss: 0.0343, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.002422
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.003023
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.002136
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.002075
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.001200
Train Epoch: 84 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0342, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.002724
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.009757
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.000743
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.002596
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.005945
Train Epoch: 85 - Loss: 0.0030, Accuracy: 99.97%

Test set: Average loss: 0.0347, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.003193
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.002521
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.003885
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.002730
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.000581
Train Epoch: 86 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0342, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.003436
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.001564
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.004198
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001121
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.002765
Train Epoch: 87 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0358, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.000579
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001695
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.000649
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.017008
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001554
Train Epoch: 88 - Loss: 0.0025, Accuracy: 99.99%

Test set: Average loss: 0.0352, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.004510
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.000341
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.000953
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.000877
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.001159
Train Epoch: 89 - Loss: 0.0025, Accuracy: 99.98%

Test set: Average loss: 0.0341, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.002644
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.000729
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.003158
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.004279
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.002565
Train Epoch: 90 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0347, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.002065
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.003775
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.000782
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.000857
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.003603
Train Epoch: 91 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0345, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.002060
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.006540
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001773
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.001702
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.003220
Train Epoch: 92 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0342, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.000880
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.001513
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.000974
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.004500
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.001082
Train Epoch: 93 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0342, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.001285
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.000438
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.003129
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.000480
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.002653
Train Epoch: 94 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0347, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.001770
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.001856
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.002599
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.000672
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.006612
Train Epoch: 95 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0343, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.002602
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.001942
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.000640
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.000570
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.001450
Train Epoch: 96 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.003077
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.000525
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.003106
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.000397
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.000224
Train Epoch: 97 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0344, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.002090
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.001186
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.002390
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.001278
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.002294
Train Epoch: 98 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.002618
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.000694
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.003321
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.000382
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.001116
Train Epoch: 99 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.001353
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.003772
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.001473
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001035
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.001266
Train Epoch: 100 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9896/10000 (98.96%)


Training completed. Best Accuracy: 98.98%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.314611
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.835374
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.345264
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.407865
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.298979
Train Epoch: 1 - Loss: 0.7209, Accuracy: 79.49%

Test set: Average loss: 0.2596, Accuracy: 9212/10000 (92.12%)

New best model saved with accuracy: 92.12%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.272329
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.254316
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.223744
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.212240
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.090785
Train Epoch: 2 - Loss: 0.2064, Accuracy: 93.90%

Test set: Average loss: 0.1434, Accuracy: 9580/10000 (95.80%)

New best model saved with accuracy: 95.80%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.139069
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.121457
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.148639
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.120201
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.173549
Train Epoch: 3 - Loss: 0.1292, Accuracy: 96.15%

Test set: Average loss: 0.1039, Accuracy: 9690/10000 (96.90%)

New best model saved with accuracy: 96.90%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.165342
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.108047
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.034730
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.121822
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.118015
Train Epoch: 4 - Loss: 0.0941, Accuracy: 97.22%

Test set: Average loss: 0.0770, Accuracy: 9760/10000 (97.60%)

New best model saved with accuracy: 97.60%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.058603
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.077977
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.078309
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.119470
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.033889
Train Epoch: 5 - Loss: 0.0747, Accuracy: 97.76%

Test set: Average loss: 0.0653, Accuracy: 9807/10000 (98.07%)

New best model saved with accuracy: 98.07%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.048169
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.048884
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.058303
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.039400
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.057164
Train Epoch: 6 - Loss: 0.0621, Accuracy: 98.17%

Test set: Average loss: 0.0544, Accuracy: 9832/10000 (98.32%)

New best model saved with accuracy: 98.32%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.017187
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.019480
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.020618
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.019444
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.026097
Train Epoch: 7 - Loss: 0.0543, Accuracy: 98.42%

Test set: Average loss: 0.0487, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.020444
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.026766
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.032829
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.021991
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.069467
Train Epoch: 8 - Loss: 0.0483, Accuracy: 98.54%

Test set: Average loss: 0.0466, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.029118
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.042095
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.031356
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.067951
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.060748
Train Epoch: 9 - Loss: 0.0432, Accuracy: 98.73%

Test set: Average loss: 0.0445, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.062130
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.039736
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.022640
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.067325
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.032460
Train Epoch: 10 - Loss: 0.0392, Accuracy: 98.85%

Test set: Average loss: 0.0417, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.028327
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.030903
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.066393
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.018172
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.009913
Train Epoch: 11 - Loss: 0.0362, Accuracy: 98.94%

Test set: Average loss: 0.0391, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.024769
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.046789
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.056083
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.037373
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.082024
Train Epoch: 12 - Loss: 0.0337, Accuracy: 99.01%

Test set: Average loss: 0.0409, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.016694
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.027057
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.027406
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.010498
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.040654
Train Epoch: 13 - Loss: 0.0318, Accuracy: 99.06%

Test set: Average loss: 0.0365, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.050097
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.021357
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.017143
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.027748
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.025507
Train Epoch: 14 - Loss: 0.0298, Accuracy: 99.14%

Test set: Average loss: 0.0353, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.044462
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.011540
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.056271
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.026830
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.011780
Train Epoch: 15 - Loss: 0.0282, Accuracy: 99.18%

Test set: Average loss: 0.0362, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.005787
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.014402
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.025783
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.018887
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.012654
Train Epoch: 16 - Loss: 0.0267, Accuracy: 99.23%

Test set: Average loss: 0.0346, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.014837
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.006248
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.077626
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.008894
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.030787
Train Epoch: 17 - Loss: 0.0257, Accuracy: 99.25%

Test set: Average loss: 0.0336, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.008279
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.065717
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.015633
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.008641
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.020426
Train Epoch: 18 - Loss: 0.0247, Accuracy: 99.29%

Test set: Average loss: 0.0344, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.041426
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.014214
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.021581
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.004276
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.027357
Train Epoch: 19 - Loss: 0.0238, Accuracy: 99.36%

Test set: Average loss: 0.0335, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.018374
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.021208
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.038082
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.018378
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.010101
Train Epoch: 20 - Loss: 0.0233, Accuracy: 99.36%

Test set: Average loss: 0.0335, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.010852
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.005963
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.006521
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.041753
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.054382
Train Epoch: 21 - Loss: 0.0228, Accuracy: 99.36%

Test set: Average loss: 0.0329, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.039862
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.010737
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.043369
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.014393
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.009099
Train Epoch: 22 - Loss: 0.0225, Accuracy: 99.37%

Test set: Average loss: 0.0327, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.011796
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.012402
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.044845
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.017436
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.045660
Train Epoch: 23 - Loss: 0.0222, Accuracy: 99.38%

Test set: Average loss: 0.0327, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.026353
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.018843
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.004643
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.015378
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.025780
Train Epoch: 24 - Loss: 0.0220, Accuracy: 99.39%

Test set: Average loss: 0.0327, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.008515
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.045432
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.076632
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.009769
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.009599
Train Epoch: 25 - Loss: 0.0219, Accuracy: 99.40%

Test set: Average loss: 0.0327, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.037080
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.019467
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.053287
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.027517
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.022582
Train Epoch: 26 - Loss: 0.0280, Accuracy: 99.14%

Test set: Average loss: 0.0352, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.011251
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.049013
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.004668
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.022311
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.026766
Train Epoch: 27 - Loss: 0.0260, Accuracy: 99.23%

Test set: Average loss: 0.0397, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.029414
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.031854
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.008806
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.038688
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.050452
Train Epoch: 28 - Loss: 0.0237, Accuracy: 99.27%

Test set: Average loss: 0.0314, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.007818
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.028157
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.013488
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.055919
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.011905
Train Epoch: 29 - Loss: 0.0222, Accuracy: 99.36%

Test set: Average loss: 0.0324, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.014920
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.019379
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.007148
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.019078
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.007093
Train Epoch: 30 - Loss: 0.0205, Accuracy: 99.40%

Test set: Average loss: 0.0325, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.005182
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.018509
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.021660
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.007022
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.020812
Train Epoch: 31 - Loss: 0.0186, Accuracy: 99.47%

Test set: Average loss: 0.0355, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.008891
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.015233
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.023551
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.009265
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.008779
Train Epoch: 32 - Loss: 0.0174, Accuracy: 99.50%

Test set: Average loss: 0.0345, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.010425
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.004979
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.003798
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.005803
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.025006
Train Epoch: 33 - Loss: 0.0158, Accuracy: 99.56%

Test set: Average loss: 0.0372, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.021884
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.024717
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.004916
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.014737
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.010581
Train Epoch: 34 - Loss: 0.0146, Accuracy: 99.61%

Test set: Average loss: 0.0308, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.027287
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.007448
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.003537
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.014448
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.003313
Train Epoch: 35 - Loss: 0.0138, Accuracy: 99.63%

Test set: Average loss: 0.0316, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.012063
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.004487
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.008434
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.012081
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.029416
Train Epoch: 36 - Loss: 0.0130, Accuracy: 99.67%

Test set: Average loss: 0.0328, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.020804
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.070812
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.008890
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.012626
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.007610
Train Epoch: 37 - Loss: 0.0120, Accuracy: 99.71%

Test set: Average loss: 0.0314, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.001998
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.004089
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.004272
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.010041
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.038894
Train Epoch: 38 - Loss: 0.0113, Accuracy: 99.73%

Test set: Average loss: 0.0316, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.002653
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.006277
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.004835
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.003775
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.022379
Train Epoch: 39 - Loss: 0.0107, Accuracy: 99.76%

Test set: Average loss: 0.0308, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.015827
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.007297
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.004073
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.009220
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.017421
Train Epoch: 40 - Loss: 0.0102, Accuracy: 99.77%

Test set: Average loss: 0.0317, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.006154
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.006076
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.003478
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.030249
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.009529
Train Epoch: 41 - Loss: 0.0096, Accuracy: 99.81%

Test set: Average loss: 0.0306, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.003187
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.008466
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.009183
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.026081
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.005848
Train Epoch: 42 - Loss: 0.0094, Accuracy: 99.80%

Test set: Average loss: 0.0304, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.014812
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.009393
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.010611
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.002913
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.000746
Train Epoch: 43 - Loss: 0.0090, Accuracy: 99.82%

Test set: Average loss: 0.0302, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.011493
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.002330
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.001488
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.007458
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.015214
Train Epoch: 44 - Loss: 0.0087, Accuracy: 99.84%

Test set: Average loss: 0.0306, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.003870
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.019012
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.008161
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.006043
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.008425
Train Epoch: 45 - Loss: 0.0085, Accuracy: 99.85%

Test set: Average loss: 0.0302, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.007304
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.004420
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.016115
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.008136
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.003383
Train Epoch: 46 - Loss: 0.0083, Accuracy: 99.85%

Test set: Average loss: 0.0303, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.006456
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.001558
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.029282
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.023601
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.003635
Train Epoch: 47 - Loss: 0.0082, Accuracy: 99.86%

Test set: Average loss: 0.0302, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.012217
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.021048
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.007895
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.003146
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.004142
Train Epoch: 48 - Loss: 0.0081, Accuracy: 99.86%

Test set: Average loss: 0.0303, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.002885
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.003224
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.004103
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.003652
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.008790
Train Epoch: 49 - Loss: 0.0080, Accuracy: 99.86%

Test set: Average loss: 0.0302, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.007038
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.004301
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.015168
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.028574
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.007483
Train Epoch: 50 - Loss: 0.0080, Accuracy: 99.86%

Test set: Average loss: 0.0302, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.019502
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.015322
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.002158
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.004672
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.012641
Train Epoch: 51 - Loss: 0.0109, Accuracy: 99.73%

Test set: Average loss: 0.0321, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.005548
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.011985
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.004226
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.013512
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.011417
Train Epoch: 52 - Loss: 0.0107, Accuracy: 99.73%

Test set: Average loss: 0.0357, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.005651
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.008402
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.007138
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.008132
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.002843
Train Epoch: 53 - Loss: 0.0099, Accuracy: 99.75%

Test set: Average loss: 0.0330, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.005403
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.001425
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.004093
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.005450
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.009204
Train Epoch: 54 - Loss: 0.0093, Accuracy: 99.77%

Test set: Average loss: 0.0318, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.010896
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.006748
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.010589
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.002829
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.002802
Train Epoch: 55 - Loss: 0.0087, Accuracy: 99.81%

Test set: Average loss: 0.0354, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.018138
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.001400
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.007724
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.014879
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.005821
Train Epoch: 56 - Loss: 0.0079, Accuracy: 99.83%

Test set: Average loss: 0.0326, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.001768
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.003512
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.021974
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.001840
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.003924
Train Epoch: 57 - Loss: 0.0074, Accuracy: 99.82%

Test set: Average loss: 0.0332, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.002177
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.007776
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.006046
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.012299
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.002599
Train Epoch: 58 - Loss: 0.0067, Accuracy: 99.88%

Test set: Average loss: 0.0327, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.003024
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.001171
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.000899
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.004483
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.005091
Train Epoch: 59 - Loss: 0.0064, Accuracy: 99.88%

Test set: Average loss: 0.0318, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.000899
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.002539
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.000834
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.003065
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.002049
Train Epoch: 60 - Loss: 0.0059, Accuracy: 99.90%

Test set: Average loss: 0.0315, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.003047
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.010479
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.013482
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.004421
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.001351
Train Epoch: 61 - Loss: 0.0055, Accuracy: 99.91%

Test set: Average loss: 0.0324, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.006022
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.004432
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.002507
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.005651
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.005548
Train Epoch: 62 - Loss: 0.0052, Accuracy: 99.92%

Test set: Average loss: 0.0327, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.001310
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.002798
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.004159
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.012004
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.002788
Train Epoch: 63 - Loss: 0.0049, Accuracy: 99.94%

Test set: Average loss: 0.0327, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.004618
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.006778
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.017943
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.001980
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.001347
Train Epoch: 64 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0323, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.004843
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.001650
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.001466
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.003536
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.008841
Train Epoch: 65 - Loss: 0.0045, Accuracy: 99.95%

Test set: Average loss: 0.0328, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.003219
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.006749
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.001477
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.002146
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.005097
Train Epoch: 66 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0326, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.001686
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.004141
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.000666
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.003780
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.003047
Train Epoch: 67 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0325, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.010474
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.003311
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.003005
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.003609
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.000940
Train Epoch: 68 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0329, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.000729
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.002643
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.025512
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.004380
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.002067
Train Epoch: 69 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0325, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.001262
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.004346
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.006297
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.002001
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.001979
Train Epoch: 70 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0326, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.003200
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.001098
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.001290
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.011719
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.003045
Train Epoch: 71 - Loss: 0.0038, Accuracy: 99.97%

Test set: Average loss: 0.0326, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.063986
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.004736
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.002337
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.003113
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.002510
Train Epoch: 72 - Loss: 0.0037, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.004841
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.003540
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.001842
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.007153
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001541
Train Epoch: 73 - Loss: 0.0037, Accuracy: 99.97%

Test set: Average loss: 0.0328, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.000271
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002311
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.001940
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.001912
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.003882
Train Epoch: 74 - Loss: 0.0036, Accuracy: 99.97%

Test set: Average loss: 0.0328, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.004245
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.007252
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.005713
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.001267
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.002509
Train Epoch: 75 - Loss: 0.0036, Accuracy: 99.97%

Test set: Average loss: 0.0327, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.003386
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.000968
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.002622
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.003783
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.000576
Train Epoch: 76 - Loss: 0.0050, Accuracy: 99.93%

Test set: Average loss: 0.0337, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.003301
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.002829
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.005871
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.003219
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.002741
Train Epoch: 77 - Loss: 0.0048, Accuracy: 99.92%

Test set: Average loss: 0.0402, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.034996
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.012219
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.004915
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.007409
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.001840
Train Epoch: 78 - Loss: 0.0044, Accuracy: 99.94%

Test set: Average loss: 0.0332, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.003726
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.000359
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.003567
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.002629
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.002021
Train Epoch: 79 - Loss: 0.0044, Accuracy: 99.93%

Test set: Average loss: 0.0345, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.001533
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.001577
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.007813
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.003608
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.001783
Train Epoch: 80 - Loss: 0.0039, Accuracy: 99.95%

Test set: Average loss: 0.0357, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.001711
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.000901
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.002102
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.003628
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.001648
Train Epoch: 81 - Loss: 0.0040, Accuracy: 99.93%

Test set: Average loss: 0.0342, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.001393
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.000727
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.001648
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.002654
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.003050
Train Epoch: 82 - Loss: 0.0035, Accuracy: 99.96%

Test set: Average loss: 0.0343, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.001319
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.001588
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.004566
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.001665
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.001880
Train Epoch: 83 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0343, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.003035
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.001710
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.001450
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.001061
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.001756
Train Epoch: 84 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0348, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.002373
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.005755
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.001034
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.001572
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.002976
Train Epoch: 85 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0350, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.001799
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.002171
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.002301
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.001636
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.001426
Train Epoch: 86 - Loss: 0.0027, Accuracy: 99.98%

Test set: Average loss: 0.0351, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.004491
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.002641
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001135
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001929
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.002106
Train Epoch: 87 - Loss: 0.0026, Accuracy: 99.97%

Test set: Average loss: 0.0349, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.002281
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001356
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.002540
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.005234
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001968
Train Epoch: 88 - Loss: 0.0024, Accuracy: 99.99%

Test set: Average loss: 0.0349, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.000874
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.001092
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.001041
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.002941
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.001678
Train Epoch: 89 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0350, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.001306
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.002155
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.004670
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001258
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.001242
Train Epoch: 90 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0350, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.001867
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.002005
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.003989
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.002398
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.002012
Train Epoch: 91 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0352, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.001904
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.002305
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.002337
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.004645
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.003011
Train Epoch: 92 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0349, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.001296
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.002302
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.002529
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001202
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.003393
Train Epoch: 93 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0350, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.005346
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.001059
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.001753
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.000527
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.001842
Train Epoch: 94 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0349, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.001019
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.000598
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.000445
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.003762
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.000805
Train Epoch: 95 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0349, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001986
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.002066
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.001046
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001016
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.001425
Train Epoch: 96 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0348, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.003387
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.001440
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.000921
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.001128
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.002020
Train Epoch: 97 - Loss: 0.0020, Accuracy: 99.99%

Test set: Average loss: 0.0348, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.000342
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.001336
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.001413
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.000749
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.002016
Train Epoch: 98 - Loss: 0.0019, Accuracy: 99.99%

Test set: Average loss: 0.0348, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.002286
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.003134
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.001995
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.000711
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.001748
Train Epoch: 99 - Loss: 0.0019, Accuracy: 99.99%

Test set: Average loss: 0.0348, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.003987
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.002294
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.002253
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001700
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.003762
Train Epoch: 100 - Loss: 0.0019, Accuracy: 99.99%

Test set: Average loss: 0.0348, Accuracy: 9892/10000 (98.92%)


Training completed. Best Accuracy: 99.00%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.295590
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.033355
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.437315
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.475118
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.199386
Train Epoch: 1 - Loss: 0.7671, Accuracy: 77.80%

Test set: Average loss: 0.2520, Accuracy: 9261/10000 (92.61%)

New best model saved with accuracy: 92.61%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.201703
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.250099
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.203404
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.138416
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.189031
Train Epoch: 2 - Loss: 0.2012, Accuracy: 93.97%

Test set: Average loss: 0.1437, Accuracy: 9576/10000 (95.76%)

New best model saved with accuracy: 95.76%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.122121
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.118658
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.079013
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.162351
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.058189
Train Epoch: 3 - Loss: 0.1267, Accuracy: 96.16%

Test set: Average loss: 0.1121, Accuracy: 9666/10000 (96.66%)

New best model saved with accuracy: 96.66%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.125714
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.094241
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.081165
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.135806
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.107862
Train Epoch: 4 - Loss: 0.0935, Accuracy: 97.22%

Test set: Average loss: 0.0791, Accuracy: 9746/10000 (97.46%)

New best model saved with accuracy: 97.46%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.049614
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.094624
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.048615
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.078939
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.146344
Train Epoch: 5 - Loss: 0.0755, Accuracy: 97.71%

Test set: Average loss: 0.0658, Accuracy: 9803/10000 (98.03%)

New best model saved with accuracy: 98.03%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.096391
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.048474
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.079867
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.075878
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.017579
Train Epoch: 6 - Loss: 0.0637, Accuracy: 98.02%

Test set: Average loss: 0.0614, Accuracy: 9801/10000 (98.01%)

Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.078234
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.080871
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.030848
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.111569
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.022743
Train Epoch: 7 - Loss: 0.0557, Accuracy: 98.36%

Test set: Average loss: 0.0520, Accuracy: 9836/10000 (98.36%)

New best model saved with accuracy: 98.36%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.061966
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.056553
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.050111
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.080772
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.064134
Train Epoch: 8 - Loss: 0.0496, Accuracy: 98.50%

Test set: Average loss: 0.0506, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.028384
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.049399
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.006394
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.060988
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.040277
Train Epoch: 9 - Loss: 0.0445, Accuracy: 98.70%

Test set: Average loss: 0.0508, Accuracy: 9832/10000 (98.32%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.014548
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.018491
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.012828
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.055804
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.044986
Train Epoch: 10 - Loss: 0.0408, Accuracy: 98.80%

Test set: Average loss: 0.0422, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.049665
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.036544
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.037192
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.024165
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.075206
Train Epoch: 11 - Loss: 0.0380, Accuracy: 98.88%

Test set: Average loss: 0.0412, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.048692
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.016249
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.118436
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.034894
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.035237
Train Epoch: 12 - Loss: 0.0350, Accuracy: 98.96%

Test set: Average loss: 0.0405, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.016645
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.007726
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.042679
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.048613
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.017336
Train Epoch: 13 - Loss: 0.0328, Accuracy: 99.05%

Test set: Average loss: 0.0395, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.047648
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.031645
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.043452
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.007445
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.017949
Train Epoch: 14 - Loss: 0.0308, Accuracy: 99.12%

Test set: Average loss: 0.0376, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.022718
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.010024
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.022032
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.010008
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.050348
Train Epoch: 15 - Loss: 0.0293, Accuracy: 99.19%

Test set: Average loss: 0.0381, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.005277
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.046292
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.059018
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.008153
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.008016
Train Epoch: 16 - Loss: 0.0280, Accuracy: 99.22%

Test set: Average loss: 0.0376, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.046025
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.018896
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.006986
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.028451
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.026928
Train Epoch: 17 - Loss: 0.0267, Accuracy: 99.24%

Test set: Average loss: 0.0373, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.031942
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.012991
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.015445
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.032804
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.017343
Train Epoch: 18 - Loss: 0.0258, Accuracy: 99.29%

Test set: Average loss: 0.0362, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.018660
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.072117
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.028354
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.019619
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.031442
Train Epoch: 19 - Loss: 0.0249, Accuracy: 99.32%

Test set: Average loss: 0.0365, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.029014
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.019685
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.048647
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.014727
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.052933
Train Epoch: 20 - Loss: 0.0243, Accuracy: 99.33%

Test set: Average loss: 0.0360, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.066833
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.030345
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.019293
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.024845
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.036280
Train Epoch: 21 - Loss: 0.0238, Accuracy: 99.36%

Test set: Average loss: 0.0362, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.016771
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.016756
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.027743
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.012902
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.018098
Train Epoch: 22 - Loss: 0.0234, Accuracy: 99.40%

Test set: Average loss: 0.0356, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.007228
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.026465
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.020688
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.035450
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.013339
Train Epoch: 23 - Loss: 0.0231, Accuracy: 99.40%

Test set: Average loss: 0.0356, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.009415
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.018728
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.042974
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.014967
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.030636
Train Epoch: 24 - Loss: 0.0229, Accuracy: 99.42%

Test set: Average loss: 0.0352, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.012181
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.020780
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.029841
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.060669
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.046752
Train Epoch: 25 - Loss: 0.0228, Accuracy: 99.43%

Test set: Average loss: 0.0352, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.015745
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.007443
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.025933
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.033245
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.022238
Train Epoch: 26 - Loss: 0.0294, Accuracy: 99.16%

Test set: Average loss: 0.0438, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.023089
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.039054
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.041528
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.040284
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.012460
Train Epoch: 27 - Loss: 0.0271, Accuracy: 99.18%

Test set: Average loss: 0.0419, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.008039
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.022101
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.014068
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.013739
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.007176
Train Epoch: 28 - Loss: 0.0254, Accuracy: 99.27%

Test set: Average loss: 0.0409, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.082029
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.010711
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.059604
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.006009
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.051463
Train Epoch: 29 - Loss: 0.0235, Accuracy: 99.30%

Test set: Average loss: 0.0324, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.017267
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.012150
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.033206
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.039093
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.024584
Train Epoch: 30 - Loss: 0.0217, Accuracy: 99.39%

Test set: Average loss: 0.0353, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.002752
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.023613
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.037835
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.018287
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.016506
Train Epoch: 31 - Loss: 0.0199, Accuracy: 99.42%

Test set: Average loss: 0.0343, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.036451
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.004413
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.011234
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.020914
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.015249
Train Epoch: 32 - Loss: 0.0184, Accuracy: 99.50%

Test set: Average loss: 0.0347, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.020767
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.010701
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.007820
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.017711
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.023515
Train Epoch: 33 - Loss: 0.0176, Accuracy: 99.52%

Test set: Average loss: 0.0361, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.005591
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.004700
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.029454
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.053720
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.009445
Train Epoch: 34 - Loss: 0.0159, Accuracy: 99.57%

Test set: Average loss: 0.0347, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.004484
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.003712
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.008132
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.005492
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.006336
Train Epoch: 35 - Loss: 0.0148, Accuracy: 99.62%

Test set: Average loss: 0.0358, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.005251
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.006210
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.022324
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.024164
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.005616
Train Epoch: 36 - Loss: 0.0138, Accuracy: 99.66%

Test set: Average loss: 0.0389, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.008493
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.002253
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.005909
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.013158
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.005804
Train Epoch: 37 - Loss: 0.0132, Accuracy: 99.68%

Test set: Average loss: 0.0316, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.004112
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.008618
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.003403
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.027033
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.014856
Train Epoch: 38 - Loss: 0.0122, Accuracy: 99.72%

Test set: Average loss: 0.0326, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.003319
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.021925
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.009204
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.004758
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.005061
Train Epoch: 39 - Loss: 0.0116, Accuracy: 99.73%

Test set: Average loss: 0.0335, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.001233
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.002863
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.012373
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.010421
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.005706
Train Epoch: 40 - Loss: 0.0111, Accuracy: 99.77%

Test set: Average loss: 0.0323, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.006290
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.007610
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.002417
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.003488
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.007575
Train Epoch: 41 - Loss: 0.0105, Accuracy: 99.78%

Test set: Average loss: 0.0321, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.004702
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.002683
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.037368
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.005061
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.002729
Train Epoch: 42 - Loss: 0.0100, Accuracy: 99.80%

Test set: Average loss: 0.0317, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.010298
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.001845
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.001603
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.004375
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.012623
Train Epoch: 43 - Loss: 0.0097, Accuracy: 99.80%

Test set: Average loss: 0.0316, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.008654
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.004439
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.007139
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.006207
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.005629
Train Epoch: 44 - Loss: 0.0094, Accuracy: 99.82%

Test set: Average loss: 0.0318, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.002277
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.003778
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.002612
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.001526
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.008201
Train Epoch: 45 - Loss: 0.0092, Accuracy: 99.83%

Test set: Average loss: 0.0317, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.003440
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.001194
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.007453
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.004429
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.003386
Train Epoch: 46 - Loss: 0.0089, Accuracy: 99.83%

Test set: Average loss: 0.0317, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.006984
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.013537
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.008716
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.002860
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.004252
Train Epoch: 47 - Loss: 0.0088, Accuracy: 99.84%

Test set: Average loss: 0.0316, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.004605
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.005159
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.016997
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.008400
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.007616
Train Epoch: 48 - Loss: 0.0087, Accuracy: 99.85%

Test set: Average loss: 0.0317, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.008290
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.002711
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.023478
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.017602
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.009928
Train Epoch: 49 - Loss: 0.0086, Accuracy: 99.84%

Test set: Average loss: 0.0316, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.009003
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.014769
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.005719
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.017178
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.008953
Train Epoch: 50 - Loss: 0.0085, Accuracy: 99.84%

Test set: Average loss: 0.0316, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.008118
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.007106
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.011690
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.016145
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.029556
Train Epoch: 51 - Loss: 0.0124, Accuracy: 99.68%

Test set: Average loss: 0.0339, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.031104
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.010381
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.008837
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.009814
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.006319
Train Epoch: 52 - Loss: 0.0112, Accuracy: 99.71%

Test set: Average loss: 0.0336, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.033597
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.008545
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.004440
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.004718
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.008764
Train Epoch: 53 - Loss: 0.0112, Accuracy: 99.70%

Test set: Average loss: 0.0326, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.041088
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.010597
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.012503
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.009369
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.001947
Train Epoch: 54 - Loss: 0.0101, Accuracy: 99.75%

Test set: Average loss: 0.0337, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.003620
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.017462
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.003644
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.004555
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.015789
Train Epoch: 55 - Loss: 0.0098, Accuracy: 99.76%

Test set: Average loss: 0.0351, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.007360
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.009824
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.002364
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.006502
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.000988
Train Epoch: 56 - Loss: 0.0086, Accuracy: 99.79%

Test set: Average loss: 0.0344, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.002095
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.005055
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.010337
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.017849
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.004911
Train Epoch: 57 - Loss: 0.0080, Accuracy: 99.84%

Test set: Average loss: 0.0360, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.008046
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.009987
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.003946
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.002400
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.003702
Train Epoch: 58 - Loss: 0.0075, Accuracy: 99.84%

Test set: Average loss: 0.0327, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.007347
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.000858
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.003498
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.008046
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.010278
Train Epoch: 59 - Loss: 0.0070, Accuracy: 99.88%

Test set: Average loss: 0.0333, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.009463
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.003407
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.001686
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.005037
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.002215
Train Epoch: 60 - Loss: 0.0065, Accuracy: 99.88%

Test set: Average loss: 0.0338, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.001729
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.001397
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.007582
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.006293
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.000602
Train Epoch: 61 - Loss: 0.0061, Accuracy: 99.90%

Test set: Average loss: 0.0330, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.005698
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.014736
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.000474
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.022570
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.010112
Train Epoch: 62 - Loss: 0.0057, Accuracy: 99.92%

Test set: Average loss: 0.0325, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.004000
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.002843
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.011531
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.001430
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.003304
Train Epoch: 63 - Loss: 0.0054, Accuracy: 99.92%

Test set: Average loss: 0.0334, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.003969
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.007198
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.005067
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.000946
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.002542
Train Epoch: 64 - Loss: 0.0051, Accuracy: 99.93%

Test set: Average loss: 0.0328, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.006278
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.003225
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.001085
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.002130
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.005459
Train Epoch: 65 - Loss: 0.0049, Accuracy: 99.95%

Test set: Average loss: 0.0334, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.005218
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.005161
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.005035
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.004714
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.005404
Train Epoch: 66 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0321, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.001320
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.001911
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.005702
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.002896
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.003373
Train Epoch: 67 - Loss: 0.0045, Accuracy: 99.95%

Test set: Average loss: 0.0320, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.005815
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.004284
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.007871
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.001350
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.004840
Train Epoch: 68 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0324, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.009280
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.001747
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.003531
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.007164
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.002738
Train Epoch: 69 - Loss: 0.0042, Accuracy: 99.96%

Test set: Average loss: 0.0328, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.002297
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.002682
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.002988
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.001785
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.007502
Train Epoch: 70 - Loss: 0.0042, Accuracy: 99.96%

Test set: Average loss: 0.0326, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.003107
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.002258
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.005235
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.001394
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.006329
Train Epoch: 71 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0324, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.032044
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.002918
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.003518
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.003560
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.003301
Train Epoch: 72 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0322, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.001939
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.002205
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.007639
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.004412
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001412
Train Epoch: 73 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0324, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.002430
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002283
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.003707
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.001062
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.003135
Train Epoch: 74 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0324, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.003265
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.003902
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.011561
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.001468
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.002728
Train Epoch: 75 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0324, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.001984
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.004838
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.001301
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.002160
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.009639
Train Epoch: 76 - Loss: 0.0058, Accuracy: 99.88%

Test set: Average loss: 0.0337, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.012101
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.005742
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.001115
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.009071
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.001747
Train Epoch: 77 - Loss: 0.0054, Accuracy: 99.90%

Test set: Average loss: 0.0345, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.007262
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.003908
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.001472
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.003078
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.003920
Train Epoch: 78 - Loss: 0.0053, Accuracy: 99.91%

Test set: Average loss: 0.0345, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.003512
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.001758
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.012132
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.005795
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.035040
Train Epoch: 79 - Loss: 0.0049, Accuracy: 99.92%

Test set: Average loss: 0.0334, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.001066
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.001145
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.004527
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.002897
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.005921
Train Epoch: 80 - Loss: 0.0045, Accuracy: 99.94%

Test set: Average loss: 0.0340, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.000850
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.002283
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.001945
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.002385
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.003147
Train Epoch: 81 - Loss: 0.0042, Accuracy: 99.94%

Test set: Average loss: 0.0358, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.003148
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002512
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.004348
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.001112
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.007419
Train Epoch: 82 - Loss: 0.0039, Accuracy: 99.95%

Test set: Average loss: 0.0328, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.001595
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.000704
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.001889
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.001170
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.007150
Train Epoch: 83 - Loss: 0.0037, Accuracy: 99.95%

Test set: Average loss: 0.0356, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.004160
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.001047
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.002432
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.012465
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.009152
Train Epoch: 84 - Loss: 0.0035, Accuracy: 99.95%

Test set: Average loss: 0.0333, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.004284
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.002419
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.001858
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.003052
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.000341
Train Epoch: 85 - Loss: 0.0032, Accuracy: 99.97%

Test set: Average loss: 0.0364, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.002764
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.001894
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.006117
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.022389
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.001385
Train Epoch: 86 - Loss: 0.0031, Accuracy: 99.96%

Test set: Average loss: 0.0337, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.001749
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.002503
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001222
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001605
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.000689
Train Epoch: 87 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0342, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.000756
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001707
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.003034
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.001412
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001000
Train Epoch: 88 - Loss: 0.0028, Accuracy: 99.97%

Test set: Average loss: 0.0336, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.003003
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.001416
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.006885
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.001258
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.002845
Train Epoch: 89 - Loss: 0.0026, Accuracy: 99.97%

Test set: Average loss: 0.0333, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.015746
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.002051
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.000842
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001996
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.001902
Train Epoch: 90 - Loss: 0.0025, Accuracy: 99.97%

Test set: Average loss: 0.0336, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.000378
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.002801
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.007444
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.002319
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.002488
Train Epoch: 91 - Loss: 0.0024, Accuracy: 99.97%

Test set: Average loss: 0.0340, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.005409
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.001661
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.002015
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.001645
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.002651
Train Epoch: 92 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0335, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.001541
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.005697
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.003513
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.000486
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.002060
Train Epoch: 93 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0338, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.001867
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.001866
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.000624
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.003148
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.022250
Train Epoch: 94 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0338, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.000732
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.000532
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.004459
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001090
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001763
Train Epoch: 95 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0339, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.002157
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.001665
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.001104
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.002073
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.005530
Train Epoch: 96 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0338, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.002039
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.005215
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.000769
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.001680
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.002103
Train Epoch: 97 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0336, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.002272
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.001027
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.001947
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.002962
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.001982
Train Epoch: 98 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0336, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.002334
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.002287
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.001999
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.002401
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.002348
Train Epoch: 99 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0336, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.000229
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.000641
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.002348
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.000638
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.001774
Train Epoch: 100 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0336, Accuracy: 9901/10000 (99.01%)


Training completed. Best Accuracy: 99.04%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.310010
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.991515
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.402567
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.311139
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.289552
Train Epoch: 1 - Loss: 0.7207, Accuracy: 79.38%

Test set: Average loss: 0.2494, Accuracy: 9248/10000 (92.48%)

New best model saved with accuracy: 92.48%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.227612
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.139687
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.220320
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.233134
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.164542
Train Epoch: 2 - Loss: 0.2059, Accuracy: 93.90%

Test set: Average loss: 0.1503, Accuracy: 9523/10000 (95.23%)

New best model saved with accuracy: 95.23%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.105198
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.127109
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.194537
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.124560
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.138000
Train Epoch: 3 - Loss: 0.1325, Accuracy: 96.07%

Test set: Average loss: 0.1013, Accuracy: 9674/10000 (96.74%)

New best model saved with accuracy: 96.74%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.114124
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.115991
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.120956
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.087932
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.093803
Train Epoch: 4 - Loss: 0.0980, Accuracy: 97.03%

Test set: Average loss: 0.0801, Accuracy: 9755/10000 (97.55%)

New best model saved with accuracy: 97.55%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.102349
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.158142
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.035419
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.049048
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.073953
Train Epoch: 5 - Loss: 0.0777, Accuracy: 97.69%

Test set: Average loss: 0.0679, Accuracy: 9782/10000 (97.82%)

New best model saved with accuracy: 97.82%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.121167
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.020385
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.100947
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.114388
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.090628
Train Epoch: 6 - Loss: 0.0654, Accuracy: 98.05%

Test set: Average loss: 0.0574, Accuracy: 9808/10000 (98.08%)

New best model saved with accuracy: 98.08%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.027019
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.078260
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.029119
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.074479
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.037555
Train Epoch: 7 - Loss: 0.0572, Accuracy: 98.30%

Test set: Average loss: 0.0526, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.071995
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.022291
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.024192
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.075803
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.023439
Train Epoch: 8 - Loss: 0.0498, Accuracy: 98.52%

Test set: Average loss: 0.0560, Accuracy: 9808/10000 (98.08%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.031001
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.024638
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.044776
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.041013
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.035514
Train Epoch: 9 - Loss: 0.0450, Accuracy: 98.66%

Test set: Average loss: 0.0515, Accuracy: 9844/10000 (98.44%)

New best model saved with accuracy: 98.44%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.075716
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.052855
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.054932
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.009362
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.063336
Train Epoch: 10 - Loss: 0.0412, Accuracy: 98.78%

Test set: Average loss: 0.0437, Accuracy: 9852/10000 (98.52%)

New best model saved with accuracy: 98.52%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.016250
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.049869
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.026228
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.017573
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.034627
Train Epoch: 11 - Loss: 0.0381, Accuracy: 98.87%

Test set: Average loss: 0.0422, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.030741
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.041633
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.053528
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.019603
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.062747
Train Epoch: 12 - Loss: 0.0354, Accuracy: 98.95%

Test set: Average loss: 0.0395, Accuracy: 9860/10000 (98.60%)

New best model saved with accuracy: 98.60%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.020796
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.011660
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.050785
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.007571
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.029417
Train Epoch: 13 - Loss: 0.0329, Accuracy: 99.04%

Test set: Average loss: 0.0387, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.156552
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.016736
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.019032
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.058738
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.005563
Train Epoch: 14 - Loss: 0.0309, Accuracy: 99.11%

Test set: Average loss: 0.0393, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.011079
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.054328
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.008159
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.078824
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.019660
Train Epoch: 15 - Loss: 0.0291, Accuracy: 99.18%

Test set: Average loss: 0.0376, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.009977
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.006058
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.037081
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.020704
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.014820
Train Epoch: 16 - Loss: 0.0277, Accuracy: 99.20%

Test set: Average loss: 0.0365, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.012588
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.044531
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.012299
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.002588
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.017672
Train Epoch: 17 - Loss: 0.0266, Accuracy: 99.27%

Test set: Average loss: 0.0357, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.034584
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.024402
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.018983
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.013630
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.032345
Train Epoch: 18 - Loss: 0.0257, Accuracy: 99.31%

Test set: Average loss: 0.0355, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.018303
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.030733
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.015316
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.035830
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.012113
Train Epoch: 19 - Loss: 0.0248, Accuracy: 99.34%

Test set: Average loss: 0.0358, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.012114
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.030549
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.037394
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.018678
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.021870
Train Epoch: 20 - Loss: 0.0242, Accuracy: 99.36%

Test set: Average loss: 0.0352, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.022618
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.017192
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.008975
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.033670
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.090185
Train Epoch: 21 - Loss: 0.0236, Accuracy: 99.39%

Test set: Average loss: 0.0347, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.052481
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.018213
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.005950
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.013166
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.057751
Train Epoch: 22 - Loss: 0.0234, Accuracy: 99.39%

Test set: Average loss: 0.0345, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.018603
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.019286
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.007045
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.053204
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.030258
Train Epoch: 23 - Loss: 0.0230, Accuracy: 99.41%

Test set: Average loss: 0.0343, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.045563
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.024075
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.006901
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.011393
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.019926
Train Epoch: 24 - Loss: 0.0228, Accuracy: 99.41%

Test set: Average loss: 0.0343, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.005707
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.005212
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.049509
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.026672
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.013101
Train Epoch: 25 - Loss: 0.0227, Accuracy: 99.41%

Test set: Average loss: 0.0343, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.019008
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.018889
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.016682
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.013410
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.027189
Train Epoch: 26 - Loss: 0.0292, Accuracy: 99.13%

Test set: Average loss: 0.0409, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.034893
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.005266
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.035139
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.043853
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.005253
Train Epoch: 27 - Loss: 0.0274, Accuracy: 99.18%

Test set: Average loss: 0.0432, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.023874
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.014906
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.055221
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.007398
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.008339
Train Epoch: 28 - Loss: 0.0250, Accuracy: 99.27%

Test set: Average loss: 0.0394, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.013936
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.052761
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.026694
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.049272
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.015384
Train Epoch: 29 - Loss: 0.0231, Accuracy: 99.35%

Test set: Average loss: 0.0346, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.025112
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.023535
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.020956
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.006186
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.019674
Train Epoch: 30 - Loss: 0.0213, Accuracy: 99.37%

Test set: Average loss: 0.0354, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.020020
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.004164
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.003150
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.010131
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.018094
Train Epoch: 31 - Loss: 0.0197, Accuracy: 99.44%

Test set: Average loss: 0.0388, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.012965
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.008890
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.006050
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.013720
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.042166
Train Epoch: 32 - Loss: 0.0184, Accuracy: 99.51%

Test set: Average loss: 0.0432, Accuracy: 9850/10000 (98.50%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.010319
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.012975
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.005884
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.006295
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.008209
Train Epoch: 33 - Loss: 0.0169, Accuracy: 99.55%

Test set: Average loss: 0.0337, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.004830
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.014811
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.005699
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.022047
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.005174
Train Epoch: 34 - Loss: 0.0156, Accuracy: 99.57%

Test set: Average loss: 0.0329, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.004446
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.007910
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.012680
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.026013
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.026092
Train Epoch: 35 - Loss: 0.0144, Accuracy: 99.64%

Test set: Average loss: 0.0321, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.002520
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.010823
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.029134
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.026899
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.012535
Train Epoch: 36 - Loss: 0.0136, Accuracy: 99.66%

Test set: Average loss: 0.0382, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.010796
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.006509
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.002262
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.014332
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.005060
Train Epoch: 37 - Loss: 0.0128, Accuracy: 99.69%

Test set: Average loss: 0.0319, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.005634
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.014134
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.003271
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.010933
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.006357
Train Epoch: 38 - Loss: 0.0119, Accuracy: 99.72%

Test set: Average loss: 0.0322, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.005453
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.002287
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.016107
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.014908
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.017178
Train Epoch: 39 - Loss: 0.0112, Accuracy: 99.74%

Test set: Average loss: 0.0317, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.009944
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.036775
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.007220
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.017920
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.008743
Train Epoch: 40 - Loss: 0.0107, Accuracy: 99.77%

Test set: Average loss: 0.0310, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.007919
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.008213
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.007530
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.013590
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.012204
Train Epoch: 41 - Loss: 0.0102, Accuracy: 99.78%

Test set: Average loss: 0.0316, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.007113
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.019499
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.025701
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.014441
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.001839
Train Epoch: 42 - Loss: 0.0098, Accuracy: 99.80%

Test set: Average loss: 0.0308, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.015002
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.005566
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.053185
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.003029
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.002924
Train Epoch: 43 - Loss: 0.0094, Accuracy: 99.83%

Test set: Average loss: 0.0314, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.002517
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.004763
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.003972
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.010985
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.005804
Train Epoch: 44 - Loss: 0.0091, Accuracy: 99.83%

Test set: Average loss: 0.0317, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.007836
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.006544
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.001458
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.011006
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.002223
Train Epoch: 45 - Loss: 0.0090, Accuracy: 99.85%

Test set: Average loss: 0.0313, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.006139
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.009214
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.012666
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.019598
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.005000
Train Epoch: 46 - Loss: 0.0087, Accuracy: 99.85%

Test set: Average loss: 0.0313, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.003472
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.013118
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.009490
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.003952
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.015393
Train Epoch: 47 - Loss: 0.0086, Accuracy: 99.86%

Test set: Average loss: 0.0310, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.011511
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.007305
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.006156
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.010056
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.006756
Train Epoch: 48 - Loss: 0.0085, Accuracy: 99.86%

Test set: Average loss: 0.0311, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.007748
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.003796
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.004491
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.004138
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.008408
Train Epoch: 49 - Loss: 0.0084, Accuracy: 99.86%

Test set: Average loss: 0.0311, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.006263
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.004305
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.011010
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.010457
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.012196
Train Epoch: 50 - Loss: 0.0084, Accuracy: 99.86%

Test set: Average loss: 0.0311, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.002147
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.001952
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.047148
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.010758
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.006770
Train Epoch: 51 - Loss: 0.0118, Accuracy: 99.71%

Test set: Average loss: 0.0337, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.002758
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.005997
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.023640
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.011192
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.041999
Train Epoch: 52 - Loss: 0.0112, Accuracy: 99.70%

Test set: Average loss: 0.0349, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.006069
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.010974
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.008391
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.012049
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.003487
Train Epoch: 53 - Loss: 0.0104, Accuracy: 99.76%

Test set: Average loss: 0.0366, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.007562
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.008523
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.010216
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.004672
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.009794
Train Epoch: 54 - Loss: 0.0099, Accuracy: 99.78%

Test set: Average loss: 0.0325, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.008313
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.007316
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.017597
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.002580
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.006338
Train Epoch: 55 - Loss: 0.0087, Accuracy: 99.81%

Test set: Average loss: 0.0353, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.005249
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.008105
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.005881
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.027317
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.013555
Train Epoch: 56 - Loss: 0.0084, Accuracy: 99.83%

Test set: Average loss: 0.0333, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.002318
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.007523
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.003868
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.008686
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.004806
Train Epoch: 57 - Loss: 0.0078, Accuracy: 99.84%

Test set: Average loss: 0.0347, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.012079
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.001995
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.007430
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.002519
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.007582
Train Epoch: 58 - Loss: 0.0072, Accuracy: 99.87%

Test set: Average loss: 0.0334, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.002017
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.002805
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.006339
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.003987
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.002763
Train Epoch: 59 - Loss: 0.0067, Accuracy: 99.88%

Test set: Average loss: 0.0332, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.095445
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.002813
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.005711
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.006720
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.004480
Train Epoch: 60 - Loss: 0.0062, Accuracy: 99.89%

Test set: Average loss: 0.0338, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.012543
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.004688
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.003326
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.007267
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.003262
Train Epoch: 61 - Loss: 0.0058, Accuracy: 99.92%

Test set: Average loss: 0.0339, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.019293
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.003827
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.004792
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.001314
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.004236
Train Epoch: 62 - Loss: 0.0056, Accuracy: 99.92%

Test set: Average loss: 0.0335, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.001752
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.002019
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.003149
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.002262
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.008732
Train Epoch: 63 - Loss: 0.0051, Accuracy: 99.93%

Test set: Average loss: 0.0346, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.001292
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.001479
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.001660
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.003004
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.002835
Train Epoch: 64 - Loss: 0.0050, Accuracy: 99.94%

Test set: Average loss: 0.0337, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.004627
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.002418
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.000904
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.003868
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.017717
Train Epoch: 65 - Loss: 0.0048, Accuracy: 99.95%

Test set: Average loss: 0.0328, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.000380
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.001800
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.001051
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.009031
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.001046
Train Epoch: 66 - Loss: 0.0046, Accuracy: 99.94%

Test set: Average loss: 0.0326, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.000843
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.004514
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.002988
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.008572
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.002936
Train Epoch: 67 - Loss: 0.0044, Accuracy: 99.94%

Test set: Average loss: 0.0327, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.002247
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.002133
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.002277
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.001370
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.003816
Train Epoch: 68 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0331, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.000674
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.003869
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.015205
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.002137
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.001112
Train Epoch: 69 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0325, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.001915
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.003570
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.003485
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.003090
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.005106
Train Epoch: 70 - Loss: 0.0041, Accuracy: 99.95%

Test set: Average loss: 0.0328, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.003575
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.002015
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.001621
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.002406
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.003140
Train Epoch: 71 - Loss: 0.0040, Accuracy: 99.95%

Test set: Average loss: 0.0326, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.003510
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.001607
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.003021
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.005977
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.001185
Train Epoch: 72 - Loss: 0.0040, Accuracy: 99.95%

Test set: Average loss: 0.0327, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.005316
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.004325
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.003284
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.004543
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.004355
Train Epoch: 73 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.001988
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002175
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.006519
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.003831
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.001802
Train Epoch: 74 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.001694
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.004264
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.002197
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.000747
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.001108
Train Epoch: 75 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.002483
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.005072
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.003042
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.003438
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.007540
Train Epoch: 76 - Loss: 0.0054, Accuracy: 99.91%

Test set: Average loss: 0.0329, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.003391
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.001762
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.003453
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.001025
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.001873
Train Epoch: 77 - Loss: 0.0050, Accuracy: 99.93%

Test set: Average loss: 0.0339, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.008102
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.002697
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.001396
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.003466
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.004685
Train Epoch: 78 - Loss: 0.0048, Accuracy: 99.92%

Test set: Average loss: 0.0340, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.002237
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.003610
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.003323
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.001879
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.002650
Train Epoch: 79 - Loss: 0.0047, Accuracy: 99.92%

Test set: Average loss: 0.0369, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.002461
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.006700
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.002188
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.000674
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.006788
Train Epoch: 80 - Loss: 0.0043, Accuracy: 99.94%

Test set: Average loss: 0.0342, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.003372
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.001713
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.001606
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.004242
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.002266
Train Epoch: 81 - Loss: 0.0039, Accuracy: 99.95%

Test set: Average loss: 0.0347, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.001828
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.003315
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.003421
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.001527
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.001238
Train Epoch: 82 - Loss: 0.0037, Accuracy: 99.95%

Test set: Average loss: 0.0358, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.003474
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.001093
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.005364
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.001225
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.002750
Train Epoch: 83 - Loss: 0.0036, Accuracy: 99.95%

Test set: Average loss: 0.0346, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.001807
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.002510
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.000717
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.001691
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.002005
Train Epoch: 84 - Loss: 0.0033, Accuracy: 99.97%

Test set: Average loss: 0.0343, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.005188
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.001794
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.003000
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.001691
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.002128
Train Epoch: 85 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0345, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.007906
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.002480
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.000620
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.002134
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.001891
Train Epoch: 86 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0339, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.005389
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.001337
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.002791
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.000918
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.003133
Train Epoch: 87 - Loss: 0.0028, Accuracy: 99.98%

Test set: Average loss: 0.0344, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.001353
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001073
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.001134
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.003239
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.002038
Train Epoch: 88 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0349, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.002058
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.003392
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.001309
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.005388
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.002114
Train Epoch: 89 - Loss: 0.0026, Accuracy: 99.98%

Test set: Average loss: 0.0344, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.000996
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.001633
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.000981
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.004263
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.000721
Train Epoch: 90 - Loss: 0.0025, Accuracy: 99.98%

Test set: Average loss: 0.0344, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.001904
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.000671
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.002847
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.001915
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.002076
Train Epoch: 91 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0343, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.001933
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.001382
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.000714
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.002254
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.006288
Train Epoch: 92 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0343, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.001698
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.003106
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.000266
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.000804
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.000317
Train Epoch: 93 - Loss: 0.0023, Accuracy: 99.99%

Test set: Average loss: 0.0342, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.001307
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.002334
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.003025
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.001629
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.002223
Train Epoch: 94 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0344, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.002798
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.002145
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.001609
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001193
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001283
Train Epoch: 95 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001455
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.002520
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.003830
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.002706
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.001057
Train Epoch: 96 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0342, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.004641
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.002758
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.003080
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.002578
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.002668
Train Epoch: 97 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0343, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.001402
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.001153
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.005460
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.001909
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.002086
Train Epoch: 98 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0342, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.000806
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.001124
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.000875
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.000700
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.001804
Train Epoch: 99 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0342, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.002018
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.000216
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.002100
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.004563
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.002305
Train Epoch: 100 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0342, Accuracy: 9895/10000 (98.95%)


Training completed. Best Accuracy: 99.00%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.304318
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.978133
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.604776
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.362110
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.177862
Train Epoch: 1 - Loss: 0.7299, Accuracy: 78.99%

Test set: Average loss: 0.2621, Accuracy: 9222/10000 (92.22%)

New best model saved with accuracy: 92.22%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.460720
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.213618
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.268701
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.182128
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.100032
Train Epoch: 2 - Loss: 0.2127, Accuracy: 93.63%

Test set: Average loss: 0.1541, Accuracy: 9529/10000 (95.29%)

New best model saved with accuracy: 95.29%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.136938
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.070604
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.052224
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.136517
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.130585
Train Epoch: 3 - Loss: 0.1350, Accuracy: 95.99%

Test set: Average loss: 0.1045, Accuracy: 9694/10000 (96.94%)

New best model saved with accuracy: 96.94%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.211160
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.118023
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.050913
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.144070
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.131699
Train Epoch: 4 - Loss: 0.0989, Accuracy: 97.09%

Test set: Average loss: 0.0819, Accuracy: 9732/10000 (97.32%)

New best model saved with accuracy: 97.32%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.032396
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.054901
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.073827
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.135738
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.054607
Train Epoch: 5 - Loss: 0.0783, Accuracy: 97.62%

Test set: Average loss: 0.0790, Accuracy: 9754/10000 (97.54%)

New best model saved with accuracy: 97.54%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.100439
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.064225
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.055438
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.056033
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.086898
Train Epoch: 6 - Loss: 0.0661, Accuracy: 98.00%

Test set: Average loss: 0.0576, Accuracy: 9825/10000 (98.25%)

New best model saved with accuracy: 98.25%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.055518
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.045513
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.095836
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.068904
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.066532
Train Epoch: 7 - Loss: 0.0574, Accuracy: 98.22%

Test set: Average loss: 0.0536, Accuracy: 9829/10000 (98.29%)

New best model saved with accuracy: 98.29%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.030829
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.052184
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.011305
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.032377
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.038256
Train Epoch: 8 - Loss: 0.0510, Accuracy: 98.48%

Test set: Average loss: 0.0506, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.041463
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.007701
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.019249
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.014827
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.027631
Train Epoch: 9 - Loss: 0.0458, Accuracy: 98.64%

Test set: Average loss: 0.0462, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.012376
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.024958
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.027784
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.018624
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.026178
Train Epoch: 10 - Loss: 0.0422, Accuracy: 98.74%

Test set: Average loss: 0.0450, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.058422
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.045583
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.050930
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.038082
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.040081
Train Epoch: 11 - Loss: 0.0385, Accuracy: 98.89%

Test set: Average loss: 0.0413, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.022191
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.023772
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.036082
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.039600
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.055299
Train Epoch: 12 - Loss: 0.0355, Accuracy: 98.96%

Test set: Average loss: 0.0412, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.037373
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.028563
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.031997
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.035620
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.006867
Train Epoch: 13 - Loss: 0.0332, Accuracy: 99.05%

Test set: Average loss: 0.0388, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.023756
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.008957
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.096199
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.041566
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.046794
Train Epoch: 14 - Loss: 0.0313, Accuracy: 99.10%

Test set: Average loss: 0.0382, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.047037
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.002979
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.005951
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.027497
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.031961
Train Epoch: 15 - Loss: 0.0295, Accuracy: 99.17%

Test set: Average loss: 0.0399, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.028505
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.050397
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.023135
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.011425
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.018112
Train Epoch: 16 - Loss: 0.0280, Accuracy: 99.23%

Test set: Average loss: 0.0360, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.006774
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.045165
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.009146
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.101638
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.010927
Train Epoch: 17 - Loss: 0.0269, Accuracy: 99.25%

Test set: Average loss: 0.0343, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.072011
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.017475
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.068914
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.014883
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.038372
Train Epoch: 18 - Loss: 0.0256, Accuracy: 99.29%

Test set: Average loss: 0.0348, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.031818
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.007375
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.005253
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.011417
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.041046
Train Epoch: 19 - Loss: 0.0249, Accuracy: 99.33%

Test set: Average loss: 0.0348, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.029743
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.023033
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.022786
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.009392
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.011670
Train Epoch: 20 - Loss: 0.0242, Accuracy: 99.33%

Test set: Average loss: 0.0343, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.032714
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.005760
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.024335
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.010540
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.044308
Train Epoch: 21 - Loss: 0.0237, Accuracy: 99.36%

Test set: Average loss: 0.0341, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.028406
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.037649
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.031992
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.015656
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.015533
Train Epoch: 22 - Loss: 0.0233, Accuracy: 99.36%

Test set: Average loss: 0.0338, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.018226
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.043942
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.015664
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.024546
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.018559
Train Epoch: 23 - Loss: 0.0230, Accuracy: 99.41%

Test set: Average loss: 0.0335, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.009554
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.018515
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.005177
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.044577
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.007208
Train Epoch: 24 - Loss: 0.0228, Accuracy: 99.41%

Test set: Average loss: 0.0336, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.030550
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.004405
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.017872
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.019304
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.028320
Train Epoch: 25 - Loss: 0.0227, Accuracy: 99.42%

Test set: Average loss: 0.0336, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.042564
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.054798
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.026224
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.015019
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.076056
Train Epoch: 26 - Loss: 0.0294, Accuracy: 99.10%

Test set: Average loss: 0.0377, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.032270
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.027587
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.005961
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.034344
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.004874
Train Epoch: 27 - Loss: 0.0269, Accuracy: 99.20%

Test set: Average loss: 0.0370, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.012681
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.048444
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.028906
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.024282
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.011336
Train Epoch: 28 - Loss: 0.0254, Accuracy: 99.23%

Test set: Average loss: 0.0374, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.024406
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.015000
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.006496
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.010963
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.007541
Train Epoch: 29 - Loss: 0.0230, Accuracy: 99.35%

Test set: Average loss: 0.0334, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.024593
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.045689
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.024933
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.006142
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.015745
Train Epoch: 30 - Loss: 0.0214, Accuracy: 99.36%

Test set: Average loss: 0.0366, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.007075
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.027647
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.010904
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.016987
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.012245
Train Epoch: 31 - Loss: 0.0194, Accuracy: 99.44%

Test set: Average loss: 0.0332, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.016109
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.009294
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.042260
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.021980
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.015470
Train Epoch: 32 - Loss: 0.0177, Accuracy: 99.51%

Test set: Average loss: 0.0302, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.021801
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.033351
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.014915
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.026899
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.006917
Train Epoch: 33 - Loss: 0.0164, Accuracy: 99.56%

Test set: Average loss: 0.0309, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.030868
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.025499
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.016159
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.013817
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.040207
Train Epoch: 34 - Loss: 0.0154, Accuracy: 99.57%

Test set: Average loss: 0.0317, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.004095
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.007119
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.005378
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.030234
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.012590
Train Epoch: 35 - Loss: 0.0142, Accuracy: 99.62%

Test set: Average loss: 0.0337, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.013650
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.012584
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.010687
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.007128
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.007300
Train Epoch: 36 - Loss: 0.0132, Accuracy: 99.67%

Test set: Average loss: 0.0293, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.002338
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.007397
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.027205
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.015426
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.002203
Train Epoch: 37 - Loss: 0.0123, Accuracy: 99.69%

Test set: Average loss: 0.0305, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.009418
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.010396
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.008544
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.003662
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.005081
Train Epoch: 38 - Loss: 0.0114, Accuracy: 99.73%

Test set: Average loss: 0.0308, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.005900
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.009429
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.010711
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.005748
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.008617
Train Epoch: 39 - Loss: 0.0108, Accuracy: 99.76%

Test set: Average loss: 0.0304, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.028334
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.003535
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.001827
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.009587
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.003453
Train Epoch: 40 - Loss: 0.0103, Accuracy: 99.77%

Test set: Average loss: 0.0300, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.025025
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.003890
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.028813
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.011060
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.016101
Train Epoch: 41 - Loss: 0.0099, Accuracy: 99.80%

Test set: Average loss: 0.0301, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.044184
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.005882
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.029281
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.007679
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.024182
Train Epoch: 42 - Loss: 0.0095, Accuracy: 99.81%

Test set: Average loss: 0.0291, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.004824
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.004671
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.020459
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.008293
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.010397
Train Epoch: 43 - Loss: 0.0091, Accuracy: 99.83%

Test set: Average loss: 0.0290, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.009682
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.008537
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.002007
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.011295
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.003900
Train Epoch: 44 - Loss: 0.0088, Accuracy: 99.83%

Test set: Average loss: 0.0292, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.009973
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.005906
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.007310
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.007307
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.005247
Train Epoch: 45 - Loss: 0.0087, Accuracy: 99.84%

Test set: Average loss: 0.0292, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.006119
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.005858
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.005091
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.002659
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.006373
Train Epoch: 46 - Loss: 0.0084, Accuracy: 99.84%

Test set: Average loss: 0.0292, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.001590
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.010884
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.007570
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.002681
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.006672
Train Epoch: 47 - Loss: 0.0083, Accuracy: 99.86%

Test set: Average loss: 0.0290, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.005052
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.015385
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.003711
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.002818
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.003665
Train Epoch: 48 - Loss: 0.0082, Accuracy: 99.86%

Test set: Average loss: 0.0290, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.006522
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.004543
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.009517
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.001997
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.009113
Train Epoch: 49 - Loss: 0.0081, Accuracy: 99.86%

Test set: Average loss: 0.0290, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.003675
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.005652
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.004128
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.015429
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.004367
Train Epoch: 50 - Loss: 0.0081, Accuracy: 99.86%

Test set: Average loss: 0.0290, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.015660
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.004165
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.011984
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.007404
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.021876
Train Epoch: 51 - Loss: 0.0115, Accuracy: 99.72%

Test set: Average loss: 0.0310, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.005852
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.003087
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.005809
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.035967
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.000886
Train Epoch: 52 - Loss: 0.0116, Accuracy: 99.68%

Test set: Average loss: 0.0305, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.016176
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.002501
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.005791
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.009476
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.001715
Train Epoch: 53 - Loss: 0.0102, Accuracy: 99.75%

Test set: Average loss: 0.0312, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.002272
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.006109
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.011548
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.002951
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.005986
Train Epoch: 54 - Loss: 0.0096, Accuracy: 99.76%

Test set: Average loss: 0.0342, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.020141
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.005049
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.013200
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.013336
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.002832
Train Epoch: 55 - Loss: 0.0090, Accuracy: 99.78%

Test set: Average loss: 0.0285, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.007299
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.011408
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.005274
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.005137
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.008226
Train Epoch: 56 - Loss: 0.0082, Accuracy: 99.82%

Test set: Average loss: 0.0297, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.006101
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.006105
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.003823
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.022382
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.002052
Train Epoch: 57 - Loss: 0.0075, Accuracy: 99.83%

Test set: Average loss: 0.0294, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.005826
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.003029
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.004259
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.010327
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.003884
Train Epoch: 58 - Loss: 0.0069, Accuracy: 99.86%

Test set: Average loss: 0.0315, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.001214
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.005862
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.001711
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.008744
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.006917
Train Epoch: 59 - Loss: 0.0067, Accuracy: 99.88%

Test set: Average loss: 0.0302, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.007478
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.027229
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.006750
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.002070
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.001923
Train Epoch: 60 - Loss: 0.0062, Accuracy: 99.89%

Test set: Average loss: 0.0321, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.006049
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.021997
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.007341
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.026880
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.006946
Train Epoch: 61 - Loss: 0.0058, Accuracy: 99.90%

Test set: Average loss: 0.0306, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.003991
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.007935
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.003969
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.001331
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.004361
Train Epoch: 62 - Loss: 0.0054, Accuracy: 99.91%

Test set: Average loss: 0.0304, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.004535
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.004059
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.002531
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.007314
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.002136
Train Epoch: 63 - Loss: 0.0051, Accuracy: 99.92%

Test set: Average loss: 0.0302, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.002166
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.004008
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.003796
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.007953
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.007076
Train Epoch: 64 - Loss: 0.0049, Accuracy: 99.94%

Test set: Average loss: 0.0298, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.008407
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.002215
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.006993
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.005929
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.002795
Train Epoch: 65 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0305, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.001415
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.003717
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.001032
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.003098
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.004942
Train Epoch: 66 - Loss: 0.0045, Accuracy: 99.94%

Test set: Average loss: 0.0301, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.003000
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.001785
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.005848
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.003672
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.002301
Train Epoch: 67 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0307, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.001418
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.002317
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.001459
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.002127
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.002963
Train Epoch: 68 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.001969
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.004911
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.003067
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.001938
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.003025
Train Epoch: 69 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0300, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.002100
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.002416
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.003811
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.002550
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.002844
Train Epoch: 70 - Loss: 0.0039, Accuracy: 99.97%

Test set: Average loss: 0.0302, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.001727
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.003312
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.002214
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.005278
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.002678
Train Epoch: 71 - Loss: 0.0039, Accuracy: 99.96%

Test set: Average loss: 0.0299, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.006318
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.006774
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.002025
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.004333
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.002868
Train Epoch: 72 - Loss: 0.0038, Accuracy: 99.96%

Test set: Average loss: 0.0300, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.005663
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.004908
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.001317
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.001981
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.005069
Train Epoch: 73 - Loss: 0.0038, Accuracy: 99.97%

Test set: Average loss: 0.0301, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.003957
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.023605
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.003447
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.001460
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.003565
Train Epoch: 74 - Loss: 0.0038, Accuracy: 99.97%

Test set: Average loss: 0.0301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.000312
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.003903
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.004694
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.003733
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.005435
Train Epoch: 75 - Loss: 0.0037, Accuracy: 99.97%

Test set: Average loss: 0.0301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.002166
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.008800
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.015186
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.002993
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.001945
Train Epoch: 76 - Loss: 0.0055, Accuracy: 99.90%

Test set: Average loss: 0.0305, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.000719
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.006136
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.002035
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.003099
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.011060
Train Epoch: 77 - Loss: 0.0055, Accuracy: 99.89%

Test set: Average loss: 0.0308, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.001530
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.010040
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.013161
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.011794
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.002449
Train Epoch: 78 - Loss: 0.0048, Accuracy: 99.91%

Test set: Average loss: 0.0308, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.002515
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.000862
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.005874
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.004661
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.003598
Train Epoch: 79 - Loss: 0.0047, Accuracy: 99.92%

Test set: Average loss: 0.0302, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.004029
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.004153
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.008992
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.002473
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.001222
Train Epoch: 80 - Loss: 0.0041, Accuracy: 99.94%

Test set: Average loss: 0.0329, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.002060
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.001879
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.005279
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.006804
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.002806
Train Epoch: 81 - Loss: 0.0041, Accuracy: 99.93%

Test set: Average loss: 0.0300, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.004298
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.005291
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.001935
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.002154
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.005576
Train Epoch: 82 - Loss: 0.0037, Accuracy: 99.95%

Test set: Average loss: 0.0324, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.001332
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.002147
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.004894
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.001535
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.006194
Train Epoch: 83 - Loss: 0.0034, Accuracy: 99.96%

Test set: Average loss: 0.0317, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.001189
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.002313
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.000840
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.000930
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.000302
Train Epoch: 84 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0327, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.001876
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.003279
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.002496
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.006116
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.003419
Train Epoch: 85 - Loss: 0.0032, Accuracy: 99.96%

Test set: Average loss: 0.0302, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.000930
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.001287
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.003266
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.002314
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.005113
Train Epoch: 86 - Loss: 0.0029, Accuracy: 99.97%

Test set: Average loss: 0.0317, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.001310
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.002230
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001266
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001831
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.004123
Train Epoch: 87 - Loss: 0.0028, Accuracy: 99.97%

Test set: Average loss: 0.0316, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.008109
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.002472
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.001579
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.011030
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.001109
Train Epoch: 88 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0321, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.001381
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.003257
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.004747
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.003740
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.002344
Train Epoch: 89 - Loss: 0.0025, Accuracy: 99.98%

Test set: Average loss: 0.0312, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.002847
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.000442
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.000294
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001123
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.000985
Train Epoch: 90 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0317, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.002236
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.003252
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.001196
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.001048
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.001782
Train Epoch: 91 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0317, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.003069
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.001362
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001522
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.000904
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.001216
Train Epoch: 92 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0312, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.002765
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.001343
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.000538
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001672
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.001966
Train Epoch: 93 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0319, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.001852
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.000762
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.001175
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.000807
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.001993
Train Epoch: 94 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0313, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.001781
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.002126
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.005090
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.000884
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001455
Train Epoch: 95 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0313, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.003701
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.003086
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.001050
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001976
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.003376
Train Epoch: 96 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0313, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.004529
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.001251
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.001320
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.002099
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.003463
Train Epoch: 97 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0314, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.002121
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.000896
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.003777
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.001071
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.001359
Train Epoch: 98 - Loss: 0.0021, Accuracy: 99.98%

Test set: Average loss: 0.0314, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.000796
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.006245
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.001369
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.000668
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.000728
Train Epoch: 99 - Loss: 0.0020, Accuracy: 99.98%

Test set: Average loss: 0.0314, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.000908
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.000775
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.001340
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001735
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.000988
Train Epoch: 100 - Loss: 0.0020, Accuracy: 99.98%

Test set: Average loss: 0.0314, Accuracy: 9900/10000 (99.00%)


Training completed. Best Accuracy: 99.07%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.306771
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.884936
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.388482
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.320989
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.194233
Train Epoch: 1 - Loss: 0.7278, Accuracy: 80.19%

Test set: Average loss: 0.2497, Accuracy: 9240/10000 (92.40%)

New best model saved with accuracy: 92.40%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.201568
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.230112
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.231215
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.208246
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.324603
Train Epoch: 2 - Loss: 0.1983, Accuracy: 94.11%

Test set: Average loss: 0.1511, Accuracy: 9548/10000 (95.48%)

New best model saved with accuracy: 95.48%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.115579
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.104763
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.097354
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.079015
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.135534
Train Epoch: 3 - Loss: 0.1277, Accuracy: 96.18%

Test set: Average loss: 0.1013, Accuracy: 9689/10000 (96.89%)

New best model saved with accuracy: 96.89%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.142494
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.068616
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.102659
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.075829
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.149038
Train Epoch: 4 - Loss: 0.0965, Accuracy: 97.17%

Test set: Average loss: 0.0933, Accuracy: 9711/10000 (97.11%)

New best model saved with accuracy: 97.11%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.080177
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.156404
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.083574
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.093822
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.018348
Train Epoch: 5 - Loss: 0.0781, Accuracy: 97.64%

Test set: Average loss: 0.0715, Accuracy: 9765/10000 (97.65%)

New best model saved with accuracy: 97.65%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.095463
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.044367
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.048684
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.076532
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.017738
Train Epoch: 6 - Loss: 0.0667, Accuracy: 97.98%

Test set: Average loss: 0.0644, Accuracy: 9798/10000 (97.98%)

New best model saved with accuracy: 97.98%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.055326
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.089422
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.056502
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.095103
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.021441
Train Epoch: 7 - Loss: 0.0580, Accuracy: 98.25%

Test set: Average loss: 0.0666, Accuracy: 9788/10000 (97.88%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.012755
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.093847
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.064995
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.025264
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.033259
Train Epoch: 8 - Loss: 0.0526, Accuracy: 98.38%

Test set: Average loss: 0.0564, Accuracy: 9808/10000 (98.08%)

New best model saved with accuracy: 98.08%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.051119
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.029838
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.039668
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.031779
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.060322
Train Epoch: 9 - Loss: 0.0470, Accuracy: 98.60%

Test set: Average loss: 0.0510, Accuracy: 9839/10000 (98.39%)

New best model saved with accuracy: 98.39%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.026354
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.051569
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.052215
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.074381
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.025279
Train Epoch: 10 - Loss: 0.0429, Accuracy: 98.72%

Test set: Average loss: 0.0491, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.034501
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.032988
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.015980
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.010485
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.031252
Train Epoch: 11 - Loss: 0.0397, Accuracy: 98.83%

Test set: Average loss: 0.0491, Accuracy: 9829/10000 (98.29%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.039683
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.022575
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.021081
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.062542
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.078288
Train Epoch: 12 - Loss: 0.0370, Accuracy: 98.91%

Test set: Average loss: 0.0441, Accuracy: 9856/10000 (98.56%)

New best model saved with accuracy: 98.56%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.011257
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.021673
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.020512
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.014973
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.035281
Train Epoch: 13 - Loss: 0.0347, Accuracy: 98.98%

Test set: Average loss: 0.0427, Accuracy: 9857/10000 (98.57%)

New best model saved with accuracy: 98.57%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.042138
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.012058
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.009043
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.019145
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.017029
Train Epoch: 14 - Loss: 0.0326, Accuracy: 99.07%

Test set: Average loss: 0.0418, Accuracy: 9860/10000 (98.60%)

New best model saved with accuracy: 98.60%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.015637
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.016688
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.023149
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.010855
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.053283
Train Epoch: 15 - Loss: 0.0308, Accuracy: 99.12%

Test set: Average loss: 0.0406, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.018531
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.017775
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.029432
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.075747
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.116076
Train Epoch: 16 - Loss: 0.0293, Accuracy: 99.18%

Test set: Average loss: 0.0394, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.019851
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.012296
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.014333
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.023415
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.029475
Train Epoch: 17 - Loss: 0.0279, Accuracy: 99.21%

Test set: Average loss: 0.0389, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.040358
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.078304
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.027791
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.022321
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.012339
Train Epoch: 18 - Loss: 0.0268, Accuracy: 99.28%

Test set: Average loss: 0.0388, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.007968
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.092198
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.019179
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.013607
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.026500
Train Epoch: 19 - Loss: 0.0261, Accuracy: 99.28%

Test set: Average loss: 0.0385, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.016329
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.008871
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.027603
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.018890
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.013906
Train Epoch: 20 - Loss: 0.0254, Accuracy: 99.30%

Test set: Average loss: 0.0390, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.015242
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.020820
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.013345
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.007763
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.008766
Train Epoch: 21 - Loss: 0.0248, Accuracy: 99.33%

Test set: Average loss: 0.0382, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.048046
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.010392
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.045227
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.079611
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.028180
Train Epoch: 22 - Loss: 0.0244, Accuracy: 99.35%

Test set: Average loss: 0.0382, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.013434
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.006683
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.010188
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.050431
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.016041
Train Epoch: 23 - Loss: 0.0241, Accuracy: 99.36%

Test set: Average loss: 0.0381, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.025335
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.059378
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.057028
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.015114
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.005886
Train Epoch: 24 - Loss: 0.0239, Accuracy: 99.36%

Test set: Average loss: 0.0381, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.018223
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.044153
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.010228
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.035158
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.030456
Train Epoch: 25 - Loss: 0.0238, Accuracy: 99.38%

Test set: Average loss: 0.0380, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.024304
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.025708
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.011446
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.021494
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.030925
Train Epoch: 26 - Loss: 0.0307, Accuracy: 99.11%

Test set: Average loss: 0.0414, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.007548
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.017129
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.014251
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.009691
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.010833
Train Epoch: 27 - Loss: 0.0289, Accuracy: 99.13%

Test set: Average loss: 0.0432, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.049557
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.021375
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.028900
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.006489
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.022939
Train Epoch: 28 - Loss: 0.0267, Accuracy: 99.18%

Test set: Average loss: 0.0401, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.028919
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.023198
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.035357
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.021896
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.034786
Train Epoch: 29 - Loss: 0.0247, Accuracy: 99.27%

Test set: Average loss: 0.0412, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.029868
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.006234
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.004887
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.005226
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.056749
Train Epoch: 30 - Loss: 0.0226, Accuracy: 99.34%

Test set: Average loss: 0.0378, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.018068
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.015423
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.010876
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.018192
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.005597
Train Epoch: 31 - Loss: 0.0206, Accuracy: 99.45%

Test set: Average loss: 0.0393, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.004797
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.007950
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.034365
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.002360
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.018564
Train Epoch: 32 - Loss: 0.0196, Accuracy: 99.44%

Test set: Average loss: 0.0360, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.024215
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.007424
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.007738
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.002868
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.007577
Train Epoch: 33 - Loss: 0.0178, Accuracy: 99.51%

Test set: Average loss: 0.0340, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.034429
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.014400
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.005579
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.007190
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.011819
Train Epoch: 34 - Loss: 0.0166, Accuracy: 99.57%

Test set: Average loss: 0.0348, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.004838
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.007589
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.019874
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.002987
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.007824
Train Epoch: 35 - Loss: 0.0156, Accuracy: 99.60%

Test set: Average loss: 0.0358, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.020567
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.006096
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.009328
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.004844
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.045297
Train Epoch: 36 - Loss: 0.0144, Accuracy: 99.63%

Test set: Average loss: 0.0349, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.008568
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.011188
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.024251
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.013496
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.015680
Train Epoch: 37 - Loss: 0.0133, Accuracy: 99.69%

Test set: Average loss: 0.0340, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.009795
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.026386
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.001784
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.012645
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.016574
Train Epoch: 38 - Loss: 0.0125, Accuracy: 99.71%

Test set: Average loss: 0.0346, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.009361
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.007092
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.004033
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.011478
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.006812
Train Epoch: 39 - Loss: 0.0119, Accuracy: 99.74%

Test set: Average loss: 0.0328, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.002104
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.005459
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.003416
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.002868
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.001897
Train Epoch: 40 - Loss: 0.0113, Accuracy: 99.77%

Test set: Average loss: 0.0329, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.034174
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.004135
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.004372
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.002723
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.014189
Train Epoch: 41 - Loss: 0.0107, Accuracy: 99.78%

Test set: Average loss: 0.0329, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.011338
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.009417
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.007544
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.012064
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.003629
Train Epoch: 42 - Loss: 0.0103, Accuracy: 99.79%

Test set: Average loss: 0.0337, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.001412
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.007093
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.005167
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.002302
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.005711
Train Epoch: 43 - Loss: 0.0099, Accuracy: 99.80%

Test set: Average loss: 0.0348, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.010516
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.012238
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.005039
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.002300
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.007428
Train Epoch: 44 - Loss: 0.0097, Accuracy: 99.83%

Test set: Average loss: 0.0336, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.023659
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.010323
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.019223
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.001775
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.007470
Train Epoch: 45 - Loss: 0.0094, Accuracy: 99.83%

Test set: Average loss: 0.0335, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.006709
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.027182
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.003341
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.003953
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.004509
Train Epoch: 46 - Loss: 0.0092, Accuracy: 99.83%

Test set: Average loss: 0.0330, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.003432
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.006289
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.003672
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.004686
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.002738
Train Epoch: 47 - Loss: 0.0090, Accuracy: 99.84%

Test set: Average loss: 0.0336, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.004123
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.006148
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.017041
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.004236
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.004500
Train Epoch: 48 - Loss: 0.0089, Accuracy: 99.85%

Test set: Average loss: 0.0332, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.006847
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.001369
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.007120
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.003755
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.005936
Train Epoch: 49 - Loss: 0.0089, Accuracy: 99.85%

Test set: Average loss: 0.0332, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.012020
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.001533
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.015748
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.008715
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.004544
Train Epoch: 50 - Loss: 0.0088, Accuracy: 99.84%

Test set: Average loss: 0.0332, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.002921
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.009914
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.006548
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.002490
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.006083
Train Epoch: 51 - Loss: 0.0125, Accuracy: 99.70%

Test set: Average loss: 0.0356, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.007523
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.009071
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.032256
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.002813
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.003709
Train Epoch: 52 - Loss: 0.0121, Accuracy: 99.70%

Test set: Average loss: 0.0339, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.003825
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.013897
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.003758
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.013890
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.002539
Train Epoch: 53 - Loss: 0.0113, Accuracy: 99.73%

Test set: Average loss: 0.0362, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.009226
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.007046
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.015063
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.006505
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.030386
Train Epoch: 54 - Loss: 0.0105, Accuracy: 99.76%

Test set: Average loss: 0.0351, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.004112
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.010683
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.008247
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.007178
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.009487
Train Epoch: 55 - Loss: 0.0097, Accuracy: 99.80%

Test set: Average loss: 0.0428, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.004744
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.004722
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.038754
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.006860
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.006796
Train Epoch: 56 - Loss: 0.0090, Accuracy: 99.81%

Test set: Average loss: 0.0359, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.003558
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.012887
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.007437
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.004001
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.011649
Train Epoch: 57 - Loss: 0.0084, Accuracy: 99.83%

Test set: Average loss: 0.0346, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.007080
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.005601
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.006321
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.008778
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.000682
Train Epoch: 58 - Loss: 0.0078, Accuracy: 99.84%

Test set: Average loss: 0.0352, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.011092
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.008226
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.002675
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.002453
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.002906
Train Epoch: 59 - Loss: 0.0071, Accuracy: 99.87%

Test set: Average loss: 0.0330, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.002825
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.004348
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.008483
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.002925
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.007786
Train Epoch: 60 - Loss: 0.0068, Accuracy: 99.87%

Test set: Average loss: 0.0349, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.001154
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.001128
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.008745
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.001500
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.005628
Train Epoch: 61 - Loss: 0.0063, Accuracy: 99.90%

Test set: Average loss: 0.0344, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.003427
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.001202
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.002252
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.006891
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.000658
Train Epoch: 62 - Loss: 0.0059, Accuracy: 99.91%

Test set: Average loss: 0.0338, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.009762
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.004560
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.003727
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.005519
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.001477
Train Epoch: 63 - Loss: 0.0055, Accuracy: 99.91%

Test set: Average loss: 0.0341, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.004750
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.001421
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.004720
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.002724
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.005428
Train Epoch: 64 - Loss: 0.0052, Accuracy: 99.93%

Test set: Average loss: 0.0368, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.002623
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.001953
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.005169
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.002490
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.017089
Train Epoch: 65 - Loss: 0.0050, Accuracy: 99.94%

Test set: Average loss: 0.0342, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.006717
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.002498
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.005321
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.001733
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.003579
Train Epoch: 66 - Loss: 0.0048, Accuracy: 99.94%

Test set: Average loss: 0.0342, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.000974
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.005881
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.003480
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.003395
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.006523
Train Epoch: 67 - Loss: 0.0047, Accuracy: 99.94%

Test set: Average loss: 0.0345, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.002450
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.005229
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.001555
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.002417
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.011899
Train Epoch: 68 - Loss: 0.0045, Accuracy: 99.95%

Test set: Average loss: 0.0344, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.000858
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.002702
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.001438
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.001617
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.003066
Train Epoch: 69 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0345, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.008181
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.002533
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.019432
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.002846
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.001316
Train Epoch: 70 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0344, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.002750
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.002531
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.005263
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.002673
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.003866
Train Epoch: 71 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0344, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.001051
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.003006
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.001633
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.060433
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.001578
Train Epoch: 72 - Loss: 0.0042, Accuracy: 99.95%

Test set: Average loss: 0.0341, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.003167
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.001593
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.003769
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.013477
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.003385
Train Epoch: 73 - Loss: 0.0041, Accuracy: 99.95%

Test set: Average loss: 0.0342, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.002168
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002371
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.003590
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.004008
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.001143
Train Epoch: 74 - Loss: 0.0041, Accuracy: 99.95%

Test set: Average loss: 0.0342, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.002448
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.005353
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.005660
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.001934
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.004153
Train Epoch: 75 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0342, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.005335
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.001754
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.004507
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.001490
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.001745
Train Epoch: 76 - Loss: 0.0060, Accuracy: 99.87%

Test set: Average loss: 0.0409, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.006716
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.001478
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.004386
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.002976
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.040021
Train Epoch: 77 - Loss: 0.0057, Accuracy: 99.89%

Test set: Average loss: 0.0393, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.002177
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.002750
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.001966
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.000424
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.001527
Train Epoch: 78 - Loss: 0.0053, Accuracy: 99.92%

Test set: Average loss: 0.0376, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.002714
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.005029
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.005182
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.002716
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.001555
Train Epoch: 79 - Loss: 0.0050, Accuracy: 99.93%

Test set: Average loss: 0.0428, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.009133
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.005860
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.002064
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.001994
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.005796
Train Epoch: 80 - Loss: 0.0046, Accuracy: 99.93%

Test set: Average loss: 0.0355, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.002099
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.001424
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.003732
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.001231
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.006704
Train Epoch: 81 - Loss: 0.0042, Accuracy: 99.93%

Test set: Average loss: 0.0360, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.003578
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002185
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.002639
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.002773
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.011027
Train Epoch: 82 - Loss: 0.0041, Accuracy: 99.94%

Test set: Average loss: 0.0368, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.001731
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.002801
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.002283
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.005298
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.003607
Train Epoch: 83 - Loss: 0.0037, Accuracy: 99.95%

Test set: Average loss: 0.0380, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.004426
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.005225
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.002396
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.002190
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.001888
Train Epoch: 84 - Loss: 0.0035, Accuracy: 99.96%

Test set: Average loss: 0.0358, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.015460
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.002231
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.002000
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.002131
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.001947
Train Epoch: 85 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0350, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.000798
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.001926
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.001420
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.001948
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.001536
Train Epoch: 86 - Loss: 0.0031, Accuracy: 99.96%

Test set: Average loss: 0.0364, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.014037
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.001131
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001486
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001344
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.001237
Train Epoch: 87 - Loss: 0.0030, Accuracy: 99.97%

Test set: Average loss: 0.0362, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.001207
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.001239
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.000820
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.001176
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.003228
Train Epoch: 88 - Loss: 0.0028, Accuracy: 99.97%

Test set: Average loss: 0.0367, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.002403
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.002067
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.002870
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.002887
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.002806
Train Epoch: 89 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0360, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.000941
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.001865
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.003794
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001723
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.000698
Train Epoch: 90 - Loss: 0.0026, Accuracy: 99.97%

Test set: Average loss: 0.0366, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.002010
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.003006
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.001338
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.001446
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.002070
Train Epoch: 91 - Loss: 0.0025, Accuracy: 99.97%

Test set: Average loss: 0.0369, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.001243
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.000621
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001966
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.002426
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.003669
Train Epoch: 92 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0364, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.003401
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.001860
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.001398
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001346
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.001029
Train Epoch: 93 - Loss: 0.0024, Accuracy: 99.97%

Test set: Average loss: 0.0363, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.000206
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.000569
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.005014
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.001001
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.006005
Train Epoch: 94 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0359, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.000789
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.001666
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.002612
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001150
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001679
Train Epoch: 95 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0360, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001978
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.000781
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.000837
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.003860
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.001160
Train Epoch: 96 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0360, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.000743
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.000770
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.000759
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.003291
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.001240
Train Epoch: 97 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0361, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.000477
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.002792
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.000948
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.002802
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.001541
Train Epoch: 98 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0361, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.001165
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.002288
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.006795
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.002563
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.002638
Train Epoch: 99 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0361, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.001308
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.001561
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.001713
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001779
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.002598
Train Epoch: 100 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0361, Accuracy: 9882/10000 (98.82%)


Training completed. Best Accuracy: 98.93%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.304430
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.913794
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.492770
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.255889
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.316565
Train Epoch: 1 - Loss: 0.7180, Accuracy: 78.81%

Test set: Average loss: 0.2496, Accuracy: 9248/10000 (92.48%)

New best model saved with accuracy: 92.48%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.251601
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.216473
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.224763
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.201692
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.111086
Train Epoch: 2 - Loss: 0.1945, Accuracy: 94.21%

Test set: Average loss: 0.1393, Accuracy: 9578/10000 (95.78%)

New best model saved with accuracy: 95.78%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.130545
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.164804
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.129753
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.065474
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.092491
Train Epoch: 3 - Loss: 0.1245, Accuracy: 96.25%

Test set: Average loss: 0.1037, Accuracy: 9688/10000 (96.88%)

New best model saved with accuracy: 96.88%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.060477
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.058550
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.091214
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.058216
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.192445
Train Epoch: 4 - Loss: 0.0929, Accuracy: 97.22%

Test set: Average loss: 0.0733, Accuracy: 9766/10000 (97.66%)

New best model saved with accuracy: 97.66%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.036054
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.042448
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.134476
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.066281
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.092652
Train Epoch: 5 - Loss: 0.0751, Accuracy: 97.75%

Test set: Average loss: 0.0607, Accuracy: 9805/10000 (98.05%)

New best model saved with accuracy: 98.05%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.056860
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.089913
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.052462
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.045095
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.043331
Train Epoch: 6 - Loss: 0.0639, Accuracy: 98.05%

Test set: Average loss: 0.0557, Accuracy: 9830/10000 (98.30%)

New best model saved with accuracy: 98.30%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.116824
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.052241
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.033749
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.024464
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.039135
Train Epoch: 7 - Loss: 0.0560, Accuracy: 98.32%

Test set: Average loss: 0.0570, Accuracy: 9807/10000 (98.07%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.027340
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.053826
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.065053
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.017516
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.056154
Train Epoch: 8 - Loss: 0.0498, Accuracy: 98.56%

Test set: Average loss: 0.0482, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.019786
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.040480
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.030030
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.054802
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.056461
Train Epoch: 9 - Loss: 0.0455, Accuracy: 98.64%

Test set: Average loss: 0.0450, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.057602
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.029640
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.042532
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.014819
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.036717
Train Epoch: 10 - Loss: 0.0418, Accuracy: 98.73%

Test set: Average loss: 0.0430, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.033113
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.004000
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.020089
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.027827
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.009100
Train Epoch: 11 - Loss: 0.0382, Accuracy: 98.86%

Test set: Average loss: 0.0416, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.076694
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.023347
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.032361
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.068971
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.024503
Train Epoch: 12 - Loss: 0.0353, Accuracy: 98.95%

Test set: Average loss: 0.0394, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.012551
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.063152
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.026959
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.022190
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.033270
Train Epoch: 13 - Loss: 0.0331, Accuracy: 99.03%

Test set: Average loss: 0.0388, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.027082
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.036632
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.024670
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.008620
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.027768
Train Epoch: 14 - Loss: 0.0313, Accuracy: 99.07%

Test set: Average loss: 0.0384, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.038140
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.013779
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.015567
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.009958
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.006363
Train Epoch: 15 - Loss: 0.0297, Accuracy: 99.13%

Test set: Average loss: 0.0389, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.022120
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.029588
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.089062
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.005508
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.017865
Train Epoch: 16 - Loss: 0.0282, Accuracy: 99.17%

Test set: Average loss: 0.0371, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.018043
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.059398
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.075991
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.005020
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.045011
Train Epoch: 17 - Loss: 0.0271, Accuracy: 99.25%

Test set: Average loss: 0.0367, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.022768
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.016952
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.036780
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.025349
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.029437
Train Epoch: 18 - Loss: 0.0261, Accuracy: 99.28%

Test set: Average loss: 0.0367, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.015478
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.029378
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.015542
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.075849
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.025691
Train Epoch: 19 - Loss: 0.0254, Accuracy: 99.29%

Test set: Average loss: 0.0363, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.005805
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.014758
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.023475
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.021256
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.022546
Train Epoch: 20 - Loss: 0.0247, Accuracy: 99.34%

Test set: Average loss: 0.0360, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.045888
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.015912
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.037742
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.024026
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.031826
Train Epoch: 21 - Loss: 0.0242, Accuracy: 99.34%

Test set: Average loss: 0.0363, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.016787
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.026707
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.019392
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.081823
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.017859
Train Epoch: 22 - Loss: 0.0238, Accuracy: 99.35%

Test set: Average loss: 0.0354, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.067939
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.052656
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.013176
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.019600
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.008272
Train Epoch: 23 - Loss: 0.0235, Accuracy: 99.36%

Test set: Average loss: 0.0354, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.026599
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.028658
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.007813
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.006631
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.019600
Train Epoch: 24 - Loss: 0.0233, Accuracy: 99.38%

Test set: Average loss: 0.0354, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.032807
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.034942
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.018874
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.021118
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.010061
Train Epoch: 25 - Loss: 0.0231, Accuracy: 99.38%

Test set: Average loss: 0.0354, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.008501
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.034318
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.042868
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.048500
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.028646
Train Epoch: 26 - Loss: 0.0299, Accuracy: 99.09%

Test set: Average loss: 0.0386, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.036227
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.046130
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.052773
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.087442
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.007662
Train Epoch: 27 - Loss: 0.0273, Accuracy: 99.16%

Test set: Average loss: 0.0441, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.040461
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.022583
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.009094
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.004783
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.071905
Train Epoch: 28 - Loss: 0.0261, Accuracy: 99.22%

Test set: Average loss: 0.0371, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.019919
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.023558
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.016463
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.024413
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.009559
Train Epoch: 29 - Loss: 0.0241, Accuracy: 99.29%

Test set: Average loss: 0.0429, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.079606
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.023228
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.018219
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.012157
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.020336
Train Epoch: 30 - Loss: 0.0219, Accuracy: 99.38%

Test set: Average loss: 0.0359, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.014878
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.022386
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.014094
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.008319
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.012020
Train Epoch: 31 - Loss: 0.0197, Accuracy: 99.42%

Test set: Average loss: 0.0325, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.007357
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.014701
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.007742
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.029790
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.014558
Train Epoch: 32 - Loss: 0.0186, Accuracy: 99.44%

Test set: Average loss: 0.0347, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.007055
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.115459
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.035663
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.006780
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.048380
Train Epoch: 33 - Loss: 0.0169, Accuracy: 99.53%

Test set: Average loss: 0.0326, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.023791
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.029973
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.008427
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.004786
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.012886
Train Epoch: 34 - Loss: 0.0158, Accuracy: 99.58%

Test set: Average loss: 0.0352, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.011053
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.010121
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.011768
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.007870
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.053976
Train Epoch: 35 - Loss: 0.0150, Accuracy: 99.59%

Test set: Average loss: 0.0350, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.012979
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.003970
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.005810
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.013849
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.005508
Train Epoch: 36 - Loss: 0.0140, Accuracy: 99.63%

Test set: Average loss: 0.0348, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.005701
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.002771
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.006306
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.006309
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.014104
Train Epoch: 37 - Loss: 0.0129, Accuracy: 99.66%

Test set: Average loss: 0.0327, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.005255
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.009286
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.015087
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.001961
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.006046
Train Epoch: 38 - Loss: 0.0123, Accuracy: 99.71%

Test set: Average loss: 0.0335, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.002663
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.006822
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.004517
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.013886
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.005633
Train Epoch: 39 - Loss: 0.0116, Accuracy: 99.73%

Test set: Average loss: 0.0344, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.006700
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.004492
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.007427
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.014867
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.008030
Train Epoch: 40 - Loss: 0.0109, Accuracy: 99.75%

Test set: Average loss: 0.0338, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.014801
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.005063
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.004812
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.006402
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.002837
Train Epoch: 41 - Loss: 0.0104, Accuracy: 99.77%

Test set: Average loss: 0.0333, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.009237
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.004250
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.011519
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.012645
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.003442
Train Epoch: 42 - Loss: 0.0100, Accuracy: 99.79%

Test set: Average loss: 0.0342, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.005108
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.004344
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.021114
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.003846
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.028124
Train Epoch: 43 - Loss: 0.0097, Accuracy: 99.81%

Test set: Average loss: 0.0332, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.003705
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.005065
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.008877
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.007855
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.012845
Train Epoch: 44 - Loss: 0.0094, Accuracy: 99.81%

Test set: Average loss: 0.0329, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.001911
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.047407
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.013872
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.010138
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.005997
Train Epoch: 45 - Loss: 0.0092, Accuracy: 99.83%

Test set: Average loss: 0.0333, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.003508
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.002324
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.002660
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.003695
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.018148
Train Epoch: 46 - Loss: 0.0090, Accuracy: 99.83%

Test set: Average loss: 0.0333, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.001816
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.008547
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.003220
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.011513
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.004953
Train Epoch: 47 - Loss: 0.0088, Accuracy: 99.83%

Test set: Average loss: 0.0332, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.000980
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.021560
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.022843
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.015102
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.004054
Train Epoch: 48 - Loss: 0.0087, Accuracy: 99.84%

Test set: Average loss: 0.0332, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.009148
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.013581
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.009088
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.009353
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.002896
Train Epoch: 49 - Loss: 0.0086, Accuracy: 99.84%

Test set: Average loss: 0.0331, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.006144
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.009478
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.005872
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.003841
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.003962
Train Epoch: 50 - Loss: 0.0086, Accuracy: 99.84%

Test set: Average loss: 0.0332, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.008079
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.014819
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.013012
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.005799
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.011946
Train Epoch: 51 - Loss: 0.0119, Accuracy: 99.67%

Test set: Average loss: 0.0357, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.058659
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.004332
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.004854
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.007604
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.004947
Train Epoch: 52 - Loss: 0.0117, Accuracy: 99.70%

Test set: Average loss: 0.0361, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.002066
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.011910
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.047097
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.010352
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.015271
Train Epoch: 53 - Loss: 0.0107, Accuracy: 99.72%

Test set: Average loss: 0.0358, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.004133
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.003244
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.005892
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.007484
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.023047
Train Epoch: 54 - Loss: 0.0098, Accuracy: 99.78%

Test set: Average loss: 0.0337, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.004737
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.001929
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.006525
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.008302
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.002103
Train Epoch: 55 - Loss: 0.0094, Accuracy: 99.78%

Test set: Average loss: 0.0358, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.008268
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.013389
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.008132
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.028424
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.002518
Train Epoch: 56 - Loss: 0.0087, Accuracy: 99.80%

Test set: Average loss: 0.0345, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.011197
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.001432
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.006798
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.001376
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.008141
Train Epoch: 57 - Loss: 0.0079, Accuracy: 99.85%

Test set: Average loss: 0.0352, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.006380
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.003818
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.002316
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.008897
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.006154
Train Epoch: 58 - Loss: 0.0075, Accuracy: 99.84%

Test set: Average loss: 0.0346, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.015236
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.001450
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.002151
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.014261
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.013938
Train Epoch: 59 - Loss: 0.0070, Accuracy: 99.86%

Test set: Average loss: 0.0368, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.001982
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.001287
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.002016
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.004318
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.004234
Train Epoch: 60 - Loss: 0.0065, Accuracy: 99.87%

Test set: Average loss: 0.0352, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.001752
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.003212
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.009196
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.026988
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.011819
Train Epoch: 61 - Loss: 0.0062, Accuracy: 99.89%

Test set: Average loss: 0.0353, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.002167
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.001718
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.008474
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.002188
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.002647
Train Epoch: 62 - Loss: 0.0058, Accuracy: 99.90%

Test set: Average loss: 0.0347, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.004043
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.002204
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.002064
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.021682
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.011087
Train Epoch: 63 - Loss: 0.0055, Accuracy: 99.91%

Test set: Average loss: 0.0351, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.002435
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.007048
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.023270
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.003948
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.001756
Train Epoch: 64 - Loss: 0.0052, Accuracy: 99.93%

Test set: Average loss: 0.0359, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.005845
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.002296
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.003887
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.001519
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.002856
Train Epoch: 65 - Loss: 0.0050, Accuracy: 99.93%

Test set: Average loss: 0.0357, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.001463
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.000826
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.004570
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.004238
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.005894
Train Epoch: 66 - Loss: 0.0048, Accuracy: 99.93%

Test set: Average loss: 0.0351, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.015398
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.001959
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.002272
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.003292
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.004343
Train Epoch: 67 - Loss: 0.0046, Accuracy: 99.95%

Test set: Average loss: 0.0352, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.002871
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.007196
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.004819
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.002194
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.000626
Train Epoch: 68 - Loss: 0.0044, Accuracy: 99.95%

Test set: Average loss: 0.0350, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.001423
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.002634
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.003561
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.010674
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.004417
Train Epoch: 69 - Loss: 0.0043, Accuracy: 99.95%

Test set: Average loss: 0.0354, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.010547
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.002423
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.005881
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.002704
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.033198
Train Epoch: 70 - Loss: 0.0042, Accuracy: 99.96%

Test set: Average loss: 0.0358, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.006834
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.001221
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.006479
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.003045
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.007374
Train Epoch: 71 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0354, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.004178
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.005233
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.003219
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.007650
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.002372
Train Epoch: 72 - Loss: 0.0041, Accuracy: 99.96%

Test set: Average loss: 0.0352, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.001765
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.005859
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.002237
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.003514
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.001272
Train Epoch: 73 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0353, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.003510
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.002286
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.003149
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.000808
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.008016
Train Epoch: 74 - Loss: 0.0040, Accuracy: 99.97%

Test set: Average loss: 0.0353, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.005076
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.002165
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.004820
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.008986
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.007569
Train Epoch: 75 - Loss: 0.0040, Accuracy: 99.96%

Test set: Average loss: 0.0353, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.002306
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.002966
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.003104
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.004034
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.003758
Train Epoch: 76 - Loss: 0.0056, Accuracy: 99.90%

Test set: Average loss: 0.0359, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.011703
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.001764
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.036736
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.003616
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.001096
Train Epoch: 77 - Loss: 0.0053, Accuracy: 99.91%

Test set: Average loss: 0.0376, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.013572
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.002455
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.000844
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.001510
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.003612
Train Epoch: 78 - Loss: 0.0051, Accuracy: 99.90%

Test set: Average loss: 0.0369, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.001591
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.001272
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.003729
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.011834
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.002870
Train Epoch: 79 - Loss: 0.0050, Accuracy: 99.91%

Test set: Average loss: 0.0367, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.003641
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.001260
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.000681
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.001755
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.002662
Train Epoch: 80 - Loss: 0.0044, Accuracy: 99.94%

Test set: Average loss: 0.0378, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.001369
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.003816
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.008692
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.008148
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.000939
Train Epoch: 81 - Loss: 0.0042, Accuracy: 99.93%

Test set: Average loss: 0.0364, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.001227
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.002867
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.001787
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.002848
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.005925
Train Epoch: 82 - Loss: 0.0039, Accuracy: 99.94%

Test set: Average loss: 0.0395, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.005451
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.002508
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.004812
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.002504
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.002908
Train Epoch: 83 - Loss: 0.0036, Accuracy: 99.96%

Test set: Average loss: 0.0365, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.001625
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.001835
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.004622
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.007427
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.004445
Train Epoch: 84 - Loss: 0.0035, Accuracy: 99.95%

Test set: Average loss: 0.0368, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.001775
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.004317
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.002267
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.001789
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.003335
Train Epoch: 85 - Loss: 0.0033, Accuracy: 99.96%

Test set: Average loss: 0.0371, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.000838
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.000828
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.007093
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.015004
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.001335
Train Epoch: 86 - Loss: 0.0031, Accuracy: 99.97%

Test set: Average loss: 0.0374, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.001437
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.002759
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.001023
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.001606
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.001439
Train Epoch: 87 - Loss: 0.0030, Accuracy: 99.96%

Test set: Average loss: 0.0389, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.006049
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.000672
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.001537
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.001208
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.006308
Train Epoch: 88 - Loss: 0.0028, Accuracy: 99.98%

Test set: Average loss: 0.0378, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.006353
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.001674
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.000622
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.001242
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.001162
Train Epoch: 89 - Loss: 0.0027, Accuracy: 99.97%

Test set: Average loss: 0.0379, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.001845
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.001624
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.000574
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.001070
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.002266
Train Epoch: 90 - Loss: 0.0025, Accuracy: 99.97%

Test set: Average loss: 0.0376, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.000494
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.001745
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.001077
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.001806
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.001955
Train Epoch: 91 - Loss: 0.0025, Accuracy: 99.98%

Test set: Average loss: 0.0373, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.002670
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.002469
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.001708
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.018239
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.000474
Train Epoch: 92 - Loss: 0.0024, Accuracy: 99.98%

Test set: Average loss: 0.0379, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.001779
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.002589
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.003578
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.001464
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.003492
Train Epoch: 93 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0374, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.000352
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.003707
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.001392
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.001202
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.011166
Train Epoch: 94 - Loss: 0.0023, Accuracy: 99.98%

Test set: Average loss: 0.0376, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.000369
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.002142
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.001657
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.001304
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.001298
Train Epoch: 95 - Loss: 0.0022, Accuracy: 99.98%

Test set: Average loss: 0.0373, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.001827
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.002204
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.003510
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.001313
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.002186
Train Epoch: 96 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0375, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.001542
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.001732
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.000675
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.003164
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.003512
Train Epoch: 97 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0375, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.000779
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.002898
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.001188
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.002244
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.001635
Train Epoch: 98 - Loss: 0.0022, Accuracy: 99.99%

Test set: Average loss: 0.0374, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.000774
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.002404
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.001170
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.001492
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.000878
Train Epoch: 99 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0374, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.001334
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.001378
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.001055
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.001371
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.002321
Train Epoch: 100 - Loss: 0.0021, Accuracy: 99.99%

Test set: Average loss: 0.0374, Accuracy: 9887/10000 (98.87%)


Training completed. Best Accuracy: 98.91%
