Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.307248
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.286114
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.075636
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.958290
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.732100
Train Epoch: 1 - Loss: 1.3709, Accuracy: 62.80%

Test set: Average loss: 0.7534, Accuracy: 9208/10000 (92.08%)

New best model saved with accuracy: 92.08%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.734364
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.710779
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.752948
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.730753
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.666385
Train Epoch: 2 - Loss: 0.7028, Accuracy: 93.85%

Test set: Average loss: 0.6484, Accuracy: 9560/10000 (95.60%)

New best model saved with accuracy: 95.60%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.620004
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.647449
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.683504
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.623782
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.649110
Train Epoch: 3 - Loss: 0.6314, Accuracy: 96.26%

Test set: Average loss: 0.6057, Accuracy: 9705/10000 (97.05%)

New best model saved with accuracy: 97.05%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.621510
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.633617
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.590717
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.572003
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.617336
Train Epoch: 4 - Loss: 0.6016, Accuracy: 97.15%

Test set: Average loss: 0.5870, Accuracy: 9743/10000 (97.43%)

New best model saved with accuracy: 97.43%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.614819
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.571632
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.570126
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.537970
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.566198
Train Epoch: 5 - Loss: 0.5852, Accuracy: 97.59%

Test set: Average loss: 0.5745, Accuracy: 9787/10000 (97.87%)

New best model saved with accuracy: 97.87%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.547689
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.601411
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.554722
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.568653
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.596887
Train Epoch: 6 - Loss: 0.5738, Accuracy: 97.89%

Test set: Average loss: 0.5680, Accuracy: 9804/10000 (98.04%)

New best model saved with accuracy: 98.04%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.560108
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.577601
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.553887
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.554480
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.531905
Train Epoch: 7 - Loss: 0.5661, Accuracy: 98.11%

Test set: Average loss: 0.5605, Accuracy: 9826/10000 (98.26%)

New best model saved with accuracy: 98.26%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.546145
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.572888
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.547234
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.558109
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.544259
Train Epoch: 8 - Loss: 0.5604, Accuracy: 98.29%

Test set: Average loss: 0.5570, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.584223
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.556517
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.568312
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.529729
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.576826
Train Epoch: 9 - Loss: 0.5558, Accuracy: 98.46%

Test set: Average loss: 0.5563, Accuracy: 9844/10000 (98.44%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.561226
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.582718
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.545004
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.557888
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.543324
Train Epoch: 10 - Loss: 0.5522, Accuracy: 98.58%

Test set: Average loss: 0.5503, Accuracy: 9854/10000 (98.54%)

New best model saved with accuracy: 98.54%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.567718
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.548252
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.556088
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.529288
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.580665
Train Epoch: 11 - Loss: 0.5491, Accuracy: 98.65%

Test set: Average loss: 0.5477, Accuracy: 9860/10000 (98.60%)

New best model saved with accuracy: 98.60%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.554054
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.540020
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.562291
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.581644
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.562111
Train Epoch: 12 - Loss: 0.5465, Accuracy: 98.73%

Test set: Average loss: 0.5480, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.528234
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.525837
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.532435
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.541307
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.539509
Train Epoch: 13 - Loss: 0.5447, Accuracy: 98.79%

Test set: Average loss: 0.5464, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.538725
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.549472
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.547847
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.542140
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.523465
Train Epoch: 14 - Loss: 0.5429, Accuracy: 98.86%

Test set: Average loss: 0.5441, Accuracy: 9870/10000 (98.70%)

New best model saved with accuracy: 98.70%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.535665
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.527224
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.525434
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.575996
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.532498
Train Epoch: 15 - Loss: 0.5415, Accuracy: 98.88%

Test set: Average loss: 0.5426, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.524352
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.528038
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.544071
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.556022
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.539533
Train Epoch: 16 - Loss: 0.5402, Accuracy: 98.93%

Test set: Average loss: 0.5419, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.543146
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.556812
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.550475
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.539080
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.520063
Train Epoch: 17 - Loss: 0.5392, Accuracy: 98.97%

Test set: Average loss: 0.5410, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.561184
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.521393
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.516634
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.564775
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.527964
Train Epoch: 18 - Loss: 0.5384, Accuracy: 99.00%

Test set: Average loss: 0.5408, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.582582
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.527551
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.546408
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.593586
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.542429
Train Epoch: 19 - Loss: 0.5378, Accuracy: 99.00%

Test set: Average loss: 0.5402, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.557074
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.552267
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.546368
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.544428
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.560938
Train Epoch: 20 - Loss: 0.5372, Accuracy: 99.01%

Test set: Average loss: 0.5399, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.521989
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.534481
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.521072
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.518333
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.528133
Train Epoch: 21 - Loss: 0.5368, Accuracy: 99.03%

Test set: Average loss: 0.5396, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.548232
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.564513
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.519060
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.532249
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.547103
Train Epoch: 22 - Loss: 0.5365, Accuracy: 99.07%

Test set: Average loss: 0.5393, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.524702
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.527508
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.527830
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.556523
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.534027
Train Epoch: 23 - Loss: 0.5364, Accuracy: 99.06%

Test set: Average loss: 0.5393, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.529429
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.519122
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.547870
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.550832
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.547130
Train Epoch: 24 - Loss: 0.5362, Accuracy: 99.08%

Test set: Average loss: 0.5393, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.516995
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.518054
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.521489
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.525960
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.522505
Train Epoch: 25 - Loss: 0.5361, Accuracy: 99.07%

Test set: Average loss: 0.5392, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.557739
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.546830
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.536542
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.541041
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.584256
Train Epoch: 26 - Loss: 0.5390, Accuracy: 98.94%

Test set: Average loss: 0.5407, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.554844
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.546895
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.539966
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.532731
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.543072
Train Epoch: 27 - Loss: 0.5367, Accuracy: 99.03%

Test set: Average loss: 0.5386, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.523643
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.595343
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.533345
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.532686
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.563140
Train Epoch: 28 - Loss: 0.5349, Accuracy: 99.04%

Test set: Average loss: 0.5373, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.520540
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.561159
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.539044
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.518786
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.524135
Train Epoch: 29 - Loss: 0.5330, Accuracy: 99.14%

Test set: Average loss: 0.5382, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.540211
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.527185
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.516174
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.515724
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.544962
Train Epoch: 30 - Loss: 0.5318, Accuracy: 99.16%

Test set: Average loss: 0.5350, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.511867
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.541683
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.544969
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.518643
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.513847
Train Epoch: 31 - Loss: 0.5304, Accuracy: 99.21%

Test set: Average loss: 0.5336, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.515681
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.519988
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.542990
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515413
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.545131
Train Epoch: 32 - Loss: 0.5289, Accuracy: 99.29%

Test set: Average loss: 0.5342, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.520791
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.543964
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.527470
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.527112
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.538383
Train Epoch: 33 - Loss: 0.5278, Accuracy: 99.28%

Test set: Average loss: 0.5331, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.517348
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.518065
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.542005
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.562658
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.541068
Train Epoch: 34 - Loss: 0.5269, Accuracy: 99.34%

Test set: Average loss: 0.5312, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.539252
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.515832
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.521696
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.510271
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.534791
Train Epoch: 35 - Loss: 0.5258, Accuracy: 99.36%

Test set: Average loss: 0.5308, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.532063
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.522724
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.534293
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.524847
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.517565
Train Epoch: 36 - Loss: 0.5250, Accuracy: 99.40%

Test set: Average loss: 0.5307, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.531267
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.533490
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.524838
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.535217
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.523090
Train Epoch: 37 - Loss: 0.5242, Accuracy: 99.42%

Test set: Average loss: 0.5304, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.546666
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.528133
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.513788
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.532023
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.536572
Train Epoch: 38 - Loss: 0.5235, Accuracy: 99.45%

Test set: Average loss: 0.5296, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.516073
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.539585
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.529519
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.551650
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.520209
Train Epoch: 39 - Loss: 0.5229, Accuracy: 99.46%

Test set: Average loss: 0.5294, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.513835
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.543095
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.524326
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.510514
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.514548
Train Epoch: 40 - Loss: 0.5224, Accuracy: 99.47%

Test set: Average loss: 0.5289, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.522576
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.512615
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.543377
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.516025
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.513853
Train Epoch: 41 - Loss: 0.5219, Accuracy: 99.50%

Test set: Average loss: 0.5285, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.521534
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.525697
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.520853
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.524010
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.510261
Train Epoch: 42 - Loss: 0.5215, Accuracy: 99.52%

Test set: Average loss: 0.5285, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.517667
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522890
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.515482
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.521859
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.527995
Train Epoch: 43 - Loss: 0.5212, Accuracy: 99.52%

Test set: Average loss: 0.5287, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.524428
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.515056
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524424
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.515778
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.512551
Train Epoch: 44 - Loss: 0.5209, Accuracy: 99.54%

Test set: Average loss: 0.5284, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.519684
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.529652
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.509899
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.514555
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.521401
Train Epoch: 45 - Loss: 0.5206, Accuracy: 99.54%

Test set: Average loss: 0.5281, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.523419
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.517586
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.514028
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.512054
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.516379
Train Epoch: 46 - Loss: 0.5205, Accuracy: 99.55%

Test set: Average loss: 0.5281, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.511559
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.516302
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.536673
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.519131
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522246
Train Epoch: 47 - Loss: 0.5203, Accuracy: 99.55%

Test set: Average loss: 0.5280, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.529607
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.518577
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.509911
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.537042
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.518184
Train Epoch: 48 - Loss: 0.5202, Accuracy: 99.56%

Test set: Average loss: 0.5280, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.519743
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.513503
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.514283
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.514639
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.508836
Train Epoch: 49 - Loss: 0.5202, Accuracy: 99.56%

Test set: Average loss: 0.5280, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.522144
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.523061
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.516881
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.519489
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.524928
Train Epoch: 50 - Loss: 0.5201, Accuracy: 99.56%

Test set: Average loss: 0.5280, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.519883
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.510817
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.517767
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.539532
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.521988
Train Epoch: 51 - Loss: 0.5224, Accuracy: 99.45%

Test set: Average loss: 0.5295, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.551068
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.511451
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.542759
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.526880
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.514222
Train Epoch: 52 - Loss: 0.5217, Accuracy: 99.47%

Test set: Average loss: 0.5285, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.507594
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.530357
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.518662
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509040
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.525404
Train Epoch: 53 - Loss: 0.5208, Accuracy: 99.53%

Test set: Average loss: 0.5279, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.510221
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.515871
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.515857
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513582
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.511973
Train Epoch: 54 - Loss: 0.5203, Accuracy: 99.51%

Test set: Average loss: 0.5294, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.511219
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.526400
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.514595
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.514928
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.519526
Train Epoch: 55 - Loss: 0.5195, Accuracy: 99.55%

Test set: Average loss: 0.5269, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.513524
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.523130
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.511585
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.532940
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.511967
Train Epoch: 56 - Loss: 0.5188, Accuracy: 99.57%

Test set: Average loss: 0.5273, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514049
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511173
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.517649
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.509041
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.514929
Train Epoch: 57 - Loss: 0.5182, Accuracy: 99.59%

Test set: Average loss: 0.5266, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.519453
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.520783
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.511351
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.509978
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.555899
Train Epoch: 58 - Loss: 0.5176, Accuracy: 99.61%

Test set: Average loss: 0.5262, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.507692
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.517735
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.538293
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.510699
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.506869
Train Epoch: 59 - Loss: 0.5171, Accuracy: 99.63%

Test set: Average loss: 0.5258, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510521
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513248
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.506198
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.540341
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.535962
Train Epoch: 60 - Loss: 0.5167, Accuracy: 99.65%

Test set: Average loss: 0.5259, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.513295
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.519909
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.513905
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.511072
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.515748
Train Epoch: 61 - Loss: 0.5162, Accuracy: 99.67%

Test set: Average loss: 0.5264, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.511647
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506528
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.509846
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.511974
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.504287
Train Epoch: 62 - Loss: 0.5157, Accuracy: 99.66%

Test set: Average loss: 0.5259, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.505625
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.509703
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.513707
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.508467
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.513847
Train Epoch: 63 - Loss: 0.5153, Accuracy: 99.69%

Test set: Average loss: 0.5253, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513094
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.533135
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506145
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.536253
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.544941
Train Epoch: 64 - Loss: 0.5151, Accuracy: 99.70%

Test set: Average loss: 0.5251, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.510437
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.525237
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.522365
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.509723
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508290
Train Epoch: 65 - Loss: 0.5147, Accuracy: 99.72%

Test set: Average loss: 0.5248, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.513187
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.523003
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.514596
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.519729
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.505747
Train Epoch: 66 - Loss: 0.5144, Accuracy: 99.72%

Test set: Average loss: 0.5250, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.511737
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.519491
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.518879
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.505107
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.505299
Train Epoch: 67 - Loss: 0.5142, Accuracy: 99.73%

Test set: Average loss: 0.5247, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.513109
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.506861
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.512111
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.510186
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.507834
Train Epoch: 68 - Loss: 0.5140, Accuracy: 99.74%

Test set: Average loss: 0.5246, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.526426
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.509606
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.511736
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.532926
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.519877
Train Epoch: 69 - Loss: 0.5138, Accuracy: 99.73%

Test set: Average loss: 0.5244, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.505431
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.508286
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.507861
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.507278
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.528978
Train Epoch: 70 - Loss: 0.5137, Accuracy: 99.74%

Test set: Average loss: 0.5245, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.517312
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.507834
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.507276
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510072
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.510599
Train Epoch: 71 - Loss: 0.5136, Accuracy: 99.75%

Test set: Average loss: 0.5246, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.515263
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.510032
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.510777
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.520763
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.528453
Train Epoch: 72 - Loss: 0.5135, Accuracy: 99.76%

Test set: Average loss: 0.5244, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505992
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.507281
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508017
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.528180
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.512564
Train Epoch: 73 - Loss: 0.5134, Accuracy: 99.76%

Test set: Average loss: 0.5243, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.514296
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.515941
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.507620
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.510235
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.506400
Train Epoch: 74 - Loss: 0.5134, Accuracy: 99.77%

Test set: Average loss: 0.5244, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.514854
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.506344
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.533704
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506389
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.511203
Train Epoch: 75 - Loss: 0.5133, Accuracy: 99.77%

Test set: Average loss: 0.5244, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.513313
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.506668
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.508471
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.506821
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.508772
Train Epoch: 76 - Loss: 0.5152, Accuracy: 99.69%

Test set: Average loss: 0.5251, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.516800
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.516042
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.514418
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.513786
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.509990
Train Epoch: 77 - Loss: 0.5148, Accuracy: 99.70%

Test set: Average loss: 0.5251, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.543690
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.515463
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.505565
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.524165
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.514505
Train Epoch: 78 - Loss: 0.5143, Accuracy: 99.70%

Test set: Average loss: 0.5246, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.506220
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.515978
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.512392
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.511526
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.513970
Train Epoch: 79 - Loss: 0.5138, Accuracy: 99.73%

Test set: Average loss: 0.5251, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.507545
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.509767
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509722
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.509089
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.518901
Train Epoch: 80 - Loss: 0.5134, Accuracy: 99.73%

Test set: Average loss: 0.5254, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.506107
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511401
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.509504
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.506421
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.511218
Train Epoch: 81 - Loss: 0.5130, Accuracy: 99.76%

Test set: Average loss: 0.5250, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.503818
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.506563
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.509344
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.522362
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.509305
Train Epoch: 82 - Loss: 0.5126, Accuracy: 99.76%

Test set: Average loss: 0.5240, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.505124
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.507083
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506680
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.512430
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.504163
Train Epoch: 83 - Loss: 0.5123, Accuracy: 99.77%

Test set: Average loss: 0.5241, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.505973
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.515538
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.512600
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.525898
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.510953
Train Epoch: 84 - Loss: 0.5120, Accuracy: 99.78%

Test set: Average loss: 0.5237, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.506435
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.508405
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.509692
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.509666
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.514298
Train Epoch: 85 - Loss: 0.5117, Accuracy: 99.81%

Test set: Average loss: 0.5234, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.506268
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.510911
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511326
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.521290
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.507146
Train Epoch: 86 - Loss: 0.5114, Accuracy: 99.81%

Test set: Average loss: 0.5235, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.505490
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.505717
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.506368
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505543
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.507259
Train Epoch: 87 - Loss: 0.5111, Accuracy: 99.81%

Test set: Average loss: 0.5233, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.507010
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.505558
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.504529
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.535136
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.513658
Train Epoch: 88 - Loss: 0.5108, Accuracy: 99.83%

Test set: Average loss: 0.5235, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.505590
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.508178
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507139
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.526055
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512410
Train Epoch: 89 - Loss: 0.5106, Accuracy: 99.84%

Test set: Average loss: 0.5232, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508094
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.506019
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507127
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508283
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509408
Train Epoch: 90 - Loss: 0.5105, Accuracy: 99.84%

Test set: Average loss: 0.5229, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.507908
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.504536
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509636
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507941
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.507443
Train Epoch: 91 - Loss: 0.5103, Accuracy: 99.84%

Test set: Average loss: 0.5230, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.505538
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.513003
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.504226
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.516944
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.507881
Train Epoch: 92 - Loss: 0.5101, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.507712
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.506667
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.506124
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.508083
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507751
Train Epoch: 93 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5227, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.509055
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.513997
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.505429
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.517088
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.505791
Train Epoch: 94 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5228, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.508468
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.505805
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508163
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.504502
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.512224
Train Epoch: 95 - Loss: 0.5098, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.527808
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.504841
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507143
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.520728
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.523540
Train Epoch: 96 - Loss: 0.5097, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.521915
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.505779
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.504664
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.505343
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.505554
Train Epoch: 97 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5226, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.505328
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.515397
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.504892
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511355
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.516443
Train Epoch: 98 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5227, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.505086
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504824
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.503881
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.505318
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.513464
Train Epoch: 99 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5226, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.535646
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.508476
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.505416
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504923
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.524110
Train Epoch: 100 - Loss: 0.5095, Accuracy: 99.86%

Test set: Average loss: 0.5227, Accuracy: 9921/10000 (99.21%)


Training completed. Best Accuracy: 99.24%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.301506
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.278136
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.016723
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.817829
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.790261
Train Epoch: 1 - Loss: 1.3391, Accuracy: 65.33%

Test set: Average loss: 0.7529, Accuracy: 9200/10000 (92.00%)

New best model saved with accuracy: 92.00%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.960977
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.717484
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.665812
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.782600
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.693181
Train Epoch: 2 - Loss: 0.7032, Accuracy: 93.72%

Test set: Average loss: 0.6512, Accuracy: 9555/10000 (95.55%)

New best model saved with accuracy: 95.55%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.680132
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.629685
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.640109
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.723071
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.619450
Train Epoch: 3 - Loss: 0.6347, Accuracy: 95.97%

Test set: Average loss: 0.6094, Accuracy: 9677/10000 (96.77%)

New best model saved with accuracy: 96.77%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.583914
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.587490
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.575818
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.599199
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.573017
Train Epoch: 4 - Loss: 0.6039, Accuracy: 96.97%

Test set: Average loss: 0.5911, Accuracy: 9730/10000 (97.30%)

New best model saved with accuracy: 97.30%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.589186
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.593540
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.594376
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.573952
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.561264
Train Epoch: 5 - Loss: 0.5872, Accuracy: 97.45%

Test set: Average loss: 0.5805, Accuracy: 9755/10000 (97.55%)

New best model saved with accuracy: 97.55%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.574452
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.597620
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.583446
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.558483
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.596338
Train Epoch: 6 - Loss: 0.5764, Accuracy: 97.81%

Test set: Average loss: 0.5725, Accuracy: 9772/10000 (97.72%)

New best model saved with accuracy: 97.72%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.558001
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.541534
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.570930
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.548680
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.546110
Train Epoch: 7 - Loss: 0.5686, Accuracy: 98.04%

Test set: Average loss: 0.5699, Accuracy: 9781/10000 (97.81%)

New best model saved with accuracy: 97.81%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.548412
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.581448
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.591237
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.566534
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.549613
Train Epoch: 8 - Loss: 0.5631, Accuracy: 98.18%

Test set: Average loss: 0.5633, Accuracy: 9797/10000 (97.97%)

New best model saved with accuracy: 97.97%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.538451
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.550750
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.545005
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.551626
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.562309
Train Epoch: 9 - Loss: 0.5585, Accuracy: 98.35%

Test set: Average loss: 0.5579, Accuracy: 9826/10000 (98.26%)

New best model saved with accuracy: 98.26%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.533516
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.547110
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.566488
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.574481
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.543316
Train Epoch: 10 - Loss: 0.5547, Accuracy: 98.43%

Test set: Average loss: 0.5557, Accuracy: 9822/10000 (98.22%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.558004
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.571696
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.554939
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.552855
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.540592
Train Epoch: 11 - Loss: 0.5514, Accuracy: 98.53%

Test set: Average loss: 0.5534, Accuracy: 9831/10000 (98.31%)

New best model saved with accuracy: 98.31%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.554526
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.552060
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.557987
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.549264
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.557663
Train Epoch: 12 - Loss: 0.5490, Accuracy: 98.59%

Test set: Average loss: 0.5509, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.537103
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.531495
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.553018
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.563965
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.519136
Train Epoch: 13 - Loss: 0.5467, Accuracy: 98.73%

Test set: Average loss: 0.5514, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.530633
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.535167
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.587577
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.538801
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.554796
Train Epoch: 14 - Loss: 0.5450, Accuracy: 98.73%

Test set: Average loss: 0.5484, Accuracy: 9849/10000 (98.49%)

New best model saved with accuracy: 98.49%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.519492
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.558147
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.534293
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.544044
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.551762
Train Epoch: 15 - Loss: 0.5434, Accuracy: 98.83%

Test set: Average loss: 0.5463, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.528083
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.541149
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.573612
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.551090
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.563489
Train Epoch: 16 - Loss: 0.5421, Accuracy: 98.88%

Test set: Average loss: 0.5461, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.544164
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.549923
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.540680
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.520547
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.524737
Train Epoch: 17 - Loss: 0.5409, Accuracy: 98.92%

Test set: Average loss: 0.5450, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.536967
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.526124
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.572200
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.545789
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.529339
Train Epoch: 18 - Loss: 0.5402, Accuracy: 98.93%

Test set: Average loss: 0.5441, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.534267
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.539773
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.520150
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.520795
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.529787
Train Epoch: 19 - Loss: 0.5394, Accuracy: 98.96%

Test set: Average loss: 0.5439, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.533284
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.578012
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.532777
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.553668
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.524949
Train Epoch: 20 - Loss: 0.5389, Accuracy: 98.97%

Test set: Average loss: 0.5435, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.526827
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.530900
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.548380
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.518052
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.552346
Train Epoch: 21 - Loss: 0.5384, Accuracy: 98.98%

Test set: Average loss: 0.5434, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.550467
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.530037
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.534843
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.536823
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.517992
Train Epoch: 22 - Loss: 0.5381, Accuracy: 99.03%

Test set: Average loss: 0.5431, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.524423
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.527760
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.542174
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.521909
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.541217
Train Epoch: 23 - Loss: 0.5379, Accuracy: 99.03%

Test set: Average loss: 0.5430, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.523109
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.530305
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.537067
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.546470
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.531479
Train Epoch: 24 - Loss: 0.5378, Accuracy: 99.03%

Test set: Average loss: 0.5430, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.527206
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.549123
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.559783
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.533779
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.542526
Train Epoch: 25 - Loss: 0.5377, Accuracy: 99.03%

Test set: Average loss: 0.5429, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.534820
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.530608
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.534375
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.521834
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.523755
Train Epoch: 26 - Loss: 0.5411, Accuracy: 98.88%

Test set: Average loss: 0.5423, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.549693
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.538103
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.549201
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.543385
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.537571
Train Epoch: 27 - Loss: 0.5387, Accuracy: 98.95%

Test set: Average loss: 0.5417, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.532438
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.536072
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.545009
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.564860
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.538126
Train Epoch: 28 - Loss: 0.5368, Accuracy: 98.99%

Test set: Average loss: 0.5408, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.533656
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.541438
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.551602
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.533055
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.522499
Train Epoch: 29 - Loss: 0.5347, Accuracy: 99.11%

Test set: Average loss: 0.5423, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.554401
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.538331
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.545586
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.517632
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.533547
Train Epoch: 30 - Loss: 0.5331, Accuracy: 99.11%

Test set: Average loss: 0.5396, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.530868
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.517116
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.523012
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.556649
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.527775
Train Epoch: 31 - Loss: 0.5315, Accuracy: 99.19%

Test set: Average loss: 0.5381, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.519868
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.530176
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.521151
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.518814
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.546228
Train Epoch: 32 - Loss: 0.5302, Accuracy: 99.24%

Test set: Average loss: 0.5368, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.556986
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.532888
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.511378
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.536123
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.520401
Train Epoch: 33 - Loss: 0.5290, Accuracy: 99.26%

Test set: Average loss: 0.5354, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.526949
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.521906
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.520122
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.520019
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.532208
Train Epoch: 34 - Loss: 0.5279, Accuracy: 99.29%

Test set: Average loss: 0.5355, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.512497
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.514636
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.550641
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.547076
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.549719
Train Epoch: 35 - Loss: 0.5269, Accuracy: 99.35%

Test set: Average loss: 0.5339, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.511442
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.557345
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.521091
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.537752
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.515377
Train Epoch: 36 - Loss: 0.5260, Accuracy: 99.38%

Test set: Average loss: 0.5331, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.553743
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516080
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.523467
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.537731
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.536694
Train Epoch: 37 - Loss: 0.5252, Accuracy: 99.39%

Test set: Average loss: 0.5346, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.513925
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.513115
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.522606
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.511334
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.528930
Train Epoch: 38 - Loss: 0.5244, Accuracy: 99.42%

Test set: Average loss: 0.5328, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.520849
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514138
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.541899
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.536128
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.522532
Train Epoch: 39 - Loss: 0.5239, Accuracy: 99.43%

Test set: Average loss: 0.5322, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.520488
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.516587
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.520661
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.520013
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.527946
Train Epoch: 40 - Loss: 0.5232, Accuracy: 99.46%

Test set: Average loss: 0.5325, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.511292
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.519878
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.507944
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.521378
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.508053
Train Epoch: 41 - Loss: 0.5228, Accuracy: 99.47%

Test set: Average loss: 0.5326, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.524058
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.525471
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.517291
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.529729
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.530400
Train Epoch: 42 - Loss: 0.5224, Accuracy: 99.50%

Test set: Average loss: 0.5317, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.510050
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.513644
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.513447
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.513386
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.529952
Train Epoch: 43 - Loss: 0.5220, Accuracy: 99.52%

Test set: Average loss: 0.5313, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.512699
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.515406
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524361
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.526090
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.515602
Train Epoch: 44 - Loss: 0.5217, Accuracy: 99.53%

Test set: Average loss: 0.5314, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.526440
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.516736
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.518286
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.530212
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.526526
Train Epoch: 45 - Loss: 0.5215, Accuracy: 99.53%

Test set: Average loss: 0.5313, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.525559
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.543374
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.532903
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.526699
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.512689
Train Epoch: 46 - Loss: 0.5213, Accuracy: 99.55%

Test set: Average loss: 0.5311, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508948
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.509590
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.513303
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.511427
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522159
Train Epoch: 47 - Loss: 0.5211, Accuracy: 99.55%

Test set: Average loss: 0.5311, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.515067
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.515791
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.510084
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.516608
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.528256
Train Epoch: 48 - Loss: 0.5210, Accuracy: 99.55%

Test set: Average loss: 0.5310, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.516530
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.525331
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.511192
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.518791
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.510843
Train Epoch: 49 - Loss: 0.5210, Accuracy: 99.56%

Test set: Average loss: 0.5310, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.509403
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.510515
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.526159
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.515528
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.513652
Train Epoch: 50 - Loss: 0.5209, Accuracy: 99.56%

Test set: Average loss: 0.5310, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.511350
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.516725
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.526226
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.518038
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.509010
Train Epoch: 51 - Loss: 0.5232, Accuracy: 99.44%

Test set: Average loss: 0.5320, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.517927
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.509141
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.510304
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.515023
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.519279
Train Epoch: 52 - Loss: 0.5226, Accuracy: 99.47%

Test set: Average loss: 0.5330, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.535188
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.539241
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.536782
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.513614
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.515752
Train Epoch: 53 - Loss: 0.5217, Accuracy: 99.50%

Test set: Average loss: 0.5301, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.538144
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.516818
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.521337
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.514306
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.524362
Train Epoch: 54 - Loss: 0.5210, Accuracy: 99.48%

Test set: Average loss: 0.5319, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.512160
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.514562
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.515188
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.516973
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.522331
Train Epoch: 55 - Loss: 0.5201, Accuracy: 99.57%

Test set: Average loss: 0.5318, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.510486
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.520410
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.532372
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.512587
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.518850
Train Epoch: 56 - Loss: 0.5196, Accuracy: 99.57%

Test set: Average loss: 0.5309, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.518210
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.530321
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.524630
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.517764
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.518372
Train Epoch: 57 - Loss: 0.5190, Accuracy: 99.59%

Test set: Average loss: 0.5303, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.524972
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.514952
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.515657
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.526315
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.515580
Train Epoch: 58 - Loss: 0.5183, Accuracy: 99.62%

Test set: Average loss: 0.5290, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.517272
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.508169
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.508226
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.508051
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.516565
Train Epoch: 59 - Loss: 0.5178, Accuracy: 99.62%

Test set: Average loss: 0.5291, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.511106
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.518498
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.530859
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.511019
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.510236
Train Epoch: 60 - Loss: 0.5173, Accuracy: 99.64%

Test set: Average loss: 0.5287, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.513609
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.522593
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.538269
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.521394
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.537129
Train Epoch: 61 - Loss: 0.5168, Accuracy: 99.67%

Test set: Average loss: 0.5282, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510170
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.509616
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.511385
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.517348
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.513748
Train Epoch: 62 - Loss: 0.5163, Accuracy: 99.67%

Test set: Average loss: 0.5300, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.508216
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.504729
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.508655
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.515672
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.524105
Train Epoch: 63 - Loss: 0.5159, Accuracy: 99.69%

Test set: Average loss: 0.5278, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.517052
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.520102
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506635
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.523050
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.511043
Train Epoch: 64 - Loss: 0.5155, Accuracy: 99.71%

Test set: Average loss: 0.5278, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.506097
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.517978
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.518361
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.506275
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.520486
Train Epoch: 65 - Loss: 0.5152, Accuracy: 99.73%

Test set: Average loss: 0.5280, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.507406
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.516859
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.510404
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.509384
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506014
Train Epoch: 66 - Loss: 0.5149, Accuracy: 99.73%

Test set: Average loss: 0.5276, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.511194
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.508058
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.515043
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.527915
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.510080
Train Epoch: 67 - Loss: 0.5147, Accuracy: 99.75%

Test set: Average loss: 0.5274, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.516455
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510819
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.520439
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515948
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.516160
Train Epoch: 68 - Loss: 0.5145, Accuracy: 99.75%

Test set: Average loss: 0.5273, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.530460
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.518871
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.510930
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.520274
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510007
Train Epoch: 69 - Loss: 0.5144, Accuracy: 99.74%

Test set: Average loss: 0.5276, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.508064
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.516916
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.522753
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.505215
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507996
Train Epoch: 70 - Loss: 0.5142, Accuracy: 99.75%

Test set: Average loss: 0.5273, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.509340
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.511757
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.508244
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.513065
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.510073
Train Epoch: 71 - Loss: 0.5140, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.521725
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.510707
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.512167
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.518081
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508782
Train Epoch: 72 - Loss: 0.5139, Accuracy: 99.76%

Test set: Average loss: 0.5272, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.525978
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508924
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.537181
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.512835
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.507069
Train Epoch: 73 - Loss: 0.5139, Accuracy: 99.76%

Test set: Average loss: 0.5272, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.507421
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508187
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.514379
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.510760
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.506000
Train Epoch: 74 - Loss: 0.5138, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.517222
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509785
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.511891
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.513779
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.544649
Train Epoch: 75 - Loss: 0.5138, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.521375
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.509719
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510065
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.531276
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507167
Train Epoch: 76 - Loss: 0.5156, Accuracy: 99.69%

Test set: Average loss: 0.5275, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.512055
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.534077
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.534926
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.512730
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.509670
Train Epoch: 77 - Loss: 0.5152, Accuracy: 99.69%

Test set: Average loss: 0.5276, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.509321
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.519496
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.515990
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.516300
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.510937
Train Epoch: 78 - Loss: 0.5147, Accuracy: 99.72%

Test set: Average loss: 0.5277, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.506985
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.519232
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.520326
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.525634
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.508313
Train Epoch: 79 - Loss: 0.5143, Accuracy: 99.72%

Test set: Average loss: 0.5270, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.506371
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.505556
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.506135
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.512661
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.504631
Train Epoch: 80 - Loss: 0.5138, Accuracy: 99.75%

Test set: Average loss: 0.5278, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508643
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.506246
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.513602
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509856
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.515719
Train Epoch: 81 - Loss: 0.5135, Accuracy: 99.75%

Test set: Average loss: 0.5264, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.510331
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.513387
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.507839
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.509447
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.523867
Train Epoch: 82 - Loss: 0.5130, Accuracy: 99.78%

Test set: Average loss: 0.5274, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.513528
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.514468
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.512267
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506535
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.510735
Train Epoch: 83 - Loss: 0.5127, Accuracy: 99.78%

Test set: Average loss: 0.5267, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509709
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.511791
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.507774
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.504797
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.516398
Train Epoch: 84 - Loss: 0.5123, Accuracy: 99.80%

Test set: Average loss: 0.5265, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.527385
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.506509
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.513026
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.510978
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.513095
Train Epoch: 85 - Loss: 0.5119, Accuracy: 99.82%

Test set: Average loss: 0.5261, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.505001
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.512675
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511220
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.513753
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.523995
Train Epoch: 86 - Loss: 0.5116, Accuracy: 99.83%

Test set: Average loss: 0.5265, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.513847
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.505673
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.505776
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.507697
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.522772
Train Epoch: 87 - Loss: 0.5114, Accuracy: 99.83%

Test set: Average loss: 0.5259, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510668
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.509660
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.520906
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.505910
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.504249
Train Epoch: 88 - Loss: 0.5112, Accuracy: 99.84%

Test set: Average loss: 0.5261, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.503840
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.511353
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.509105
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.507360
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.518612
Train Epoch: 89 - Loss: 0.5109, Accuracy: 99.85%

Test set: Average loss: 0.5259, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508701
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.512775
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.508619
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.507211
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.513262
Train Epoch: 90 - Loss: 0.5107, Accuracy: 99.85%

Test set: Average loss: 0.5262, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.515502
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.504698
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.515350
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.506204
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.507586
Train Epoch: 91 - Loss: 0.5105, Accuracy: 99.86%

Test set: Average loss: 0.5255, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.505381
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.515257
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.528766
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.510788
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.511115
Train Epoch: 92 - Loss: 0.5104, Accuracy: 99.86%

Test set: Average loss: 0.5254, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.507410
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.513197
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.510680
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.508485
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.506808
Train Epoch: 93 - Loss: 0.5102, Accuracy: 99.86%

Test set: Average loss: 0.5257, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.507450
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507056
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.512272
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.525250
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.512317
Train Epoch: 94 - Loss: 0.5101, Accuracy: 99.87%

Test set: Average loss: 0.5255, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.509121
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.506963
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506198
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.507380
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.526517
Train Epoch: 95 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5254, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.506951
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.505597
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508237
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.506987
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.503706
Train Epoch: 96 - Loss: 0.5099, Accuracy: 99.86%

Test set: Average loss: 0.5253, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.507659
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.517277
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.508913
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.519032
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507831
Train Epoch: 97 - Loss: 0.5098, Accuracy: 99.87%

Test set: Average loss: 0.5254, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.510559
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509714
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.509410
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.507715
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.509365
Train Epoch: 98 - Loss: 0.5098, Accuracy: 99.86%

Test set: Average loss: 0.5254, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.512662
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.507395
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506500
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.505234
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.504482
Train Epoch: 99 - Loss: 0.5098, Accuracy: 99.87%

Test set: Average loss: 0.5254, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.509106
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.506530
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.507544
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.506685
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.507173
Train Epoch: 100 - Loss: 0.5097, Accuracy: 99.87%

Test set: Average loss: 0.5254, Accuracy: 9897/10000 (98.97%)


Training completed. Best Accuracy: 99.02%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.308774
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.293123
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.569811
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.004305
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.905246
Train Epoch: 1 - Loss: 1.4820, Accuracy: 57.30%

Test set: Average loss: 0.7692, Accuracy: 9143/10000 (91.43%)

New best model saved with accuracy: 91.43%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.762797
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.774619
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.736841
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.696597
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.705718
Train Epoch: 2 - Loss: 0.7124, Accuracy: 93.34%

Test set: Average loss: 0.6563, Accuracy: 9540/10000 (95.40%)

New best model saved with accuracy: 95.40%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.692141
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.609749
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.665039
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.642738
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.611324
Train Epoch: 3 - Loss: 0.6388, Accuracy: 95.95%

Test set: Average loss: 0.6161, Accuracy: 9662/10000 (96.62%)

New best model saved with accuracy: 96.62%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.656867
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.599742
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.597197
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.599655
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.636633
Train Epoch: 4 - Loss: 0.6066, Accuracy: 96.91%

Test set: Average loss: 0.5977, Accuracy: 9736/10000 (97.36%)

New best model saved with accuracy: 97.36%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.601395
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.622700
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.591681
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.567522
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.559925
Train Epoch: 5 - Loss: 0.5885, Accuracy: 97.51%

Test set: Average loss: 0.5825, Accuracy: 9757/10000 (97.57%)

New best model saved with accuracy: 97.57%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.583553
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.593860
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.598395
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.578795
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.596421
Train Epoch: 6 - Loss: 0.5770, Accuracy: 97.80%

Test set: Average loss: 0.5755, Accuracy: 9775/10000 (97.75%)

New best model saved with accuracy: 97.75%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.583694
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.549534
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.553359
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.589649
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.573896
Train Epoch: 7 - Loss: 0.5688, Accuracy: 98.06%

Test set: Average loss: 0.5639, Accuracy: 9813/10000 (98.13%)

New best model saved with accuracy: 98.13%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.573512
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.557401
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.594402
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.534139
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.575071
Train Epoch: 8 - Loss: 0.5628, Accuracy: 98.25%

Test set: Average loss: 0.5603, Accuracy: 9819/10000 (98.19%)

New best model saved with accuracy: 98.19%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.565047
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.549621
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.556600
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.563501
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.551340
Train Epoch: 9 - Loss: 0.5581, Accuracy: 98.39%

Test set: Average loss: 0.5572, Accuracy: 9824/10000 (98.24%)

New best model saved with accuracy: 98.24%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.559662
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.555545
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.561571
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.537808
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.567667
Train Epoch: 10 - Loss: 0.5541, Accuracy: 98.53%

Test set: Average loss: 0.5557, Accuracy: 9835/10000 (98.35%)

New best model saved with accuracy: 98.35%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.565250
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.563043
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.560859
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.543697
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.545293
Train Epoch: 11 - Loss: 0.5511, Accuracy: 98.67%

Test set: Average loss: 0.5538, Accuracy: 9841/10000 (98.41%)

New best model saved with accuracy: 98.41%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.556376
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.549888
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.541305
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.551663
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.552208
Train Epoch: 12 - Loss: 0.5483, Accuracy: 98.73%

Test set: Average loss: 0.5495, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.574065
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.544651
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.535104
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.544957
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.556990
Train Epoch: 13 - Loss: 0.5461, Accuracy: 98.77%

Test set: Average loss: 0.5486, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.543422
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.546637
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.543708
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.535073
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.530184
Train Epoch: 14 - Loss: 0.5443, Accuracy: 98.87%

Test set: Average loss: 0.5471, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.572955
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.556702
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.531396
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.517416
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.542918
Train Epoch: 15 - Loss: 0.5429, Accuracy: 98.90%

Test set: Average loss: 0.5461, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.555198
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.533639
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.531772
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.522164
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.541002
Train Epoch: 16 - Loss: 0.5418, Accuracy: 98.93%

Test set: Average loss: 0.5457, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.525963
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.522683
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.536151
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.559845
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.536104
Train Epoch: 17 - Loss: 0.5407, Accuracy: 98.97%

Test set: Average loss: 0.5451, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.523120
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.544456
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.524560
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.560314
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.551038
Train Epoch: 18 - Loss: 0.5398, Accuracy: 98.99%

Test set: Average loss: 0.5438, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.534530
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.526374
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.546566
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.524919
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.529467
Train Epoch: 19 - Loss: 0.5391, Accuracy: 99.00%

Test set: Average loss: 0.5441, Accuracy: 9870/10000 (98.70%)

New best model saved with accuracy: 98.70%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.554981
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.541752
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.554650
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.533240
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.524624
Train Epoch: 20 - Loss: 0.5386, Accuracy: 99.04%

Test set: Average loss: 0.5434, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.539846
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.530397
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.514831
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.526373
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.589355
Train Epoch: 21 - Loss: 0.5382, Accuracy: 99.06%

Test set: Average loss: 0.5431, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.525926
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.522903
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.542925
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.535606
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.529446
Train Epoch: 22 - Loss: 0.5379, Accuracy: 99.06%

Test set: Average loss: 0.5430, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.519193
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.523638
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.525840
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.516067
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.529472
Train Epoch: 23 - Loss: 0.5376, Accuracy: 99.07%

Test set: Average loss: 0.5429, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.530894
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.559352
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.536258
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.533804
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.516195
Train Epoch: 24 - Loss: 0.5375, Accuracy: 99.08%

Test set: Average loss: 0.5428, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.547958
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.533385
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.535506
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.525239
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.540719
Train Epoch: 25 - Loss: 0.5375, Accuracy: 99.08%

Test set: Average loss: 0.5428, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.551071
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.517301
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.540290
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.533773
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.547444
Train Epoch: 26 - Loss: 0.5406, Accuracy: 98.94%

Test set: Average loss: 0.5451, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.523980
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.536094
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.535193
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.531816
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.517417
Train Epoch: 27 - Loss: 0.5383, Accuracy: 99.02%

Test set: Average loss: 0.5411, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.540273
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.519724
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.535664
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.523719
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.533089
Train Epoch: 28 - Loss: 0.5362, Accuracy: 99.11%

Test set: Average loss: 0.5392, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.525230
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.535092
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.544316
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.528373
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.515085
Train Epoch: 29 - Loss: 0.5346, Accuracy: 99.13%

Test set: Average loss: 0.5438, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.535129
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.515259
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.542255
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.529774
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.518575
Train Epoch: 30 - Loss: 0.5329, Accuracy: 99.19%

Test set: Average loss: 0.5403, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.516877
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.541350
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.542182
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.524088
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.524310
Train Epoch: 31 - Loss: 0.5314, Accuracy: 99.23%

Test set: Average loss: 0.5374, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.544182
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.517292
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.530958
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.530988
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.536712
Train Epoch: 32 - Loss: 0.5301, Accuracy: 99.28%

Test set: Average loss: 0.5366, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.518525
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.519805
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.520200
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.522587
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.530388
Train Epoch: 33 - Loss: 0.5290, Accuracy: 99.30%

Test set: Average loss: 0.5361, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.523994
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.536158
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.527052
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.514888
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.519799
Train Epoch: 34 - Loss: 0.5280, Accuracy: 99.34%

Test set: Average loss: 0.5340, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.527737
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.522236
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.524408
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.519917
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.524636
Train Epoch: 35 - Loss: 0.5270, Accuracy: 99.38%

Test set: Average loss: 0.5346, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.510066
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.532352
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.547435
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.527490
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.533950
Train Epoch: 36 - Loss: 0.5262, Accuracy: 99.38%

Test set: Average loss: 0.5331, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.517349
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.535639
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.511770
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.549371
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.517507
Train Epoch: 37 - Loss: 0.5253, Accuracy: 99.44%

Test set: Average loss: 0.5326, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.557276
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.514256
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.525491
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.539760
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.530472
Train Epoch: 38 - Loss: 0.5247, Accuracy: 99.44%

Test set: Average loss: 0.5324, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.524224
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.512248
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.515938
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.524207
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516520
Train Epoch: 39 - Loss: 0.5241, Accuracy: 99.46%

Test set: Average loss: 0.5324, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.539543
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.541028
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.526895
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.515904
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.550291
Train Epoch: 40 - Loss: 0.5235, Accuracy: 99.48%

Test set: Average loss: 0.5317, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.511629
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.510444
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.529403
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.520470
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.513086
Train Epoch: 41 - Loss: 0.5230, Accuracy: 99.50%

Test set: Average loss: 0.5315, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.519139
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.514121
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.536410
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.517465
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.543340
Train Epoch: 42 - Loss: 0.5226, Accuracy: 99.53%

Test set: Average loss: 0.5316, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.525475
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.517805
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.513752
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.535246
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.512702
Train Epoch: 43 - Loss: 0.5222, Accuracy: 99.54%

Test set: Average loss: 0.5311, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.526541
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.512964
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.523094
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.522789
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.536811
Train Epoch: 44 - Loss: 0.5220, Accuracy: 99.53%

Test set: Average loss: 0.5311, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.520524
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.515611
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.523591
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.518415
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.557174
Train Epoch: 45 - Loss: 0.5217, Accuracy: 99.55%

Test set: Average loss: 0.5308, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.533913
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.514722
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.515409
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.520093
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.512837
Train Epoch: 46 - Loss: 0.5215, Accuracy: 99.56%

Test set: Average loss: 0.5308, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.511119
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.511899
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.525535
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.511240
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.508703
Train Epoch: 47 - Loss: 0.5214, Accuracy: 99.57%

Test set: Average loss: 0.5307, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.514489
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.511679
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.527229
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.512080
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.519924
Train Epoch: 48 - Loss: 0.5213, Accuracy: 99.58%

Test set: Average loss: 0.5307, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.519139
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.512266
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.518906
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.516232
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.513774
Train Epoch: 49 - Loss: 0.5212, Accuracy: 99.58%

Test set: Average loss: 0.5307, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.532902
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.513987
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.518475
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.534184
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.513237
Train Epoch: 50 - Loss: 0.5212, Accuracy: 99.58%

Test set: Average loss: 0.5307, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513526
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.510662
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.533025
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.543451
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.507628
Train Epoch: 51 - Loss: 0.5237, Accuracy: 99.43%

Test set: Average loss: 0.5322, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.524659
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.515796
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.515783
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.551415
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.507730
Train Epoch: 52 - Loss: 0.5227, Accuracy: 99.51%

Test set: Average loss: 0.5359, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.519691
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.521581
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.521179
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.507919
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.543360
Train Epoch: 53 - Loss: 0.5219, Accuracy: 99.51%

Test set: Average loss: 0.5311, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.563482
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521646
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.528736
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.515059
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.508920
Train Epoch: 54 - Loss: 0.5212, Accuracy: 99.53%

Test set: Average loss: 0.5334, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.518987
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.514084
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.519513
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.523693
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.517159
Train Epoch: 55 - Loss: 0.5206, Accuracy: 99.56%

Test set: Average loss: 0.5295, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.514712
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.515933
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.512158
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.545239
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.517813
Train Epoch: 56 - Loss: 0.5199, Accuracy: 99.56%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.523673
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.514550
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.509832
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.512439
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.512405
Train Epoch: 57 - Loss: 0.5193, Accuracy: 99.60%

Test set: Average loss: 0.5284, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.537672
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.512590
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.516420
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.519140
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.512181
Train Epoch: 58 - Loss: 0.5187, Accuracy: 99.61%

Test set: Average loss: 0.5290, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514518
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.512311
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.523651
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.515305
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.528697
Train Epoch: 59 - Loss: 0.5180, Accuracy: 99.65%

Test set: Average loss: 0.5287, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.523120
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.514570
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.517350
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.511227
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515607
Train Epoch: 60 - Loss: 0.5176, Accuracy: 99.67%

Test set: Average loss: 0.5277, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.522593
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.516768
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.512272
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.520923
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.519898
Train Epoch: 61 - Loss: 0.5171, Accuracy: 99.66%

Test set: Average loss: 0.5293, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.529719
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.513610
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.510307
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.509078
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.518545
Train Epoch: 62 - Loss: 0.5167, Accuracy: 99.70%

Test set: Average loss: 0.5276, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.507527
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506883
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.516451
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.509097
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.518733
Train Epoch: 63 - Loss: 0.5163, Accuracy: 99.71%

Test set: Average loss: 0.5271, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.510766
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.516175
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.510185
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.530653
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.510097
Train Epoch: 64 - Loss: 0.5159, Accuracy: 99.72%

Test set: Average loss: 0.5272, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.521155
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.531210
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.519319
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.526999
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508452
Train Epoch: 65 - Loss: 0.5156, Accuracy: 99.72%

Test set: Average loss: 0.5267, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.511531
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.524205
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.506693
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.523572
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.512482
Train Epoch: 66 - Loss: 0.5154, Accuracy: 99.73%

Test set: Average loss: 0.5267, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.541431
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.515095
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.517591
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.512515
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509275
Train Epoch: 67 - Loss: 0.5151, Accuracy: 99.75%

Test set: Average loss: 0.5268, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.511634
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.527997
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.526480
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.522178
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.518585
Train Epoch: 68 - Loss: 0.5148, Accuracy: 99.77%

Test set: Average loss: 0.5264, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.508386
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.515261
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.510192
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.508894
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.519906
Train Epoch: 69 - Loss: 0.5147, Accuracy: 99.77%

Test set: Average loss: 0.5265, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.504798
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.510273
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.516344
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.508612
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.515350
Train Epoch: 70 - Loss: 0.5145, Accuracy: 99.76%

Test set: Average loss: 0.5265, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508543
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.505427
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.508741
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.514336
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.517411
Train Epoch: 71 - Loss: 0.5144, Accuracy: 99.77%

Test set: Average loss: 0.5264, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509173
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.512679
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.507789
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512534
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.510812
Train Epoch: 72 - Loss: 0.5143, Accuracy: 99.78%

Test set: Average loss: 0.5262, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.514068
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.527294
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.513671
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.512958
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.515524
Train Epoch: 73 - Loss: 0.5142, Accuracy: 99.78%

Test set: Average loss: 0.5263, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.509780
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.511212
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.523738
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.522194
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.530675
Train Epoch: 74 - Loss: 0.5142, Accuracy: 99.78%

Test set: Average loss: 0.5262, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.514373
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.507907
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.519348
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.510040
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.518468
Train Epoch: 75 - Loss: 0.5141, Accuracy: 99.78%

Test set: Average loss: 0.5262, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.514919
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.519328
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.514538
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.509365
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.508009
Train Epoch: 76 - Loss: 0.5160, Accuracy: 99.70%

Test set: Average loss: 0.5272, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.509592
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.512563
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.512240
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511362
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.527011
Train Epoch: 77 - Loss: 0.5154, Accuracy: 99.72%

Test set: Average loss: 0.5267, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.509946
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.516366
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.531137
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.518215
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.530203
Train Epoch: 78 - Loss: 0.5151, Accuracy: 99.73%

Test set: Average loss: 0.5269, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.508758
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.511643
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.510838
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.506145
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.514378
Train Epoch: 79 - Loss: 0.5146, Accuracy: 99.74%

Test set: Average loss: 0.5258, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.510800
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.508581
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510254
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.507959
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.545097
Train Epoch: 80 - Loss: 0.5142, Accuracy: 99.75%

Test set: Average loss: 0.5255, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.505047
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511434
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.508273
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.507945
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.506246
Train Epoch: 81 - Loss: 0.5138, Accuracy: 99.78%

Test set: Average loss: 0.5256, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.515578
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.516321
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.519283
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.509617
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.507403
Train Epoch: 82 - Loss: 0.5135, Accuracy: 99.76%

Test set: Average loss: 0.5265, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.519278
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.511029
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.513035
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.508395
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.506451
Train Epoch: 83 - Loss: 0.5130, Accuracy: 99.78%

Test set: Average loss: 0.5255, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509533
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.515152
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.533771
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509834
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.510332
Train Epoch: 84 - Loss: 0.5127, Accuracy: 99.80%

Test set: Average loss: 0.5248, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.507148
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.524010
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.517666
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.504181
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515560
Train Epoch: 85 - Loss: 0.5123, Accuracy: 99.81%

Test set: Average loss: 0.5255, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510394
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.543597
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.528590
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.521831
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.509256
Train Epoch: 86 - Loss: 0.5121, Accuracy: 99.82%

Test set: Average loss: 0.5246, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.508929
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.509049
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.507335
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505839
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.512897
Train Epoch: 87 - Loss: 0.5117, Accuracy: 99.84%

Test set: Average loss: 0.5249, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.507427
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.506964
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.517263
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.530404
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.508346
Train Epoch: 88 - Loss: 0.5115, Accuracy: 99.84%

Test set: Average loss: 0.5246, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.510997
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.511129
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.503531
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.511020
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.509890
Train Epoch: 89 - Loss: 0.5113, Accuracy: 99.84%

Test set: Average loss: 0.5244, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.507378
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.525380
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.504664
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.511102
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507712
Train Epoch: 90 - Loss: 0.5111, Accuracy: 99.85%

Test set: Average loss: 0.5245, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504749
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510754
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.507163
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511028
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.510117
Train Epoch: 91 - Loss: 0.5108, Accuracy: 99.86%

Test set: Average loss: 0.5244, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.506284
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.515335
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.505341
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.526774
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.508419
Train Epoch: 92 - Loss: 0.5107, Accuracy: 99.86%

Test set: Average loss: 0.5242, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.508613
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.506804
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.512048
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511180
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.512118
Train Epoch: 93 - Loss: 0.5105, Accuracy: 99.86%

Test set: Average loss: 0.5241, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505325
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504636
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.506254
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.506384
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.505385
Train Epoch: 94 - Loss: 0.5104, Accuracy: 99.87%

Test set: Average loss: 0.5241, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506997
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.505928
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506268
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.510525
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.506756
Train Epoch: 95 - Loss: 0.5103, Accuracy: 99.87%

Test set: Average loss: 0.5240, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.506681
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508740
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.504579
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.507871
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511605
Train Epoch: 96 - Loss: 0.5102, Accuracy: 99.87%

Test set: Average loss: 0.5240, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505779
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.506358
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.505146
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.517019
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507591
Train Epoch: 97 - Loss: 0.5102, Accuracy: 99.88%

Test set: Average loss: 0.5239, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506650
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.518074
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.526024
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511598
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.510058
Train Epoch: 98 - Loss: 0.5101, Accuracy: 99.88%

Test set: Average loss: 0.5240, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.505333
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.504942
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510687
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.515325
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503697
Train Epoch: 99 - Loss: 0.5101, Accuracy: 99.88%

Test set: Average loss: 0.5239, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506790
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.505682
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.504386
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.511181
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505474
Train Epoch: 100 - Loss: 0.5101, Accuracy: 99.88%

Test set: Average loss: 0.5240, Accuracy: 9920/10000 (99.20%)


Training completed. Best Accuracy: 99.21%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.311370
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.290101
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.378641
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.970067
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.874331
Train Epoch: 1 - Loss: 1.4460, Accuracy: 58.78%

Test set: Average loss: 0.7702, Accuracy: 9166/10000 (91.66%)

New best model saved with accuracy: 91.66%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.724934
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.730921
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.699963
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.737648
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.696071
Train Epoch: 2 - Loss: 0.7107, Accuracy: 93.49%

Test set: Average loss: 0.6551, Accuracy: 9552/10000 (95.52%)

New best model saved with accuracy: 95.52%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.679010
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.676667
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.611598
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.650924
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.644678
Train Epoch: 3 - Loss: 0.6374, Accuracy: 95.93%

Test set: Average loss: 0.6096, Accuracy: 9692/10000 (96.92%)

New best model saved with accuracy: 96.92%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.580563
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.614418
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.583411
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.549912
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.585278
Train Epoch: 4 - Loss: 0.6051, Accuracy: 96.94%

Test set: Average loss: 0.5926, Accuracy: 9736/10000 (97.36%)

New best model saved with accuracy: 97.36%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.574385
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.579013
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.564003
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.611479
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.598309
Train Epoch: 5 - Loss: 0.5870, Accuracy: 97.52%

Test set: Average loss: 0.5777, Accuracy: 9774/10000 (97.74%)

New best model saved with accuracy: 97.74%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.562242
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.606013
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.557688
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.562776
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.556440
Train Epoch: 6 - Loss: 0.5756, Accuracy: 97.83%

Test set: Average loss: 0.5705, Accuracy: 9780/10000 (97.80%)

New best model saved with accuracy: 97.80%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.584362
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.551842
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.553416
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.574619
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.563898
Train Epoch: 7 - Loss: 0.5679, Accuracy: 98.08%

Test set: Average loss: 0.5655, Accuracy: 9803/10000 (98.03%)

New best model saved with accuracy: 98.03%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.560844
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.567810
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.539052
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.560162
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.567226
Train Epoch: 8 - Loss: 0.5617, Accuracy: 98.27%

Test set: Average loss: 0.5644, Accuracy: 9809/10000 (98.09%)

New best model saved with accuracy: 98.09%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.562536
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.552254
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.540461
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.551188
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.554803
Train Epoch: 9 - Loss: 0.5574, Accuracy: 98.40%

Test set: Average loss: 0.5615, Accuracy: 9812/10000 (98.12%)

New best model saved with accuracy: 98.12%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.565994
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.567072
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.529132
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.525931
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.532314
Train Epoch: 10 - Loss: 0.5537, Accuracy: 98.50%

Test set: Average loss: 0.5553, Accuracy: 9821/10000 (98.21%)

New best model saved with accuracy: 98.21%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.566589
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.536298
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.531782
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.557368
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.550515
Train Epoch: 11 - Loss: 0.5507, Accuracy: 98.58%

Test set: Average loss: 0.5514, Accuracy: 9839/10000 (98.39%)

New best model saved with accuracy: 98.39%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.544140
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.535086
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.554514
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.542551
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.529235
Train Epoch: 12 - Loss: 0.5483, Accuracy: 98.64%

Test set: Average loss: 0.5507, Accuracy: 9840/10000 (98.40%)

New best model saved with accuracy: 98.40%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.519343
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.551193
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.558567
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.531804
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.544724
Train Epoch: 13 - Loss: 0.5460, Accuracy: 98.72%

Test set: Average loss: 0.5492, Accuracy: 9849/10000 (98.49%)

New best model saved with accuracy: 98.49%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.533429
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.533245
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.543059
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.564638
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.525375
Train Epoch: 14 - Loss: 0.5444, Accuracy: 98.80%

Test set: Average loss: 0.5473, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.542249
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.525053
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.535903
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.532362
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.542504
Train Epoch: 15 - Loss: 0.5429, Accuracy: 98.84%

Test set: Average loss: 0.5466, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.538445
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.542581
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.540038
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.544288
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.527340
Train Epoch: 16 - Loss: 0.5417, Accuracy: 98.87%

Test set: Average loss: 0.5470, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.545727
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.531094
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.536348
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.540764
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.570476
Train Epoch: 17 - Loss: 0.5407, Accuracy: 98.91%

Test set: Average loss: 0.5451, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.540398
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.533173
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.539997
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.544420
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.548480
Train Epoch: 18 - Loss: 0.5399, Accuracy: 98.93%

Test set: Average loss: 0.5443, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.517210
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.579122
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.531836
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.558140
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.514224
Train Epoch: 19 - Loss: 0.5392, Accuracy: 98.96%

Test set: Average loss: 0.5439, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.531547
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.531183
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.545649
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.531486
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.537450
Train Epoch: 20 - Loss: 0.5386, Accuracy: 98.97%

Test set: Average loss: 0.5436, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.522298
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.544264
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.524799
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.517527
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.538018
Train Epoch: 21 - Loss: 0.5383, Accuracy: 99.00%

Test set: Average loss: 0.5435, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.537093
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.539254
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.545735
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.523421
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.525336
Train Epoch: 22 - Loss: 0.5380, Accuracy: 98.99%

Test set: Average loss: 0.5434, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.543092
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.553830
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.531953
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.523092
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.555635
Train Epoch: 23 - Loss: 0.5378, Accuracy: 99.00%

Test set: Average loss: 0.5432, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.526011
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.540489
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.516017
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.531473
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.528573
Train Epoch: 24 - Loss: 0.5376, Accuracy: 99.01%

Test set: Average loss: 0.5432, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.554693
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.533892
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.569502
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.544309
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.550293
Train Epoch: 25 - Loss: 0.5375, Accuracy: 99.01%

Test set: Average loss: 0.5432, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.538485
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.540526
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.567249
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.535308
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.568653
Train Epoch: 26 - Loss: 0.5400, Accuracy: 98.89%

Test set: Average loss: 0.5445, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.557217
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.518322
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.523101
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.526633
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.552799
Train Epoch: 27 - Loss: 0.5380, Accuracy: 98.99%

Test set: Average loss: 0.5421, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.532244
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.528294
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.520358
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.544759
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.528792
Train Epoch: 28 - Loss: 0.5363, Accuracy: 98.99%

Test set: Average loss: 0.5452, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.515979
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.520917
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.510857
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522916
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.532351
Train Epoch: 29 - Loss: 0.5345, Accuracy: 99.09%

Test set: Average loss: 0.5391, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.552586
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.524524
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.542301
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.531308
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.538559
Train Epoch: 30 - Loss: 0.5329, Accuracy: 99.14%

Test set: Average loss: 0.5392, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.521123
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.520034
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.539309
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.526937
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.543212
Train Epoch: 31 - Loss: 0.5316, Accuracy: 99.19%

Test set: Average loss: 0.5365, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.553105
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.564583
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.543809
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515536
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.518935
Train Epoch: 32 - Loss: 0.5302, Accuracy: 99.19%

Test set: Average loss: 0.5368, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.517893
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.535415
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.525052
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.510568
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.516579
Train Epoch: 33 - Loss: 0.5290, Accuracy: 99.25%

Test set: Average loss: 0.5362, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.527763
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.531348
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.524915
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.525729
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.515584
Train Epoch: 34 - Loss: 0.5280, Accuracy: 99.28%

Test set: Average loss: 0.5342, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.548703
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.519167
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.511728
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.514522
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.516419
Train Epoch: 35 - Loss: 0.5269, Accuracy: 99.33%

Test set: Average loss: 0.5337, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.508239
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.518729
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.523192
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.561537
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.516728
Train Epoch: 36 - Loss: 0.5262, Accuracy: 99.34%

Test set: Average loss: 0.5343, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.533573
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.514528
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.526641
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.525389
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.516680
Train Epoch: 37 - Loss: 0.5254, Accuracy: 99.39%

Test set: Average loss: 0.5326, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.544026
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.512498
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.520521
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.515593
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.543666
Train Epoch: 38 - Loss: 0.5247, Accuracy: 99.39%

Test set: Average loss: 0.5328, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.521569
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.525303
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.535186
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.527140
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.521386
Train Epoch: 39 - Loss: 0.5241, Accuracy: 99.40%

Test set: Average loss: 0.5321, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.518585
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.526066
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.519687
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.515278
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.513831
Train Epoch: 40 - Loss: 0.5235, Accuracy: 99.44%

Test set: Average loss: 0.5314, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.557000
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.547625
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.518819
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.523725
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.523327
Train Epoch: 41 - Loss: 0.5231, Accuracy: 99.44%

Test set: Average loss: 0.5319, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.509696
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.518440
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.516160
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.520533
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.515971
Train Epoch: 42 - Loss: 0.5227, Accuracy: 99.46%

Test set: Average loss: 0.5313, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.510791
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.523103
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.510842
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.521192
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.506235
Train Epoch: 43 - Loss: 0.5222, Accuracy: 99.48%

Test set: Average loss: 0.5311, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.515553
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.517399
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524473
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.522823
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516597
Train Epoch: 44 - Loss: 0.5220, Accuracy: 99.50%

Test set: Average loss: 0.5306, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.519512
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.510705
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.538361
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.549846
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.515748
Train Epoch: 45 - Loss: 0.5217, Accuracy: 99.50%

Test set: Average loss: 0.5305, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.533586
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.513750
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.511676
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.510362
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.515043
Train Epoch: 46 - Loss: 0.5215, Accuracy: 99.51%

Test set: Average loss: 0.5305, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.543458
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.511840
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.523439
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.508286
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.526618
Train Epoch: 47 - Loss: 0.5214, Accuracy: 99.52%

Test set: Average loss: 0.5304, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.523960
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.514097
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.536465
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.528667
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512837
Train Epoch: 48 - Loss: 0.5213, Accuracy: 99.52%

Test set: Average loss: 0.5305, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.546993
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.509708
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.511950
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.517736
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.530849
Train Epoch: 49 - Loss: 0.5212, Accuracy: 99.52%

Test set: Average loss: 0.5304, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.508872
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.518366
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.514537
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.526142
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.516809
Train Epoch: 50 - Loss: 0.5212, Accuracy: 99.52%

Test set: Average loss: 0.5304, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.523854
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.516772
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.544207
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.513326
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.554625
Train Epoch: 51 - Loss: 0.5234, Accuracy: 99.42%

Test set: Average loss: 0.5316, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.520678
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.544230
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.522738
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.574970
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513842
Train Epoch: 52 - Loss: 0.5227, Accuracy: 99.45%

Test set: Average loss: 0.5306, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.519802
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.530904
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.517035
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.529675
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.524143
Train Epoch: 53 - Loss: 0.5219, Accuracy: 99.46%

Test set: Average loss: 0.5324, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.538734
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.523314
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.510514
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.520657
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.531275
Train Epoch: 54 - Loss: 0.5211, Accuracy: 99.51%

Test set: Average loss: 0.5299, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.508254
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.526328
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.525014
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.525452
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.532667
Train Epoch: 55 - Loss: 0.5204, Accuracy: 99.53%

Test set: Average loss: 0.5293, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.549975
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.506858
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.513958
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.518030
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.508743
Train Epoch: 56 - Loss: 0.5197, Accuracy: 99.54%

Test set: Average loss: 0.5283, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.525531
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.523411
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.521132
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.516605
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.513875
Train Epoch: 57 - Loss: 0.5191, Accuracy: 99.56%

Test set: Average loss: 0.5291, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.526083
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.510549
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.519469
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.513611
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.516875
Train Epoch: 58 - Loss: 0.5185, Accuracy: 99.59%

Test set: Average loss: 0.5280, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.532625
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.510787
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.524393
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.540464
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.510305
Train Epoch: 59 - Loss: 0.5179, Accuracy: 99.61%

Test set: Average loss: 0.5281, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.516286
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.534046
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.518322
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.511674
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.520556
Train Epoch: 60 - Loss: 0.5175, Accuracy: 99.62%

Test set: Average loss: 0.5275, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.509786
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.523114
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.515648
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.507404
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.512533
Train Epoch: 61 - Loss: 0.5171, Accuracy: 99.65%

Test set: Average loss: 0.5268, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.512198
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.506346
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.508539
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.520559
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509303
Train Epoch: 62 - Loss: 0.5165, Accuracy: 99.67%

Test set: Average loss: 0.5274, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.520026
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.516582
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.533119
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.514371
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.509769
Train Epoch: 63 - Loss: 0.5161, Accuracy: 99.65%

Test set: Average loss: 0.5269, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.514016
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.510045
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.510427
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508651
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.510056
Train Epoch: 64 - Loss: 0.5158, Accuracy: 99.68%

Test set: Average loss: 0.5270, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.515487
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.507898
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.510891
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.522513
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508792
Train Epoch: 65 - Loss: 0.5155, Accuracy: 99.69%

Test set: Average loss: 0.5269, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.508823
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.513976
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.543018
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.507456
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.525548
Train Epoch: 66 - Loss: 0.5152, Accuracy: 99.69%

Test set: Average loss: 0.5262, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.524575
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.525014
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.517178
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.514402
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.514985
Train Epoch: 67 - Loss: 0.5149, Accuracy: 99.70%

Test set: Average loss: 0.5265, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.506274
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.525024
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.505317
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.511682
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.513573
Train Epoch: 68 - Loss: 0.5147, Accuracy: 99.71%

Test set: Average loss: 0.5261, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.508687
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.512818
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507477
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.510496
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.507679
Train Epoch: 69 - Loss: 0.5145, Accuracy: 99.72%

Test set: Average loss: 0.5261, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.506397
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.507593
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510816
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.541923
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507141
Train Epoch: 70 - Loss: 0.5144, Accuracy: 99.72%

Test set: Average loss: 0.5260, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.511899
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.519457
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.518984
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.514864
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.511201
Train Epoch: 71 - Loss: 0.5143, Accuracy: 99.72%

Test set: Average loss: 0.5260, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509579
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.511368
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.518847
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.513809
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.518011
Train Epoch: 72 - Loss: 0.5142, Accuracy: 99.74%

Test set: Average loss: 0.5260, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.511882
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.515651
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.518898
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.521953
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.509461
Train Epoch: 73 - Loss: 0.5141, Accuracy: 99.73%

Test set: Average loss: 0.5259, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.505958
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.511089
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.505497
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.505288
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.509028
Train Epoch: 74 - Loss: 0.5141, Accuracy: 99.73%

Test set: Average loss: 0.5259, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.507946
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.521048
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.547205
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.510860
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.505856
Train Epoch: 75 - Loss: 0.5140, Accuracy: 99.74%

Test set: Average loss: 0.5259, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.514174
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.520900
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.513215
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.511855
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.507701
Train Epoch: 76 - Loss: 0.5157, Accuracy: 99.68%

Test set: Average loss: 0.5262, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.513038
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.520028
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.505142
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.514494
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.508973
Train Epoch: 77 - Loss: 0.5156, Accuracy: 99.67%

Test set: Average loss: 0.5294, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.515469
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.517300
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.519523
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509239
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.521177
Train Epoch: 78 - Loss: 0.5150, Accuracy: 99.72%

Test set: Average loss: 0.5256, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.518834
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.508029
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.518300
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.506491
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.509709
Train Epoch: 79 - Loss: 0.5145, Accuracy: 99.70%

Test set: Average loss: 0.5257, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.506486
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.509240
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509141
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.515478
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.507369
Train Epoch: 80 - Loss: 0.5141, Accuracy: 99.72%

Test set: Average loss: 0.5256, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.512701
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.508676
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.509401
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.514946
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.509230
Train Epoch: 81 - Loss: 0.5137, Accuracy: 99.72%

Test set: Average loss: 0.5253, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.509885
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.507311
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.505647
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.508271
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.507440
Train Epoch: 82 - Loss: 0.5132, Accuracy: 99.75%

Test set: Average loss: 0.5256, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.511025
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.511445
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.513532
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.507412
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.522676
Train Epoch: 83 - Loss: 0.5129, Accuracy: 99.76%

Test set: Average loss: 0.5251, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.510220
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.510811
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.507796
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.508291
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.510233
Train Epoch: 84 - Loss: 0.5126, Accuracy: 99.78%

Test set: Average loss: 0.5248, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508865
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.509936
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512145
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.511645
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.520020
Train Epoch: 85 - Loss: 0.5123, Accuracy: 99.78%

Test set: Average loss: 0.5249, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.528022
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.519140
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.520547
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506521
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.508417
Train Epoch: 86 - Loss: 0.5119, Accuracy: 99.81%

Test set: Average loss: 0.5247, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.509899
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.510634
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.536209
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.510002
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.521301
Train Epoch: 87 - Loss: 0.5116, Accuracy: 99.80%

Test set: Average loss: 0.5241, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510138
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507695
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.503742
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512560
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.510468
Train Epoch: 88 - Loss: 0.5114, Accuracy: 99.82%

Test set: Average loss: 0.5244, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507492
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505274
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.506497
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.519831
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.506207
Train Epoch: 89 - Loss: 0.5111, Accuracy: 99.82%

Test set: Average loss: 0.5245, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.507700
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.514249
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.509587
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.517588
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.506211
Train Epoch: 90 - Loss: 0.5109, Accuracy: 99.83%

Test set: Average loss: 0.5240, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.515805
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.516653
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.538300
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.513001
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.518625
Train Epoch: 91 - Loss: 0.5107, Accuracy: 99.83%

Test set: Average loss: 0.5241, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.505940
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.506145
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507577
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.506785
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.515213
Train Epoch: 92 - Loss: 0.5106, Accuracy: 99.84%

Test set: Average loss: 0.5239, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.509102
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.512091
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.507531
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511266
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.516695
Train Epoch: 93 - Loss: 0.5104, Accuracy: 99.84%

Test set: Average loss: 0.5239, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.506775
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.510646
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.509702
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508144
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.512972
Train Epoch: 94 - Loss: 0.5103, Accuracy: 99.85%

Test set: Average loss: 0.5240, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504416
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508400
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.511975
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.509460
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.504001
Train Epoch: 95 - Loss: 0.5102, Accuracy: 99.84%

Test set: Average loss: 0.5237, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.508260
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.513711
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.509829
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.506002
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.520643
Train Epoch: 96 - Loss: 0.5101, Accuracy: 99.85%

Test set: Average loss: 0.5237, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505131
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.505127
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.518559
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.510756
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.506607
Train Epoch: 97 - Loss: 0.5100, Accuracy: 99.85%

Test set: Average loss: 0.5239, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.511985
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.506510
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507738
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.506345
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.512039
Train Epoch: 98 - Loss: 0.5100, Accuracy: 99.85%

Test set: Average loss: 0.5238, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.509513
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.505754
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.535661
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.512632
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.506655
Train Epoch: 99 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5238, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.505762
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.507996
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.504959
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.504185
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505153
Train Epoch: 100 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5238, Accuracy: 9916/10000 (99.16%)


Training completed. Best Accuracy: 99.18%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.300347
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.299014
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.403568
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.877647
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.810248
Train Epoch: 1 - Loss: 1.4607, Accuracy: 58.37%

Test set: Average loss: 0.7585, Accuracy: 9209/10000 (92.09%)

New best model saved with accuracy: 92.09%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.697339
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.694647
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.682343
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.691848
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.677433
Train Epoch: 2 - Loss: 0.7071, Accuracy: 93.69%

Test set: Average loss: 0.6521, Accuracy: 9542/10000 (95.42%)

New best model saved with accuracy: 95.42%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.614302
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.594309
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.614293
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.606459
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.620349
Train Epoch: 3 - Loss: 0.6363, Accuracy: 96.05%

Test set: Average loss: 0.6115, Accuracy: 9679/10000 (96.79%)

New best model saved with accuracy: 96.79%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.633694
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.630864
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.624099
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.589569
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.583930
Train Epoch: 4 - Loss: 0.6049, Accuracy: 96.92%

Test set: Average loss: 0.5939, Accuracy: 9716/10000 (97.16%)

New best model saved with accuracy: 97.16%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.577726
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.583060
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.641876
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.574100
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.567675
Train Epoch: 5 - Loss: 0.5871, Accuracy: 97.50%

Test set: Average loss: 0.5772, Accuracy: 9767/10000 (97.67%)

New best model saved with accuracy: 97.67%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.541502
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.564100
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.591830
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.582812
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.545459
Train Epoch: 6 - Loss: 0.5757, Accuracy: 97.79%

Test set: Average loss: 0.5675, Accuracy: 9788/10000 (97.88%)

New best model saved with accuracy: 97.88%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.577524
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.602521
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.596507
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.554442
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.539757
Train Epoch: 7 - Loss: 0.5683, Accuracy: 97.96%

Test set: Average loss: 0.5616, Accuracy: 9809/10000 (98.09%)

New best model saved with accuracy: 98.09%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.555359
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.585951
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.539409
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.559610
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.583462
Train Epoch: 8 - Loss: 0.5620, Accuracy: 98.22%

Test set: Average loss: 0.5648, Accuracy: 9812/10000 (98.12%)

New best model saved with accuracy: 98.12%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.530433
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.556756
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.527711
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.536573
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.560583
Train Epoch: 9 - Loss: 0.5573, Accuracy: 98.36%

Test set: Average loss: 0.5581, Accuracy: 9819/10000 (98.19%)

New best model saved with accuracy: 98.19%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.558960
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.579141
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.543600
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.545818
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.559070
Train Epoch: 10 - Loss: 0.5534, Accuracy: 98.46%

Test set: Average loss: 0.5527, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.555488
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.555171
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.555892
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.555856
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.543496
Train Epoch: 11 - Loss: 0.5503, Accuracy: 98.59%

Test set: Average loss: 0.5531, Accuracy: 9841/10000 (98.41%)

New best model saved with accuracy: 98.41%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.594630
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.537592
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.583844
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.546456
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.553581
Train Epoch: 12 - Loss: 0.5479, Accuracy: 98.69%

Test set: Average loss: 0.5476, Accuracy: 9850/10000 (98.50%)

New best model saved with accuracy: 98.50%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.534818
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.527359
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.550394
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.541405
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.556111
Train Epoch: 13 - Loss: 0.5460, Accuracy: 98.75%

Test set: Average loss: 0.5757, Accuracy: 9758/10000 (97.58%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.542527
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.564600
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.547907
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.544903
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.553736
Train Epoch: 14 - Loss: 0.5440, Accuracy: 98.81%

Test set: Average loss: 0.5454, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.554664
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.542725
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.549380
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.547484
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.555239
Train Epoch: 15 - Loss: 0.5426, Accuracy: 98.84%

Test set: Average loss: 0.5453, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.554557
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.522699
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.552321
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.535478
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.527311
Train Epoch: 16 - Loss: 0.5412, Accuracy: 98.88%

Test set: Average loss: 0.5432, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.550191
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.557359
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.566400
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.527208
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.552149
Train Epoch: 17 - Loss: 0.5402, Accuracy: 98.94%

Test set: Average loss: 0.5433, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.547452
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.529248
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.531125
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.535194
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.534807
Train Epoch: 18 - Loss: 0.5393, Accuracy: 98.92%

Test set: Average loss: 0.5419, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.541090
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.534488
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.520774
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.512947
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.525358
Train Epoch: 19 - Loss: 0.5386, Accuracy: 98.96%

Test set: Average loss: 0.5422, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.515833
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.567174
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.558370
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.530412
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.537485
Train Epoch: 20 - Loss: 0.5381, Accuracy: 99.00%

Test set: Average loss: 0.5415, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.535803
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.534262
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.542512
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.538355
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.545811
Train Epoch: 21 - Loss: 0.5376, Accuracy: 99.01%

Test set: Average loss: 0.5410, Accuracy: 9865/10000 (98.65%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.532678
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.526341
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.518536
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.523433
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.559212
Train Epoch: 22 - Loss: 0.5373, Accuracy: 99.03%

Test set: Average loss: 0.5407, Accuracy: 9866/10000 (98.66%)

New best model saved with accuracy: 98.66%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.536400
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.584105
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.525793
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.523696
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.522904
Train Epoch: 23 - Loss: 0.5371, Accuracy: 99.03%

Test set: Average loss: 0.5409, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.536347
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.546376
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.526356
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.539224
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.541537
Train Epoch: 24 - Loss: 0.5370, Accuracy: 99.04%

Test set: Average loss: 0.5406, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.536123
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.529044
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.537265
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.554198
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.538910
Train Epoch: 25 - Loss: 0.5369, Accuracy: 99.03%

Test set: Average loss: 0.5406, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.551616
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.524721
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.519617
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.542285
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.530457
Train Epoch: 26 - Loss: 0.5423, Accuracy: 98.83%

Test set: Average loss: 0.5433, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.554047
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.515533
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.518769
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.549027
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.563664
Train Epoch: 27 - Loss: 0.5381, Accuracy: 98.95%

Test set: Average loss: 0.5401, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.526636
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.517130
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.561289
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.522400
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.526977
Train Epoch: 28 - Loss: 0.5364, Accuracy: 99.02%

Test set: Average loss: 0.5422, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.547982
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.537512
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.534790
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.531202
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.550336
Train Epoch: 29 - Loss: 0.5345, Accuracy: 99.07%

Test set: Average loss: 0.5372, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.546706
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.517747
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.570443
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.518642
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.521103
Train Epoch: 30 - Loss: 0.5329, Accuracy: 99.13%

Test set: Average loss: 0.5362, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.527701
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.541443
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.538936
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.521513
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.516471
Train Epoch: 31 - Loss: 0.5315, Accuracy: 99.16%

Test set: Average loss: 0.5364, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.530504
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.522934
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.545167
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.524770
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.543540
Train Epoch: 32 - Loss: 0.5303, Accuracy: 99.18%

Test set: Average loss: 0.5349, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.535676
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.549215
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.521666
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.545417
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.527981
Train Epoch: 33 - Loss: 0.5291, Accuracy: 99.25%

Test set: Average loss: 0.5337, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.588018
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.524042
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.526311
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.527918
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.522184
Train Epoch: 34 - Loss: 0.5278, Accuracy: 99.28%

Test set: Average loss: 0.5333, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.527820
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.536435
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.512511
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.517632
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.534224
Train Epoch: 35 - Loss: 0.5269, Accuracy: 99.30%

Test set: Average loss: 0.5323, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.508159
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.512814
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.543927
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.519517
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.537630
Train Epoch: 36 - Loss: 0.5262, Accuracy: 99.36%

Test set: Average loss: 0.5319, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.539224
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.564549
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.521938
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.549924
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.530849
Train Epoch: 37 - Loss: 0.5253, Accuracy: 99.39%

Test set: Average loss: 0.5309, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.512420
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.529584
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.515703
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.550436
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.524796
Train Epoch: 38 - Loss: 0.5247, Accuracy: 99.38%

Test set: Average loss: 0.5306, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.513100
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.534660
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.527486
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.507219
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515644
Train Epoch: 39 - Loss: 0.5241, Accuracy: 99.43%

Test set: Average loss: 0.5308, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.509255
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.517919
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.534427
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.520753
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.526314
Train Epoch: 40 - Loss: 0.5234, Accuracy: 99.45%

Test set: Average loss: 0.5303, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.522869
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.525582
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.510711
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.510234
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.520892
Train Epoch: 41 - Loss: 0.5230, Accuracy: 99.45%

Test set: Average loss: 0.5308, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.535721
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.525384
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.526585
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.510780
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.519758
Train Epoch: 42 - Loss: 0.5225, Accuracy: 99.47%

Test set: Average loss: 0.5299, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.510604
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522295
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.530201
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.532072
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.511330
Train Epoch: 43 - Loss: 0.5222, Accuracy: 99.49%

Test set: Average loss: 0.5294, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.527561
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.522212
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.511593
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.511364
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.545046
Train Epoch: 44 - Loss: 0.5219, Accuracy: 99.50%

Test set: Average loss: 0.5295, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.515028
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.525320
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.520042
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.515216
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.520542
Train Epoch: 45 - Loss: 0.5217, Accuracy: 99.52%

Test set: Average loss: 0.5294, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515074
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.507379
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.508881
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.534668
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.526782
Train Epoch: 46 - Loss: 0.5215, Accuracy: 99.51%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.517882
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.514839
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.526421
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.514530
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.516180
Train Epoch: 47 - Loss: 0.5213, Accuracy: 99.53%

Test set: Average loss: 0.5293, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.510948
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.527945
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.522399
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.529774
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.515438
Train Epoch: 48 - Loss: 0.5212, Accuracy: 99.53%

Test set: Average loss: 0.5291, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.533259
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.518576
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.523906
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.512479
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.513452
Train Epoch: 49 - Loss: 0.5211, Accuracy: 99.53%

Test set: Average loss: 0.5291, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.514322
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.520116
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.527051
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.513499
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.524078
Train Epoch: 50 - Loss: 0.5211, Accuracy: 99.54%

Test set: Average loss: 0.5291, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.515838
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.511192
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.517618
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.508548
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.513067
Train Epoch: 51 - Loss: 0.5238, Accuracy: 99.43%

Test set: Average loss: 0.5307, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.515792
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.522449
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.528993
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.520593
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.549791
Train Epoch: 52 - Loss: 0.5226, Accuracy: 99.48%

Test set: Average loss: 0.5307, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.511668
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.520087
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.522439
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.524967
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.524899
Train Epoch: 53 - Loss: 0.5220, Accuracy: 99.49%

Test set: Average loss: 0.5428, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.550623
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.512493
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.524114
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.531378
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.524167
Train Epoch: 54 - Loss: 0.5213, Accuracy: 99.52%

Test set: Average loss: 0.5290, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.545045
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.525397
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.533166
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.519613
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.519035
Train Epoch: 55 - Loss: 0.5209, Accuracy: 99.53%

Test set: Average loss: 0.5280, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.520350
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.516395
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.539002
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.529191
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.534215
Train Epoch: 56 - Loss: 0.5199, Accuracy: 99.56%

Test set: Average loss: 0.5291, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.511230
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511966
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.515794
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.506489
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.529800
Train Epoch: 57 - Loss: 0.5194, Accuracy: 99.56%

Test set: Average loss: 0.5299, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.546387
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.509945
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.515899
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.514804
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.506192
Train Epoch: 58 - Loss: 0.5186, Accuracy: 99.59%

Test set: Average loss: 0.5284, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.525593
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.521651
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.515136
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.513024
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.519610
Train Epoch: 59 - Loss: 0.5182, Accuracy: 99.61%

Test set: Average loss: 0.5276, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.520025
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.515244
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.518798
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.557719
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.517867
Train Epoch: 60 - Loss: 0.5176, Accuracy: 99.64%

Test set: Average loss: 0.5268, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.507574
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.537669
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.509664
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.515289
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.514070
Train Epoch: 61 - Loss: 0.5173, Accuracy: 99.65%

Test set: Average loss: 0.5272, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.520490
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519730
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.511926
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.507663
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.532759
Train Epoch: 62 - Loss: 0.5168, Accuracy: 99.64%

Test set: Average loss: 0.5264, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.511042
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.508322
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.512773
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510095
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.504738
Train Epoch: 63 - Loss: 0.5164, Accuracy: 99.67%

Test set: Average loss: 0.5265, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.511914
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.510183
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.509508
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.509190
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.511340
Train Epoch: 64 - Loss: 0.5160, Accuracy: 99.70%

Test set: Average loss: 0.5266, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.508915
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.516304
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.508524
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.509181
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506056
Train Epoch: 65 - Loss: 0.5157, Accuracy: 99.69%

Test set: Average loss: 0.5257, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.517789
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.516962
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.518414
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.517748
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.513126
Train Epoch: 66 - Loss: 0.5154, Accuracy: 99.71%

Test set: Average loss: 0.5260, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.505650
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.520806
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.522773
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.512771
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513627
Train Epoch: 67 - Loss: 0.5152, Accuracy: 99.71%

Test set: Average loss: 0.5259, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.529944
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.512277
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.516777
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.511884
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.513121
Train Epoch: 68 - Loss: 0.5150, Accuracy: 99.72%

Test set: Average loss: 0.5250, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.511610
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.516930
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.516322
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.510027
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.524434
Train Epoch: 69 - Loss: 0.5147, Accuracy: 99.72%

Test set: Average loss: 0.5252, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.523475
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.512737
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.520263
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512857
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.514922
Train Epoch: 70 - Loss: 0.5146, Accuracy: 99.73%

Test set: Average loss: 0.5251, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.512863
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.506264
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.507976
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.525619
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.519207
Train Epoch: 71 - Loss: 0.5145, Accuracy: 99.73%

Test set: Average loss: 0.5253, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.515436
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.541668
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.518742
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.520353
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.513539
Train Epoch: 72 - Loss: 0.5144, Accuracy: 99.74%

Test set: Average loss: 0.5252, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.528252
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.514981
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508525
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.519366
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510236
Train Epoch: 73 - Loss: 0.5143, Accuracy: 99.75%

Test set: Average loss: 0.5252, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.508339
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.511984
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.505381
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.511384
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.505709
Train Epoch: 74 - Loss: 0.5143, Accuracy: 99.75%

Test set: Average loss: 0.5252, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.509053
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.529016
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.511692
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508960
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.509426
Train Epoch: 75 - Loss: 0.5142, Accuracy: 99.75%

Test set: Average loss: 0.5251, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.504408
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.506058
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.514875
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.507922
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.508120
Train Epoch: 76 - Loss: 0.5159, Accuracy: 99.68%

Test set: Average loss: 0.5260, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511295
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.515090
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.512617
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.519867
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.507315
Train Epoch: 77 - Loss: 0.5159, Accuracy: 99.67%

Test set: Average loss: 0.5257, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.508091
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.513079
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.514913
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.511393
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.508533
Train Epoch: 78 - Loss: 0.5153, Accuracy: 99.71%

Test set: Average loss: 0.5264, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.509263
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.516440
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.507685
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.508730
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.514109
Train Epoch: 79 - Loss: 0.5148, Accuracy: 99.71%

Test set: Average loss: 0.5264, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.513050
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.521075
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.518669
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.512707
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.525093
Train Epoch: 80 - Loss: 0.5144, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.546548
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.513260
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.504664
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.526108
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.507243
Train Epoch: 81 - Loss: 0.5140, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508356
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.510064
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.521648
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.515675
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.507076
Train Epoch: 82 - Loss: 0.5137, Accuracy: 99.73%

Test set: Average loss: 0.5246, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.522099
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.517176
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.505659
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.507851
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.510010
Train Epoch: 83 - Loss: 0.5132, Accuracy: 99.77%

Test set: Average loss: 0.5255, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.513481
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509369
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.510201
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.512465
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508293
Train Epoch: 84 - Loss: 0.5128, Accuracy: 99.78%

Test set: Average loss: 0.5244, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508132
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.503666
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.509328
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.507241
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515083
Train Epoch: 85 - Loss: 0.5126, Accuracy: 99.78%

Test set: Average loss: 0.5254, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.507862
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.513282
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.509431
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.507693
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.516077
Train Epoch: 86 - Loss: 0.5123, Accuracy: 99.79%

Test set: Average loss: 0.5242, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.508317
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.518610
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.504956
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505747
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.507131
Train Epoch: 87 - Loss: 0.5120, Accuracy: 99.80%

Test set: Average loss: 0.5246, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.522016
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.512586
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.505992
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512333
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505362
Train Epoch: 88 - Loss: 0.5117, Accuracy: 99.82%

Test set: Average loss: 0.5239, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.508008
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.512482
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.509271
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.508160
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.507115
Train Epoch: 89 - Loss: 0.5114, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.503469
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.511779
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.511896
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.519829
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.525631
Train Epoch: 90 - Loss: 0.5112, Accuracy: 99.83%

Test set: Average loss: 0.5235, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.510922
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510101
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509299
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.536020
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.505721
Train Epoch: 91 - Loss: 0.5111, Accuracy: 99.83%

Test set: Average loss: 0.5237, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.510026
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.507848
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.523088
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.506880
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.509778
Train Epoch: 92 - Loss: 0.5109, Accuracy: 99.84%

Test set: Average loss: 0.5239, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506475
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.511447
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.506953
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.506325
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.512372
Train Epoch: 93 - Loss: 0.5107, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.512969
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509385
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507367
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.506919
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.507792
Train Epoch: 94 - Loss: 0.5106, Accuracy: 99.85%

Test set: Average loss: 0.5235, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504752
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.510750
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507952
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.507474
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509949
Train Epoch: 95 - Loss: 0.5105, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507815
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.504958
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508857
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.528972
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.506705
Train Epoch: 96 - Loss: 0.5104, Accuracy: 99.86%

Test set: Average loss: 0.5234, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.508653
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.514740
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507394
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.518604
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.510921
Train Epoch: 97 - Loss: 0.5104, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.511754
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.507376
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.510759
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.507835
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.527606
Train Epoch: 98 - Loss: 0.5103, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.508906
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.512852
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.507121
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.515950
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.508075
Train Epoch: 99 - Loss: 0.5103, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506172
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.534893
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.515464
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.506150
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.509228
Train Epoch: 100 - Loss: 0.5103, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9911/10000 (99.11%)


Training completed. Best Accuracy: 99.16%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.311535
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.287930
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.210326
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.842149
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.761495
Train Epoch: 1 - Loss: 1.4094, Accuracy: 62.29%

Test set: Average loss: 0.7680, Accuracy: 9139/10000 (91.39%)

New best model saved with accuracy: 91.39%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.747633
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.771987
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.709957
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.677599
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.629812
Train Epoch: 2 - Loss: 0.7097, Accuracy: 93.50%

Test set: Average loss: 0.6533, Accuracy: 9551/10000 (95.51%)

New best model saved with accuracy: 95.51%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.671868
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.629210
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.649407
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.647748
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.621640
Train Epoch: 3 - Loss: 0.6352, Accuracy: 96.05%

Test set: Average loss: 0.6086, Accuracy: 9701/10000 (97.01%)

New best model saved with accuracy: 97.01%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.603056
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.587958
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.626033
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.649897
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.584914
Train Epoch: 4 - Loss: 0.6015, Accuracy: 97.08%

Test set: Average loss: 0.5861, Accuracy: 9763/10000 (97.63%)

New best model saved with accuracy: 97.63%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.611975
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.578652
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.597775
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.566845
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.609811
Train Epoch: 5 - Loss: 0.5833, Accuracy: 97.58%

Test set: Average loss: 0.5727, Accuracy: 9814/10000 (98.14%)

New best model saved with accuracy: 98.14%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.552090
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.605389
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.556460
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.579666
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.571051
Train Epoch: 6 - Loss: 0.5720, Accuracy: 97.93%

Test set: Average loss: 0.5634, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.550203
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.576052
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.572233
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.572452
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.558067
Train Epoch: 7 - Loss: 0.5640, Accuracy: 98.22%

Test set: Average loss: 0.5583, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.563187
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.544758
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.559436
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.552259
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.563830
Train Epoch: 8 - Loss: 0.5582, Accuracy: 98.40%

Test set: Average loss: 0.5562, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.542707
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.548769
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.545810
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.524557
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.545382
Train Epoch: 9 - Loss: 0.5540, Accuracy: 98.49%

Test set: Average loss: 0.5512, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.543841
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.553018
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.552897
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.542371
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.574962
Train Epoch: 10 - Loss: 0.5505, Accuracy: 98.59%

Test set: Average loss: 0.5492, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.539954
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.555614
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.536025
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.522818
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.550265
Train Epoch: 11 - Loss: 0.5476, Accuracy: 98.73%

Test set: Average loss: 0.5465, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.528741
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.560346
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.545840
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.564189
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.579293
Train Epoch: 12 - Loss: 0.5452, Accuracy: 98.78%

Test set: Average loss: 0.5453, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.532123
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.599583
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.531286
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.578367
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.522789
Train Epoch: 13 - Loss: 0.5432, Accuracy: 98.84%

Test set: Average loss: 0.5437, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.528694
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.586204
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.536844
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.532277
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.545856
Train Epoch: 14 - Loss: 0.5416, Accuracy: 98.90%

Test set: Average loss: 0.5421, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.538827
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.516576
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.552290
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.527659
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.563185
Train Epoch: 15 - Loss: 0.5401, Accuracy: 98.95%

Test set: Average loss: 0.5417, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.542523
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.526959
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.542447
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.525732
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.525828
Train Epoch: 16 - Loss: 0.5391, Accuracy: 98.99%

Test set: Average loss: 0.5409, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.529592
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.550425
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.520213
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.534786
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.536893
Train Epoch: 17 - Loss: 0.5381, Accuracy: 99.02%

Test set: Average loss: 0.5403, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.534972
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.553420
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.544634
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.552886
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.528540
Train Epoch: 18 - Loss: 0.5373, Accuracy: 99.05%

Test set: Average loss: 0.5396, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.528931
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.534420
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.538569
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.540266
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.541049
Train Epoch: 19 - Loss: 0.5366, Accuracy: 99.08%

Test set: Average loss: 0.5395, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.536073
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.527427
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.546717
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.528387
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.524459
Train Epoch: 20 - Loss: 0.5361, Accuracy: 99.13%

Test set: Average loss: 0.5391, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.546814
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.556222
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.530075
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.519329
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.531096
Train Epoch: 21 - Loss: 0.5357, Accuracy: 99.12%

Test set: Average loss: 0.5393, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.523303
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.517122
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.533747
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.528370
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.541489
Train Epoch: 22 - Loss: 0.5355, Accuracy: 99.13%

Test set: Average loss: 0.5387, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.526101
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.545729
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.524804
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.540460
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.519893
Train Epoch: 23 - Loss: 0.5352, Accuracy: 99.16%

Test set: Average loss: 0.5387, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.519147
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.533817
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.521332
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.559087
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.543024
Train Epoch: 24 - Loss: 0.5351, Accuracy: 99.16%

Test set: Average loss: 0.5386, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.521926
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.531098
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.522196
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.534905
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.527492
Train Epoch: 25 - Loss: 0.5350, Accuracy: 99.16%

Test set: Average loss: 0.5386, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.544859
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.527666
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.534959
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.522817
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.558194
Train Epoch: 26 - Loss: 0.5382, Accuracy: 99.03%

Test set: Average loss: 0.5392, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.525781
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.557189
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.544663
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.531977
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.527122
Train Epoch: 27 - Loss: 0.5359, Accuracy: 99.07%

Test set: Average loss: 0.5385, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.530437
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.585178
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.526586
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.529205
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.525143
Train Epoch: 28 - Loss: 0.5341, Accuracy: 99.15%

Test set: Average loss: 0.5365, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.531412
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.528658
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.548467
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.536845
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.557815
Train Epoch: 29 - Loss: 0.5324, Accuracy: 99.22%

Test set: Average loss: 0.5350, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.530349
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.549682
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.532217
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.522072
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.534090
Train Epoch: 30 - Loss: 0.5307, Accuracy: 99.26%

Test set: Average loss: 0.5346, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.533536
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.520722
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.520855
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.521315
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.563999
Train Epoch: 31 - Loss: 0.5294, Accuracy: 99.30%

Test set: Average loss: 0.5341, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.532613
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.520932
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.516350
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.516411
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.526457
Train Epoch: 32 - Loss: 0.5285, Accuracy: 99.30%

Test set: Average loss: 0.5332, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.530032
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.517504
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.524489
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.524365
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.520470
Train Epoch: 33 - Loss: 0.5271, Accuracy: 99.38%

Test set: Average loss: 0.5318, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.521470
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.513273
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.530541
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.512973
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.531163
Train Epoch: 34 - Loss: 0.5262, Accuracy: 99.40%

Test set: Average loss: 0.5334, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.531405
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.524219
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.520761
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.522282
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.530935
Train Epoch: 35 - Loss: 0.5254, Accuracy: 99.41%

Test set: Average loss: 0.5311, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.525146
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.514959
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.512756
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.533609
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.536059
Train Epoch: 36 - Loss: 0.5245, Accuracy: 99.41%

Test set: Average loss: 0.5305, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.514874
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.528026
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.518446
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.519317
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.555716
Train Epoch: 37 - Loss: 0.5238, Accuracy: 99.47%

Test set: Average loss: 0.5304, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.513954
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.513050
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.523010
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.518607
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.519943
Train Epoch: 38 - Loss: 0.5231, Accuracy: 99.47%

Test set: Average loss: 0.5300, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.515873
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.536441
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.508212
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513341
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.508237
Train Epoch: 39 - Loss: 0.5225, Accuracy: 99.50%

Test set: Average loss: 0.5293, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.538674
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.514034
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.548494
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.511299
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.535558
Train Epoch: 40 - Loss: 0.5220, Accuracy: 99.50%

Test set: Average loss: 0.5295, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.515928
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.531101
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.509988
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.511548
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.525126
Train Epoch: 41 - Loss: 0.5216, Accuracy: 99.52%

Test set: Average loss: 0.5292, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.540760
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.520566
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.522031
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.510351
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.512176
Train Epoch: 42 - Loss: 0.5212, Accuracy: 99.55%

Test set: Average loss: 0.5293, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.523258
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.510208
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.516772
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.560016
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.549265
Train Epoch: 43 - Loss: 0.5209, Accuracy: 99.55%

Test set: Average loss: 0.5287, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.519007
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.527005
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.527911
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.521933
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.515666
Train Epoch: 44 - Loss: 0.5206, Accuracy: 99.59%

Test set: Average loss: 0.5287, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.518507
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.511717
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.506411
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.508686
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.560690
Train Epoch: 45 - Loss: 0.5204, Accuracy: 99.58%

Test set: Average loss: 0.5285, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.539615
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.517821
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.527926
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.537376
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.514695
Train Epoch: 46 - Loss: 0.5202, Accuracy: 99.58%

Test set: Average loss: 0.5284, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.520312
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.510409
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.513449
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.516795
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.561325
Train Epoch: 47 - Loss: 0.5201, Accuracy: 99.58%

Test set: Average loss: 0.5284, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.546512
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.526450
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.524793
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.527057
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.534430
Train Epoch: 48 - Loss: 0.5200, Accuracy: 99.59%

Test set: Average loss: 0.5285, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.524275
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.519625
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.508386
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.528530
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.535915
Train Epoch: 49 - Loss: 0.5199, Accuracy: 99.59%

Test set: Average loss: 0.5284, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.512740
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.509740
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.522798
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.539991
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.509700
Train Epoch: 50 - Loss: 0.5199, Accuracy: 99.59%

Test set: Average loss: 0.5284, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.527003
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.515554
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.521969
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.531482
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.509319
Train Epoch: 51 - Loss: 0.5221, Accuracy: 99.50%

Test set: Average loss: 0.5294, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.524279
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.531481
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.517680
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.518667
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.544291
Train Epoch: 52 - Loss: 0.5213, Accuracy: 99.52%

Test set: Average loss: 0.5286, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.518169
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.518117
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.524404
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.519917
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.512327
Train Epoch: 53 - Loss: 0.5206, Accuracy: 99.53%

Test set: Average loss: 0.5307, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.517807
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.511606
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.525539
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.509693
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517614
Train Epoch: 54 - Loss: 0.5201, Accuracy: 99.56%

Test set: Average loss: 0.5287, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.530235
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.516595
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.530325
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.538050
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.511815
Train Epoch: 55 - Loss: 0.5193, Accuracy: 99.57%

Test set: Average loss: 0.5280, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.511075
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.535737
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.517052
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.517222
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.515643
Train Epoch: 56 - Loss: 0.5186, Accuracy: 99.60%

Test set: Average loss: 0.5296, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514850
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.510587
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.521744
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.544293
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.511901
Train Epoch: 57 - Loss: 0.5181, Accuracy: 99.64%

Test set: Average loss: 0.5266, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.535794
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.508753
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.527092
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.516450
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.516855
Train Epoch: 58 - Loss: 0.5174, Accuracy: 99.64%

Test set: Average loss: 0.5265, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.511571
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.519450
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.515509
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.520125
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.511019
Train Epoch: 59 - Loss: 0.5170, Accuracy: 99.67%

Test set: Average loss: 0.5272, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.514133
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.511424
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.534053
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.509271
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.544045
Train Epoch: 60 - Loss: 0.5165, Accuracy: 99.68%

Test set: Average loss: 0.5267, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.508240
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.515820
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.527302
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.526544
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.516081
Train Epoch: 61 - Loss: 0.5162, Accuracy: 99.69%

Test set: Average loss: 0.5259, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.511516
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519775
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.516894
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.516473
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509537
Train Epoch: 62 - Loss: 0.5157, Accuracy: 99.70%

Test set: Average loss: 0.5257, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.507794
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.520490
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.508504
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.511258
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.512723
Train Epoch: 63 - Loss: 0.5153, Accuracy: 99.70%

Test set: Average loss: 0.5253, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.515944
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.516898
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.518783
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.514989
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.511950
Train Epoch: 64 - Loss: 0.5151, Accuracy: 99.72%

Test set: Average loss: 0.5251, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.538517
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509200
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.519978
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.514052
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508467
Train Epoch: 65 - Loss: 0.5147, Accuracy: 99.73%

Test set: Average loss: 0.5252, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.506694
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.510115
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.519939
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.520388
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.517499
Train Epoch: 66 - Loss: 0.5145, Accuracy: 99.74%

Test set: Average loss: 0.5251, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.514332
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.523498
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.514305
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.507737
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.512268
Train Epoch: 67 - Loss: 0.5143, Accuracy: 99.75%

Test set: Average loss: 0.5256, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.508577
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.512281
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.508007
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.521406
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.522996
Train Epoch: 68 - Loss: 0.5141, Accuracy: 99.77%

Test set: Average loss: 0.5249, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.509687
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.514412
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.510599
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.507783
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.507330
Train Epoch: 69 - Loss: 0.5139, Accuracy: 99.76%

Test set: Average loss: 0.5247, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.508427
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.507016
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.509684
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.523194
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.506499
Train Epoch: 70 - Loss: 0.5137, Accuracy: 99.77%

Test set: Average loss: 0.5248, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.519155
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.526799
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.518996
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.507865
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.508798
Train Epoch: 71 - Loss: 0.5136, Accuracy: 99.78%

Test set: Average loss: 0.5247, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.508204
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.507184
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509311
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.515350
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508629
Train Epoch: 72 - Loss: 0.5136, Accuracy: 99.78%

Test set: Average loss: 0.5247, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.505672
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.506150
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.518006
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.514822
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.517552
Train Epoch: 73 - Loss: 0.5135, Accuracy: 99.78%

Test set: Average loss: 0.5246, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.529696
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508457
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.509509
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.511225
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.519393
Train Epoch: 74 - Loss: 0.5134, Accuracy: 99.78%

Test set: Average loss: 0.5247, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.519022
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.517500
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.507771
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.511854
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506887
Train Epoch: 75 - Loss: 0.5134, Accuracy: 99.79%

Test set: Average loss: 0.5246, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.512409
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.506409
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.511560
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.528238
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.511777
Train Epoch: 76 - Loss: 0.5151, Accuracy: 99.70%

Test set: Average loss: 0.5252, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.514677
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.512462
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.506144
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.507120
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.512190
Train Epoch: 77 - Loss: 0.5147, Accuracy: 99.74%

Test set: Average loss: 0.5254, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.511615
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.531132
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.527695
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.512954
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.508554
Train Epoch: 78 - Loss: 0.5143, Accuracy: 99.73%

Test set: Average loss: 0.5248, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.514411
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.514643
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.509828
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507360
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.507710
Train Epoch: 79 - Loss: 0.5140, Accuracy: 99.73%

Test set: Average loss: 0.5259, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.515437
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.513747
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.508013
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.508186
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.507069
Train Epoch: 80 - Loss: 0.5136, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508100
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.534278
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.506905
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.513642
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.515649
Train Epoch: 81 - Loss: 0.5132, Accuracy: 99.76%

Test set: Average loss: 0.5251, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.515232
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.507001
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.519059
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.510558
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.507597
Train Epoch: 82 - Loss: 0.5128, Accuracy: 99.79%

Test set: Average loss: 0.5242, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.507500
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512410
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.513692
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511848
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.506219
Train Epoch: 83 - Loss: 0.5124, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.507727
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509830
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.521051
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.506894
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509459
Train Epoch: 84 - Loss: 0.5121, Accuracy: 99.80%

Test set: Average loss: 0.5246, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.521262
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.510174
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512708
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.511576
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.506422
Train Epoch: 85 - Loss: 0.5118, Accuracy: 99.82%

Test set: Average loss: 0.5238, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.507646
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.505480
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511327
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.513792
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.507391
Train Epoch: 86 - Loss: 0.5115, Accuracy: 99.83%

Test set: Average loss: 0.5240, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.505294
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506045
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.506616
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509233
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.512115
Train Epoch: 87 - Loss: 0.5112, Accuracy: 99.83%

Test set: Average loss: 0.5238, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.509535
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.505769
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.517626
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.506199
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.518128
Train Epoch: 88 - Loss: 0.5110, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.508632
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.509445
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.506749
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.505445
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.508447
Train Epoch: 89 - Loss: 0.5108, Accuracy: 99.86%

Test set: Average loss: 0.5233, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508127
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.506877
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513274
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.504409
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.515918
Train Epoch: 90 - Loss: 0.5106, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.504421
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.504518
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510382
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507075
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506000
Train Epoch: 91 - Loss: 0.5104, Accuracy: 99.86%

Test set: Average loss: 0.5233, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.504903
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504988
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.512237
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.510954
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.504876
Train Epoch: 92 - Loss: 0.5103, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506136
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508374
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511914
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.510055
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.509913
Train Epoch: 93 - Loss: 0.5101, Accuracy: 99.88%

Test set: Average loss: 0.5231, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505506
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.511825
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508228
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.510505
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.507683
Train Epoch: 94 - Loss: 0.5100, Accuracy: 99.87%

Test set: Average loss: 0.5232, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.508447
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.528523
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507024
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.516121
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508149
Train Epoch: 95 - Loss: 0.5099, Accuracy: 99.88%

Test set: Average loss: 0.5230, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.521393
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.514060
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507755
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.526455
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.509446
Train Epoch: 96 - Loss: 0.5099, Accuracy: 99.88%

Test set: Average loss: 0.5229, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506797
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509202
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.515418
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.515475
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.509833
Train Epoch: 97 - Loss: 0.5098, Accuracy: 99.88%

Test set: Average loss: 0.5228, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506588
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509325
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505981
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.503774
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.507521
Train Epoch: 98 - Loss: 0.5098, Accuracy: 99.88%

Test set: Average loss: 0.5229, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.508800
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.506493
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.508145
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.507230
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.504910
Train Epoch: 99 - Loss: 0.5097, Accuracy: 99.88%

Test set: Average loss: 0.5228, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506006
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.507460
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.511735
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.506379
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.507356
Train Epoch: 100 - Loss: 0.5097, Accuracy: 99.88%

Test set: Average loss: 0.5228, Accuracy: 9924/10000 (99.24%)


Training completed. Best Accuracy: 99.24%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.303465
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.297721
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.436758
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.921689
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.834486
Train Epoch: 1 - Loss: 1.4739, Accuracy: 57.85%

Test set: Average loss: 0.7751, Accuracy: 9126/10000 (91.26%)

New best model saved with accuracy: 91.26%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.810456
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.747870
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.718987
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.671168
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.657304
Train Epoch: 2 - Loss: 0.7140, Accuracy: 93.38%

Test set: Average loss: 0.6595, Accuracy: 9522/10000 (95.22%)

New best model saved with accuracy: 95.22%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.643748
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.649193
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.680753
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.646354
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.603614
Train Epoch: 3 - Loss: 0.6349, Accuracy: 96.06%

Test set: Average loss: 0.6144, Accuracy: 9681/10000 (96.81%)

New best model saved with accuracy: 96.81%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.671489
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.612582
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.597839
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.562452
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.602571
Train Epoch: 4 - Loss: 0.6007, Accuracy: 97.15%

Test set: Average loss: 0.5858, Accuracy: 9759/10000 (97.59%)

New best model saved with accuracy: 97.59%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.570309
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.583110
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.600396
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.615141
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.579558
Train Epoch: 5 - Loss: 0.5824, Accuracy: 97.66%

Test set: Average loss: 0.5757, Accuracy: 9789/10000 (97.89%)

New best model saved with accuracy: 97.89%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.560441
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.556934
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.531550
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.597073
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.577689
Train Epoch: 6 - Loss: 0.5708, Accuracy: 98.02%

Test set: Average loss: 0.5656, Accuracy: 9808/10000 (98.08%)

New best model saved with accuracy: 98.08%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.594046
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.575216
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.545185
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.523925
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.545752
Train Epoch: 7 - Loss: 0.5630, Accuracy: 98.26%

Test set: Average loss: 0.5601, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.572483
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.578909
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.530544
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.582869
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.558579
Train Epoch: 8 - Loss: 0.5575, Accuracy: 98.40%

Test set: Average loss: 0.5532, Accuracy: 9841/10000 (98.41%)

New best model saved with accuracy: 98.41%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.546594
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.547472
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.530404
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.584955
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.544906
Train Epoch: 9 - Loss: 0.5529, Accuracy: 98.56%

Test set: Average loss: 0.5506, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.526243
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.550620
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.531695
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.533753
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.549763
Train Epoch: 10 - Loss: 0.5495, Accuracy: 98.65%

Test set: Average loss: 0.5475, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.520120
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.628066
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.534234
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.563950
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.541291
Train Epoch: 11 - Loss: 0.5466, Accuracy: 98.76%

Test set: Average loss: 0.5458, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.527400
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.522144
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.563399
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.543845
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.545293
Train Epoch: 12 - Loss: 0.5443, Accuracy: 98.82%

Test set: Average loss: 0.5441, Accuracy: 9866/10000 (98.66%)

New best model saved with accuracy: 98.66%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.617610
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.541817
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.530433
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.537987
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.550510
Train Epoch: 13 - Loss: 0.5424, Accuracy: 98.85%

Test set: Average loss: 0.5431, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.555010
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.543330
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.528807
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.554729
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.562521
Train Epoch: 14 - Loss: 0.5406, Accuracy: 98.92%

Test set: Average loss: 0.5412, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.543406
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.552318
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.551588
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.558822
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.533258
Train Epoch: 15 - Loss: 0.5394, Accuracy: 98.97%

Test set: Average loss: 0.5406, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.516693
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.517981
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.522575
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.532070
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.539987
Train Epoch: 16 - Loss: 0.5382, Accuracy: 99.01%

Test set: Average loss: 0.5398, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.529618
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.534072
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.522642
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.540027
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.530066
Train Epoch: 17 - Loss: 0.5373, Accuracy: 99.02%

Test set: Average loss: 0.5393, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.534460
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.543859
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.538807
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.541619
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.556771
Train Epoch: 18 - Loss: 0.5365, Accuracy: 99.03%

Test set: Average loss: 0.5383, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.536239
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.526498
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.546124
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.548384
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.561690
Train Epoch: 19 - Loss: 0.5359, Accuracy: 99.06%

Test set: Average loss: 0.5379, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.521262
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.522767
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.551061
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.523572
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.515364
Train Epoch: 20 - Loss: 0.5354, Accuracy: 99.07%

Test set: Average loss: 0.5379, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.543600
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.538831
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.529217
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.521844
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.525776
Train Epoch: 21 - Loss: 0.5350, Accuracy: 99.09%

Test set: Average loss: 0.5376, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.534802
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.537541
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.512844
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.526712
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.552336
Train Epoch: 22 - Loss: 0.5347, Accuracy: 99.10%

Test set: Average loss: 0.5375, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.523528
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.522321
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.544584
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.519994
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.524446
Train Epoch: 23 - Loss: 0.5346, Accuracy: 99.11%

Test set: Average loss: 0.5374, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.532413
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.542696
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.542330
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.523637
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.518179
Train Epoch: 24 - Loss: 0.5344, Accuracy: 99.11%

Test set: Average loss: 0.5373, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.540941
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.537664
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.550480
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.515415
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.518634
Train Epoch: 25 - Loss: 0.5344, Accuracy: 99.11%

Test set: Average loss: 0.5373, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.525405
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.537726
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.531564
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.542472
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.545216
Train Epoch: 26 - Loss: 0.5373, Accuracy: 99.02%

Test set: Average loss: 0.5377, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.540789
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.523376
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.562887
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.546487
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.523215
Train Epoch: 27 - Loss: 0.5353, Accuracy: 99.06%

Test set: Average loss: 0.5363, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.554653
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.523158
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.558584
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.523950
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.530064
Train Epoch: 28 - Loss: 0.5335, Accuracy: 99.12%

Test set: Average loss: 0.5351, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.562794
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.521874
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.528540
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.545363
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.524540
Train Epoch: 29 - Loss: 0.5318, Accuracy: 99.19%

Test set: Average loss: 0.5341, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.534687
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.537400
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.524143
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.528307
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.519121
Train Epoch: 30 - Loss: 0.5306, Accuracy: 99.21%

Test set: Average loss: 0.5347, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.527160
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.512758
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.515112
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.522572
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.526282
Train Epoch: 31 - Loss: 0.5292, Accuracy: 99.24%

Test set: Average loss: 0.5323, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.527285
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.525055
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.533723
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514905
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.543418
Train Epoch: 32 - Loss: 0.5280, Accuracy: 99.29%

Test set: Average loss: 0.5316, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.545995
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.514351
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.544092
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.521734
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.528988
Train Epoch: 33 - Loss: 0.5268, Accuracy: 99.32%

Test set: Average loss: 0.5310, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.516712
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.560110
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.531346
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.546231
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.516299
Train Epoch: 34 - Loss: 0.5260, Accuracy: 99.31%

Test set: Average loss: 0.5302, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.520575
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.520322
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.555682
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.513342
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.516689
Train Epoch: 35 - Loss: 0.5250, Accuracy: 99.36%

Test set: Average loss: 0.5298, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.510671
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.523283
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.511123
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.519686
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.509458
Train Epoch: 36 - Loss: 0.5244, Accuracy: 99.41%

Test set: Average loss: 0.5307, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.530004
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.530760
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.528535
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.514836
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.533409
Train Epoch: 37 - Loss: 0.5236, Accuracy: 99.42%

Test set: Average loss: 0.5295, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.510076
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.542267
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.511117
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.553554
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.524120
Train Epoch: 38 - Loss: 0.5230, Accuracy: 99.44%

Test set: Average loss: 0.5285, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.512634
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.534422
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.507303
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.530649
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.509981
Train Epoch: 39 - Loss: 0.5224, Accuracy: 99.45%

Test set: Average loss: 0.5310, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.552707
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.513004
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.533533
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.525623
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.516249
Train Epoch: 40 - Loss: 0.5220, Accuracy: 99.49%

Test set: Average loss: 0.5282, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.522152
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.533493
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.514308
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.527753
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.515695
Train Epoch: 41 - Loss: 0.5215, Accuracy: 99.49%

Test set: Average loss: 0.5284, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.509648
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.530305
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.555129
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.515411
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.521763
Train Epoch: 42 - Loss: 0.5211, Accuracy: 99.52%

Test set: Average loss: 0.5276, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.511465
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.525477
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.511035
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.527402
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.514963
Train Epoch: 43 - Loss: 0.5208, Accuracy: 99.51%

Test set: Average loss: 0.5273, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.509210
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.529502
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.507120
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.515731
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.545035
Train Epoch: 44 - Loss: 0.5205, Accuracy: 99.54%

Test set: Average loss: 0.5275, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.508941
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.523327
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.536697
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.536654
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.515883
Train Epoch: 45 - Loss: 0.5203, Accuracy: 99.54%

Test set: Average loss: 0.5273, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.512858
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.513648
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.532846
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.520451
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.520803
Train Epoch: 46 - Loss: 0.5201, Accuracy: 99.56%

Test set: Average loss: 0.5272, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.513357
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.526462
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.514634
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.513930
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.528288
Train Epoch: 47 - Loss: 0.5199, Accuracy: 99.56%

Test set: Average loss: 0.5272, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.510060
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.530942
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.509606
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.533308
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.516533
Train Epoch: 48 - Loss: 0.5198, Accuracy: 99.57%

Test set: Average loss: 0.5272, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.512169
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.526794
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.511198
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.528138
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.509991
Train Epoch: 49 - Loss: 0.5198, Accuracy: 99.57%

Test set: Average loss: 0.5272, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513593
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.509348
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.519065
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.515337
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.519147
Train Epoch: 50 - Loss: 0.5197, Accuracy: 99.57%

Test set: Average loss: 0.5272, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.546601
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.515886
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.533830
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.518776
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.506937
Train Epoch: 51 - Loss: 0.5221, Accuracy: 99.46%

Test set: Average loss: 0.5283, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.539859
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.519183
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.509084
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.511372
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.515852
Train Epoch: 52 - Loss: 0.5214, Accuracy: 99.49%

Test set: Average loss: 0.5279, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.522753
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.524167
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.509839
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.535522
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.522250
Train Epoch: 53 - Loss: 0.5207, Accuracy: 99.48%

Test set: Average loss: 0.5267, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.529419
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.538138
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.511727
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.521069
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.531266
Train Epoch: 54 - Loss: 0.5199, Accuracy: 99.51%

Test set: Average loss: 0.5283, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.509265
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.509770
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.521491
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.520141
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.518381
Train Epoch: 55 - Loss: 0.5194, Accuracy: 99.54%

Test set: Average loss: 0.5287, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.513404
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.514880
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.521840
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.535824
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.514739
Train Epoch: 56 - Loss: 0.5187, Accuracy: 99.56%

Test set: Average loss: 0.5266, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.515188
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.514305
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.525325
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.521331
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.527191
Train Epoch: 57 - Loss: 0.5181, Accuracy: 99.59%

Test set: Average loss: 0.5265, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.508126
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.507877
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.507215
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.511521
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.513976
Train Epoch: 58 - Loss: 0.5174, Accuracy: 99.60%

Test set: Average loss: 0.5251, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.511796
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.521934
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.508116
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.512623
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.512005
Train Epoch: 59 - Loss: 0.5170, Accuracy: 99.62%

Test set: Average loss: 0.5253, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.509386
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.511435
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.532525
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.516208
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.521964
Train Epoch: 60 - Loss: 0.5165, Accuracy: 99.64%

Test set: Average loss: 0.5247, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.518182
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.515174
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.508261
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.512461
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.514545
Train Epoch: 61 - Loss: 0.5162, Accuracy: 99.67%

Test set: Average loss: 0.5248, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.507646
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.518409
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.525182
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.520291
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.527951
Train Epoch: 62 - Loss: 0.5157, Accuracy: 99.66%

Test set: Average loss: 0.5248, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.506006
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.526754
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.526262
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.508249
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.511718
Train Epoch: 63 - Loss: 0.5153, Accuracy: 99.70%

Test set: Average loss: 0.5243, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.507865
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.509978
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506297
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.510276
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.511044
Train Epoch: 64 - Loss: 0.5150, Accuracy: 99.70%

Test set: Average loss: 0.5242, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.512133
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.518992
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.514569
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.511815
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.509721
Train Epoch: 65 - Loss: 0.5147, Accuracy: 99.71%

Test set: Average loss: 0.5242, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.512172
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.513423
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508502
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.508964
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.510052
Train Epoch: 66 - Loss: 0.5144, Accuracy: 99.72%

Test set: Average loss: 0.5241, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.517188
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.522502
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.511869
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.517232
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513973
Train Epoch: 67 - Loss: 0.5142, Accuracy: 99.71%

Test set: Average loss: 0.5243, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.507197
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.530701
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.511949
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.507201
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.524905
Train Epoch: 68 - Loss: 0.5140, Accuracy: 99.73%

Test set: Average loss: 0.5241, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512074
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.510988
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.509179
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.515114
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506619
Train Epoch: 69 - Loss: 0.5138, Accuracy: 99.73%

Test set: Average loss: 0.5238, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512836
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.511534
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.513797
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.506913
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.515673
Train Epoch: 70 - Loss: 0.5137, Accuracy: 99.75%

Test set: Average loss: 0.5240, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.535353
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.536763
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.512329
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510605
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.507196
Train Epoch: 71 - Loss: 0.5136, Accuracy: 99.75%

Test set: Average loss: 0.5236, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511897
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.507536
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508657
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.517738
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.513804
Train Epoch: 72 - Loss: 0.5135, Accuracy: 99.76%

Test set: Average loss: 0.5239, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.507275
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508894
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.509448
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.514949
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.511693
Train Epoch: 73 - Loss: 0.5134, Accuracy: 99.76%

Test set: Average loss: 0.5237, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.505912
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.507208
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.508070
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.516923
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.514538
Train Epoch: 74 - Loss: 0.5134, Accuracy: 99.77%

Test set: Average loss: 0.5236, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.513907
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509271
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.512958
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508086
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.520385
Train Epoch: 75 - Loss: 0.5133, Accuracy: 99.77%

Test set: Average loss: 0.5236, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.511334
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.519062
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.516935
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.514300
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.505919
Train Epoch: 76 - Loss: 0.5153, Accuracy: 99.67%

Test set: Average loss: 0.5240, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.506712
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.510317
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.519192
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.507664
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.514167
Train Epoch: 77 - Loss: 0.5146, Accuracy: 99.70%

Test set: Average loss: 0.5242, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.524552
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.507957
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.513641
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509773
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.516995
Train Epoch: 78 - Loss: 0.5142, Accuracy: 99.71%

Test set: Average loss: 0.5234, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.510917
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.508110
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.508586
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507782
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.505765
Train Epoch: 79 - Loss: 0.5139, Accuracy: 99.73%

Test set: Average loss: 0.5238, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.526139
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.510746
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.507074
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.508972
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.505017
Train Epoch: 80 - Loss: 0.5136, Accuracy: 99.74%

Test set: Average loss: 0.5249, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.510124
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.527993
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.508403
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.514067
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.510283
Train Epoch: 81 - Loss: 0.5131, Accuracy: 99.75%

Test set: Average loss: 0.5235, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.512362
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.506560
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.513087
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.511149
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.510502
Train Epoch: 82 - Loss: 0.5127, Accuracy: 99.76%

Test set: Average loss: 0.5241, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.506010
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.524124
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.506430
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.508108
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.521666
Train Epoch: 83 - Loss: 0.5125, Accuracy: 99.75%

Test set: Average loss: 0.5235, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.517103
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.514872
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508550
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.515018
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.507498
Train Epoch: 84 - Loss: 0.5120, Accuracy: 99.77%

Test set: Average loss: 0.5272, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510798
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.505566
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512823
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.516483
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509079
Train Epoch: 85 - Loss: 0.5117, Accuracy: 99.79%

Test set: Average loss: 0.5235, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.506541
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.505734
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.508344
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511244
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.509243
Train Epoch: 86 - Loss: 0.5114, Accuracy: 99.80%

Test set: Average loss: 0.5236, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.503670
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.524547
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.504614
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509662
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.510387
Train Epoch: 87 - Loss: 0.5111, Accuracy: 99.83%

Test set: Average loss: 0.5226, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510608
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.516616
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509823
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.506848
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.506444
Train Epoch: 88 - Loss: 0.5109, Accuracy: 99.81%

Test set: Average loss: 0.5224, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.508007
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507572
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.508402
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.507478
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.511314
Train Epoch: 89 - Loss: 0.5107, Accuracy: 99.82%

Test set: Average loss: 0.5222, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.507272
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.514119
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507670
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.504144
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507901
Train Epoch: 90 - Loss: 0.5105, Accuracy: 99.84%

Test set: Average loss: 0.5225, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511827
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.506560
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510150
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.518119
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.515283
Train Epoch: 91 - Loss: 0.5103, Accuracy: 99.83%

Test set: Average loss: 0.5226, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.509045
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.504332
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510854
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507693
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.504010
Train Epoch: 92 - Loss: 0.5102, Accuracy: 99.84%

Test set: Average loss: 0.5221, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.505373
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.517537
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.507793
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.510790
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507945
Train Epoch: 93 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505149
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.517345
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508077
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.517472
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.523605
Train Epoch: 94 - Loss: 0.5099, Accuracy: 99.84%

Test set: Average loss: 0.5221, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.504897
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.507917
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.505300
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508679
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.511909
Train Epoch: 95 - Loss: 0.5098, Accuracy: 99.85%

Test set: Average loss: 0.5220, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.506199
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.514014
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.505549
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508637
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511403
Train Epoch: 96 - Loss: 0.5097, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.510134
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.506466
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.506871
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.510732
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.514965
Train Epoch: 97 - Loss: 0.5097, Accuracy: 99.86%

Test set: Average loss: 0.5221, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506509
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.505046
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505812
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.506072
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.511723
Train Epoch: 98 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.521979
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.506696
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.504610
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506170
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.510511
Train Epoch: 99 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.505525
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.505273
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508869
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.509634
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505678
Train Epoch: 100 - Loss: 0.5096, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9925/10000 (99.25%)


Training completed. Best Accuracy: 99.26%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.307392
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.298823
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.799817
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.870358
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.772113
Train Epoch: 1 - Loss: 1.5179, Accuracy: 55.70%

Test set: Average loss: 0.7695, Accuracy: 9150/10000 (91.50%)

New best model saved with accuracy: 91.50%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.839525
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.672338
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.749997
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.713042
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.666915
Train Epoch: 2 - Loss: 0.7136, Accuracy: 93.26%

Test set: Average loss: 0.6569, Accuracy: 9537/10000 (95.37%)

New best model saved with accuracy: 95.37%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.631219
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.647243
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.664501
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.658872
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.654050
Train Epoch: 3 - Loss: 0.6409, Accuracy: 95.80%

Test set: Average loss: 0.6232, Accuracy: 9636/10000 (96.36%)

New best model saved with accuracy: 96.36%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.630520
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.700595
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.560754
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.599719
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.618559
Train Epoch: 4 - Loss: 0.6070, Accuracy: 96.84%

Test set: Average loss: 0.5926, Accuracy: 9726/10000 (97.26%)

New best model saved with accuracy: 97.26%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.633713
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.616518
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.625662
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.565136
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.566254
Train Epoch: 5 - Loss: 0.5890, Accuracy: 97.41%

Test set: Average loss: 0.5793, Accuracy: 9771/10000 (97.71%)

New best model saved with accuracy: 97.71%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.575031
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.572990
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.616000
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.606647
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.543803
Train Epoch: 6 - Loss: 0.5770, Accuracy: 97.75%

Test set: Average loss: 0.5722, Accuracy: 9787/10000 (97.87%)

New best model saved with accuracy: 97.87%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.566570
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.550380
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.540439
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.540965
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.560748
Train Epoch: 7 - Loss: 0.5690, Accuracy: 97.98%

Test set: Average loss: 0.5665, Accuracy: 9802/10000 (98.02%)

New best model saved with accuracy: 98.02%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.554054
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.550976
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.542245
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.547181
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.584345
Train Epoch: 8 - Loss: 0.5626, Accuracy: 98.19%

Test set: Average loss: 0.5603, Accuracy: 9817/10000 (98.17%)

New best model saved with accuracy: 98.17%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.546692
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.559036
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.551524
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.534302
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.554364
Train Epoch: 9 - Loss: 0.5577, Accuracy: 98.41%

Test set: Average loss: 0.5569, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.524498
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.603589
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.542534
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.550372
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.560618
Train Epoch: 10 - Loss: 0.5540, Accuracy: 98.46%

Test set: Average loss: 0.5533, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.560268
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.555087
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.561220
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.560819
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.535607
Train Epoch: 11 - Loss: 0.5508, Accuracy: 98.60%

Test set: Average loss: 0.5516, Accuracy: 9849/10000 (98.49%)

New best model saved with accuracy: 98.49%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.553835
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.550485
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.532437
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.565058
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.535650
Train Epoch: 12 - Loss: 0.5482, Accuracy: 98.68%

Test set: Average loss: 0.5481, Accuracy: 9856/10000 (98.56%)

New best model saved with accuracy: 98.56%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.542852
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.544386
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.523940
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.543549
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.536264
Train Epoch: 13 - Loss: 0.5460, Accuracy: 98.75%

Test set: Average loss: 0.5467, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.546015
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.539974
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.542384
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.558357
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.545578
Train Epoch: 14 - Loss: 0.5443, Accuracy: 98.81%

Test set: Average loss: 0.5475, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.529705
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.543017
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.525046
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.521832
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.533724
Train Epoch: 15 - Loss: 0.5428, Accuracy: 98.86%

Test set: Average loss: 0.5450, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.547004
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.532976
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.530006
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.563473
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.541543
Train Epoch: 16 - Loss: 0.5415, Accuracy: 98.89%

Test set: Average loss: 0.5447, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.528718
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.553500
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.528715
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.555040
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.515425
Train Epoch: 17 - Loss: 0.5405, Accuracy: 98.92%

Test set: Average loss: 0.5433, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.534389
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.529252
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.526767
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.548219
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.540202
Train Epoch: 18 - Loss: 0.5396, Accuracy: 98.97%

Test set: Average loss: 0.5426, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.524077
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.536758
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.547857
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.536103
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.542168
Train Epoch: 19 - Loss: 0.5389, Accuracy: 99.01%

Test set: Average loss: 0.5425, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.548653
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.549828
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.525407
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.528797
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.531211
Train Epoch: 20 - Loss: 0.5384, Accuracy: 99.02%

Test set: Average loss: 0.5418, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.522113
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.518704
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.533212
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.520384
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.538521
Train Epoch: 21 - Loss: 0.5379, Accuracy: 99.05%

Test set: Average loss: 0.5416, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.539647
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.524577
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.521319
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.533650
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.558138
Train Epoch: 22 - Loss: 0.5376, Accuracy: 99.05%

Test set: Average loss: 0.5414, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.527453
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.540471
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.574059
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.556085
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.522516
Train Epoch: 23 - Loss: 0.5374, Accuracy: 99.06%

Test set: Average loss: 0.5413, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.523536
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.518972
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.543227
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.543034
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.520160
Train Epoch: 24 - Loss: 0.5373, Accuracy: 99.07%

Test set: Average loss: 0.5412, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.526122
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.544194
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.566875
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.516637
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.543287
Train Epoch: 25 - Loss: 0.5372, Accuracy: 99.06%

Test set: Average loss: 0.5412, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.551528
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.523736
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.535891
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.561350
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.540839
Train Epoch: 26 - Loss: 0.5405, Accuracy: 98.91%

Test set: Average loss: 0.5436, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.539628
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.556246
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.532345
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.521825
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.549348
Train Epoch: 27 - Loss: 0.5381, Accuracy: 98.99%

Test set: Average loss: 0.5402, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.519646
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.532389
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.513020
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.555236
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.541534
Train Epoch: 28 - Loss: 0.5360, Accuracy: 99.03%

Test set: Average loss: 0.5398, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.532631
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.544100
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.520696
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.513885
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.549855
Train Epoch: 29 - Loss: 0.5344, Accuracy: 99.09%

Test set: Average loss: 0.5392, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.521776
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.528908
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.549642
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.527140
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.522242
Train Epoch: 30 - Loss: 0.5326, Accuracy: 99.19%

Test set: Average loss: 0.5373, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.549056
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.513527
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.524419
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.555755
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.528120
Train Epoch: 31 - Loss: 0.5314, Accuracy: 99.22%

Test set: Average loss: 0.5361, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.535767
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.523128
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.526573
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.525462
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.517949
Train Epoch: 32 - Loss: 0.5300, Accuracy: 99.25%

Test set: Average loss: 0.5346, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.532435
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.526719
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.519995
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.544678
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.510541
Train Epoch: 33 - Loss: 0.5288, Accuracy: 99.28%

Test set: Average loss: 0.5383, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.513460
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.542029
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.556321
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.524587
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.516695
Train Epoch: 34 - Loss: 0.5280, Accuracy: 99.32%

Test set: Average loss: 0.5339, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.530730
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.512480
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.522382
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.526223
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.523576
Train Epoch: 35 - Loss: 0.5267, Accuracy: 99.38%

Test set: Average loss: 0.5328, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.535468
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.517410
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.520680
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.513609
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.554384
Train Epoch: 36 - Loss: 0.5258, Accuracy: 99.42%

Test set: Average loss: 0.5320, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.518326
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.529864
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.525349
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.522240
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.539051
Train Epoch: 37 - Loss: 0.5252, Accuracy: 99.41%

Test set: Average loss: 0.5313, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.526177
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.535006
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.525467
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.532230
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.522992
Train Epoch: 38 - Loss: 0.5245, Accuracy: 99.44%

Test set: Average loss: 0.5316, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.509583
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.540517
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.515368
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.511346
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.536399
Train Epoch: 39 - Loss: 0.5238, Accuracy: 99.46%

Test set: Average loss: 0.5306, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.535666
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.531728
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.508204
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.521584
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.524469
Train Epoch: 40 - Loss: 0.5233, Accuracy: 99.47%

Test set: Average loss: 0.5311, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.515722
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.532086
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.509912
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.531483
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.530859
Train Epoch: 41 - Loss: 0.5228, Accuracy: 99.48%

Test set: Average loss: 0.5304, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.519462
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.514496
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.518160
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.518698
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.532296
Train Epoch: 42 - Loss: 0.5224, Accuracy: 99.51%

Test set: Average loss: 0.5300, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.527825
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.534960
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.519834
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.514232
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.510894
Train Epoch: 43 - Loss: 0.5221, Accuracy: 99.52%

Test set: Average loss: 0.5300, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.513851
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.533997
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.531413
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.531519
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.538770
Train Epoch: 44 - Loss: 0.5218, Accuracy: 99.53%

Test set: Average loss: 0.5300, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.519167
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.514172
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.524553
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.521084
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.529891
Train Epoch: 45 - Loss: 0.5215, Accuracy: 99.54%

Test set: Average loss: 0.5296, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.517040
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.521938
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.532879
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.531342
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.515021
Train Epoch: 46 - Loss: 0.5214, Accuracy: 99.54%

Test set: Average loss: 0.5296, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.522846
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.515217
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.520248
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.519301
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.513333
Train Epoch: 47 - Loss: 0.5212, Accuracy: 99.55%

Test set: Average loss: 0.5295, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.513783
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.516001
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.524093
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.520597
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512610
Train Epoch: 48 - Loss: 0.5211, Accuracy: 99.56%

Test set: Average loss: 0.5295, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.512595
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.517282
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.508141
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.512726
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.511726
Train Epoch: 49 - Loss: 0.5210, Accuracy: 99.55%

Test set: Average loss: 0.5295, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.520774
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.538480
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.523080
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.533543
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.532841
Train Epoch: 50 - Loss: 0.5210, Accuracy: 99.56%

Test set: Average loss: 0.5295, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.525135
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.523612
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.529439
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.552803
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.534811
Train Epoch: 51 - Loss: 0.5234, Accuracy: 99.47%

Test set: Average loss: 0.5301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.509364
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.531024
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.524985
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.518309
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.528253
Train Epoch: 52 - Loss: 0.5225, Accuracy: 99.47%

Test set: Average loss: 0.5294, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.544030
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.535626
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.517080
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.536000
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.526362
Train Epoch: 53 - Loss: 0.5217, Accuracy: 99.50%

Test set: Average loss: 0.5294, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.510240
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.519681
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.516447
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.519655
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.521569
Train Epoch: 54 - Loss: 0.5211, Accuracy: 99.53%

Test set: Average loss: 0.5290, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.516787
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.524264
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.512856
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.507031
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.531365
Train Epoch: 55 - Loss: 0.5204, Accuracy: 99.53%

Test set: Average loss: 0.5313, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.543228
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.523721
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.572105
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.523847
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.510808
Train Epoch: 56 - Loss: 0.5197, Accuracy: 99.57%

Test set: Average loss: 0.5285, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.516558
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.517509
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.519352
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.514224
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.511709
Train Epoch: 57 - Loss: 0.5192, Accuracy: 99.56%

Test set: Average loss: 0.5279, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.513321
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.523616
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.522015
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.513215
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.518813
Train Epoch: 58 - Loss: 0.5185, Accuracy: 99.60%

Test set: Average loss: 0.5278, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.518098
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.521442
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.518142
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.517928
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.523816
Train Epoch: 59 - Loss: 0.5180, Accuracy: 99.62%

Test set: Average loss: 0.5271, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.508596
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.543279
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.522069
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.511909
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.525527
Train Epoch: 60 - Loss: 0.5174, Accuracy: 99.62%

Test set: Average loss: 0.5271, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.507096
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.513608
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.511885
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.506288
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.516126
Train Epoch: 61 - Loss: 0.5170, Accuracy: 99.64%

Test set: Average loss: 0.5267, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.508676
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.512006
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.509605
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.519737
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.515376
Train Epoch: 62 - Loss: 0.5166, Accuracy: 99.69%

Test set: Average loss: 0.5267, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.524814
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.521906
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.528778
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.522715
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.519598
Train Epoch: 63 - Loss: 0.5161, Accuracy: 99.69%

Test set: Average loss: 0.5270, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.512180
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.510936
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.516100
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.512058
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.542058
Train Epoch: 64 - Loss: 0.5158, Accuracy: 99.69%

Test set: Average loss: 0.5264, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.523067
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.511890
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.539155
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.509812
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.509696
Train Epoch: 65 - Loss: 0.5154, Accuracy: 99.70%

Test set: Average loss: 0.5261, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.511860
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.508088
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.521247
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505343
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.521931
Train Epoch: 66 - Loss: 0.5152, Accuracy: 99.73%

Test set: Average loss: 0.5263, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.512632
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.507051
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.516892
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.511098
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.514792
Train Epoch: 67 - Loss: 0.5149, Accuracy: 99.74%

Test set: Average loss: 0.5259, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.505156
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.522001
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.527984
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.526886
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.505823
Train Epoch: 68 - Loss: 0.5147, Accuracy: 99.73%

Test set: Average loss: 0.5259, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.508162
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.534539
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507687
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.508377
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.523959
Train Epoch: 69 - Loss: 0.5145, Accuracy: 99.74%

Test set: Average loss: 0.5259, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.511934
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.547602
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.514732
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.506945
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.508456
Train Epoch: 70 - Loss: 0.5144, Accuracy: 99.74%

Test set: Average loss: 0.5258, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.510854
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.506981
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.512104
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.508994
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.522117
Train Epoch: 71 - Loss: 0.5143, Accuracy: 99.75%

Test set: Average loss: 0.5257, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.517711
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509329
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.516527
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512800
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.514859
Train Epoch: 72 - Loss: 0.5142, Accuracy: 99.76%

Test set: Average loss: 0.5256, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.509557
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.524192
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.510728
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.509656
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.529760
Train Epoch: 73 - Loss: 0.5141, Accuracy: 99.75%

Test set: Average loss: 0.5257, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.512637
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508787
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.508635
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.512532
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.507160
Train Epoch: 74 - Loss: 0.5141, Accuracy: 99.75%

Test set: Average loss: 0.5257, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.508711
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.527764
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.505999
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508660
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.511039
Train Epoch: 75 - Loss: 0.5140, Accuracy: 99.76%

Test set: Average loss: 0.5257, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.513458
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.517437
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.511740
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.532466
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.512447
Train Epoch: 76 - Loss: 0.5159, Accuracy: 99.69%

Test set: Average loss: 0.5272, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.517108
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.507775
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.516917
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.521472
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.521590
Train Epoch: 77 - Loss: 0.5155, Accuracy: 99.70%

Test set: Average loss: 0.5274, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.505729
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.526574
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.513541
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.516980
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.505489
Train Epoch: 78 - Loss: 0.5151, Accuracy: 99.70%

Test set: Average loss: 0.5259, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.524401
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.515284
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.508684
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.516978
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.514110
Train Epoch: 79 - Loss: 0.5145, Accuracy: 99.72%

Test set: Average loss: 0.5261, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.508961
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.506507
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510129
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.516119
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.516964
Train Epoch: 80 - Loss: 0.5141, Accuracy: 99.73%

Test set: Average loss: 0.5261, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.512441
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.513644
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.516651
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.507738
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.519855
Train Epoch: 81 - Loss: 0.5137, Accuracy: 99.75%

Test set: Average loss: 0.5258, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.517688
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.519188
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.505590
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.515769
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.512616
Train Epoch: 82 - Loss: 0.5133, Accuracy: 99.77%

Test set: Average loss: 0.5251, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.512269
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.516354
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.507045
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.508612
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.510108
Train Epoch: 83 - Loss: 0.5128, Accuracy: 99.79%

Test set: Average loss: 0.5258, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.510388
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.507272
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.518215
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.505406
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.505769
Train Epoch: 84 - Loss: 0.5126, Accuracy: 99.78%

Test set: Average loss: 0.5250, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.509992
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.506636
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.525058
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.516549
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.518052
Train Epoch: 85 - Loss: 0.5122, Accuracy: 99.81%

Test set: Average loss: 0.5248, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.508302
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.513087
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.507957
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506061
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.513998
Train Epoch: 86 - Loss: 0.5119, Accuracy: 99.80%

Test set: Average loss: 0.5245, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.523868
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.515782
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.508730
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.505028
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.512503
Train Epoch: 87 - Loss: 0.5116, Accuracy: 99.81%

Test set: Average loss: 0.5244, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.504512
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.504671
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509441
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.513821
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505427
Train Epoch: 88 - Loss: 0.5114, Accuracy: 99.82%

Test set: Average loss: 0.5252, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.520073
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507130
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.509307
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.507438
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.516558
Train Epoch: 89 - Loss: 0.5112, Accuracy: 99.81%

Test set: Average loss: 0.5247, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.515987
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.504939
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.533063
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.505152
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.513516
Train Epoch: 90 - Loss: 0.5110, Accuracy: 99.84%

Test set: Average loss: 0.5242, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.507157
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510102
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.514664
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.505720
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506918
Train Epoch: 91 - Loss: 0.5108, Accuracy: 99.85%

Test set: Average loss: 0.5242, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.507222
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.505929
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.522373
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507575
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.508665
Train Epoch: 92 - Loss: 0.5106, Accuracy: 99.83%

Test set: Average loss: 0.5245, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.509527
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.504223
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.514454
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.506864
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.518833
Train Epoch: 93 - Loss: 0.5105, Accuracy: 99.85%

Test set: Average loss: 0.5242, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.505929
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509669
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.506999
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.507333
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.507002
Train Epoch: 94 - Loss: 0.5103, Accuracy: 99.85%

Test set: Average loss: 0.5244, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.512586
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.506541
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.509174
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.539406
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.505407
Train Epoch: 95 - Loss: 0.5103, Accuracy: 99.85%

Test set: Average loss: 0.5242, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.508808
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.506078
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.511492
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.533115
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.524912
Train Epoch: 96 - Loss: 0.5102, Accuracy: 99.86%

Test set: Average loss: 0.5242, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505660
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.510377
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.511359
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.507909
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.535533
Train Epoch: 97 - Loss: 0.5101, Accuracy: 99.86%

Test set: Average loss: 0.5239, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506417
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.507235
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508851
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.510417
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.504698
Train Epoch: 98 - Loss: 0.5101, Accuracy: 99.86%

Test set: Average loss: 0.5240, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.504816
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.505339
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.508659
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.505727
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.507788
Train Epoch: 99 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5240, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.511052
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.509250
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.504631
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510823
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.504256
Train Epoch: 100 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5240, Accuracy: 9915/10000 (99.15%)


Training completed. Best Accuracy: 99.19%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.304515
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.293002
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.343340
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.937852
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.752886
Train Epoch: 1 - Loss: 1.4287, Accuracy: 59.83%

Test set: Average loss: 0.7672, Accuracy: 9159/10000 (91.59%)

New best model saved with accuracy: 91.59%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.845891
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.717336
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.755221
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.675976
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.674807
Train Epoch: 2 - Loss: 0.7088, Accuracy: 93.51%

Test set: Average loss: 0.6540, Accuracy: 9543/10000 (95.43%)

New best model saved with accuracy: 95.43%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.637410
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.691993
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.642992
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.626912
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.627480
Train Epoch: 3 - Loss: 0.6345, Accuracy: 96.03%

Test set: Average loss: 0.6070, Accuracy: 9685/10000 (96.85%)

New best model saved with accuracy: 96.85%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.595711
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.591787
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.624106
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.579348
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.600746
Train Epoch: 4 - Loss: 0.6018, Accuracy: 97.09%

Test set: Average loss: 0.5868, Accuracy: 9742/10000 (97.42%)

New best model saved with accuracy: 97.42%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.588471
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.577681
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.592480
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.556880
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.577095
Train Epoch: 5 - Loss: 0.5839, Accuracy: 97.67%

Test set: Average loss: 0.5746, Accuracy: 9779/10000 (97.79%)

New best model saved with accuracy: 97.79%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.555951
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.595291
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.555411
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.592066
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.570846
Train Epoch: 6 - Loss: 0.5730, Accuracy: 97.98%

Test set: Average loss: 0.5665, Accuracy: 9800/10000 (98.00%)

New best model saved with accuracy: 98.00%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.560390
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.558458
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.557259
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.601440
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.543565
Train Epoch: 7 - Loss: 0.5653, Accuracy: 98.21%

Test set: Average loss: 0.5612, Accuracy: 9817/10000 (98.17%)

New best model saved with accuracy: 98.17%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.570904
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.550612
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.563839
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.533440
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.548488
Train Epoch: 8 - Loss: 0.5596, Accuracy: 98.37%

Test set: Average loss: 0.5572, Accuracy: 9817/10000 (98.17%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.580264
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.537882
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.561919
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.552730
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.564662
Train Epoch: 9 - Loss: 0.5551, Accuracy: 98.49%

Test set: Average loss: 0.5537, Accuracy: 9836/10000 (98.36%)

New best model saved with accuracy: 98.36%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.549154
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.552747
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.543596
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.539590
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.542783
Train Epoch: 10 - Loss: 0.5518, Accuracy: 98.60%

Test set: Average loss: 0.5525, Accuracy: 9827/10000 (98.27%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.534883
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.537769
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.562290
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.538949
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.532682
Train Epoch: 11 - Loss: 0.5487, Accuracy: 98.71%

Test set: Average loss: 0.5483, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.539285
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.556615
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.530110
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.581407
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.555142
Train Epoch: 12 - Loss: 0.5464, Accuracy: 98.76%

Test set: Average loss: 0.5458, Accuracy: 9856/10000 (98.56%)

New best model saved with accuracy: 98.56%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.522209
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.531705
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.542558
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.534714
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.538380
Train Epoch: 13 - Loss: 0.5442, Accuracy: 98.84%

Test set: Average loss: 0.5463, Accuracy: 9853/10000 (98.53%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.534692
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.547592
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.546983
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.531892
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.534261
Train Epoch: 14 - Loss: 0.5425, Accuracy: 98.89%

Test set: Average loss: 0.5438, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.524153
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.562806
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.535893
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.528599
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.540846
Train Epoch: 15 - Loss: 0.5412, Accuracy: 98.92%

Test set: Average loss: 0.5435, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.544864
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.552183
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.535977
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.555549
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.525425
Train Epoch: 16 - Loss: 0.5401, Accuracy: 98.93%

Test set: Average loss: 0.5430, Accuracy: 9866/10000 (98.66%)

New best model saved with accuracy: 98.66%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.551506
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.569222
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.531510
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.553906
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.555955
Train Epoch: 17 - Loss: 0.5389, Accuracy: 98.98%

Test set: Average loss: 0.5415, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.536879
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.531902
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.554400
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.538105
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.545120
Train Epoch: 18 - Loss: 0.5381, Accuracy: 99.01%

Test set: Average loss: 0.5419, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.529530
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.515823
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.537745
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.529951
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.540895
Train Epoch: 19 - Loss: 0.5375, Accuracy: 99.04%

Test set: Average loss: 0.5404, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.541444
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.569766
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.515887
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.530817
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.542418
Train Epoch: 20 - Loss: 0.5369, Accuracy: 99.07%

Test set: Average loss: 0.5409, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.534837
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.523890
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.516850
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.530901
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.533125
Train Epoch: 21 - Loss: 0.5366, Accuracy: 99.08%

Test set: Average loss: 0.5400, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.538342
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.529391
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.517905
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.521252
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.542075
Train Epoch: 22 - Loss: 0.5363, Accuracy: 99.09%

Test set: Average loss: 0.5398, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.542293
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.527544
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.539498
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.531317
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.515512
Train Epoch: 23 - Loss: 0.5360, Accuracy: 99.10%

Test set: Average loss: 0.5398, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.536636
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.528720
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.522382
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.544414
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.521386
Train Epoch: 24 - Loss: 0.5359, Accuracy: 99.10%

Test set: Average loss: 0.5397, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.543032
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.532137
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.537686
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.526465
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.516095
Train Epoch: 25 - Loss: 0.5358, Accuracy: 99.11%

Test set: Average loss: 0.5397, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.532120
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.575121
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.546188
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.569443
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.547467
Train Epoch: 26 - Loss: 0.5389, Accuracy: 99.00%

Test set: Average loss: 0.5395, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.535266
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.564356
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.522625
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.521027
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.564998
Train Epoch: 27 - Loss: 0.5367, Accuracy: 99.04%

Test set: Average loss: 0.5386, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.524535
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.532669
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.552986
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.523502
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.535598
Train Epoch: 28 - Loss: 0.5348, Accuracy: 99.10%

Test set: Average loss: 0.5376, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.532441
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.522206
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.534702
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.523322
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.534842
Train Epoch: 29 - Loss: 0.5332, Accuracy: 99.11%

Test set: Average loss: 0.5360, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.521297
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.521856
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.543070
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.531016
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.566712
Train Epoch: 30 - Loss: 0.5318, Accuracy: 99.19%

Test set: Average loss: 0.5373, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.511301
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.527194
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.516741
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.513685
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.536129
Train Epoch: 31 - Loss: 0.5304, Accuracy: 99.22%

Test set: Average loss: 0.5349, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.529845
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.516016
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.520840
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.513222
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.528785
Train Epoch: 32 - Loss: 0.5289, Accuracy: 99.26%

Test set: Average loss: 0.5336, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.515320
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.553233
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.531090
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.526290
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.514163
Train Epoch: 33 - Loss: 0.5279, Accuracy: 99.31%

Test set: Average loss: 0.5330, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.513116
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.511178
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.513834
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.523468
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.549350
Train Epoch: 34 - Loss: 0.5269, Accuracy: 99.31%

Test set: Average loss: 0.5325, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.510231
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.533047
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.513868
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.536137
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.518690
Train Epoch: 35 - Loss: 0.5259, Accuracy: 99.35%

Test set: Average loss: 0.5317, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.532344
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.540294
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.516996
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.515013
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.507303
Train Epoch: 36 - Loss: 0.5250, Accuracy: 99.38%

Test set: Average loss: 0.5311, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.517255
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.515471
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.564864
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.530080
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.513189
Train Epoch: 37 - Loss: 0.5243, Accuracy: 99.39%

Test set: Average loss: 0.5314, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.530198
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.550497
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.509058
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.514445
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.528168
Train Epoch: 38 - Loss: 0.5236, Accuracy: 99.45%

Test set: Average loss: 0.5301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.533634
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.534684
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.513913
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.515831
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.533478
Train Epoch: 39 - Loss: 0.5231, Accuracy: 99.43%

Test set: Average loss: 0.5296, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.546689
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.512992
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.529356
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.522020
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.514300
Train Epoch: 40 - Loss: 0.5226, Accuracy: 99.47%

Test set: Average loss: 0.5295, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.537405
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.513928
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.514941
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.522125
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.528833
Train Epoch: 41 - Loss: 0.5222, Accuracy: 99.47%

Test set: Average loss: 0.5293, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.535055
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.511890
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.524199
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.533169
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.512311
Train Epoch: 42 - Loss: 0.5217, Accuracy: 99.49%

Test set: Average loss: 0.5291, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.533381
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.539390
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.523501
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.525754
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.510228
Train Epoch: 43 - Loss: 0.5214, Accuracy: 99.50%

Test set: Average loss: 0.5290, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.516602
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.511327
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.525632
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.519835
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.526614
Train Epoch: 44 - Loss: 0.5211, Accuracy: 99.51%

Test set: Average loss: 0.5288, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.540768
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.516650
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.523839
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.516899
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.519948
Train Epoch: 45 - Loss: 0.5209, Accuracy: 99.52%

Test set: Average loss: 0.5286, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.512876
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.519193
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.509826
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.527071
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.524985
Train Epoch: 46 - Loss: 0.5207, Accuracy: 99.52%

Test set: Average loss: 0.5286, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.530585
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.517637
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.522773
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512066
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.525928
Train Epoch: 47 - Loss: 0.5206, Accuracy: 99.52%

Test set: Average loss: 0.5286, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.517340
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.513273
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.510641
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.528553
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512474
Train Epoch: 48 - Loss: 0.5205, Accuracy: 99.53%

Test set: Average loss: 0.5286, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.510837
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.527707
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.521073
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.526132
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.517100
Train Epoch: 49 - Loss: 0.5204, Accuracy: 99.54%

Test set: Average loss: 0.5285, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.516946
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.507682
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.516359
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.508891
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.517665
Train Epoch: 50 - Loss: 0.5204, Accuracy: 99.54%

Test set: Average loss: 0.5285, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513318
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.521547
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.515079
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.511226
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.516004
Train Epoch: 51 - Loss: 0.5224, Accuracy: 99.46%

Test set: Average loss: 0.5292, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.524315
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.522222
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.511261
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.515684
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.521920
Train Epoch: 52 - Loss: 0.5220, Accuracy: 99.46%

Test set: Average loss: 0.5305, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.507637
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.523462
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.506746
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.523154
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.512082
Train Epoch: 53 - Loss: 0.5211, Accuracy: 99.50%

Test set: Average loss: 0.5278, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.520144
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.542535
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.520037
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.525085
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.516236
Train Epoch: 54 - Loss: 0.5206, Accuracy: 99.51%

Test set: Average loss: 0.5279, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.510369
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.507845
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.524911
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.509163
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.518698
Train Epoch: 55 - Loss: 0.5198, Accuracy: 99.52%

Test set: Average loss: 0.5295, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.522480
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.531016
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.539523
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.516956
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.544188
Train Epoch: 56 - Loss: 0.5191, Accuracy: 99.54%

Test set: Average loss: 0.5276, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.522114
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.509292
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.509420
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.509205
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.527126
Train Epoch: 57 - Loss: 0.5185, Accuracy: 99.58%

Test set: Average loss: 0.5278, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.510893
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.511054
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.533516
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.511402
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.522404
Train Epoch: 58 - Loss: 0.5179, Accuracy: 99.60%

Test set: Average loss: 0.5271, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.512524
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.523186
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.512102
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.508243
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.514361
Train Epoch: 59 - Loss: 0.5174, Accuracy: 99.62%

Test set: Average loss: 0.5261, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.505562
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.510603
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.507783
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517430
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.507060
Train Epoch: 60 - Loss: 0.5169, Accuracy: 99.61%

Test set: Average loss: 0.5279, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.521771
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.510063
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.548379
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.523708
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.509260
Train Epoch: 61 - Loss: 0.5165, Accuracy: 99.65%

Test set: Average loss: 0.5269, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.529042
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.521500
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.510673
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.506594
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.505453
Train Epoch: 62 - Loss: 0.5161, Accuracy: 99.64%

Test set: Average loss: 0.5257, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.518878
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.507888
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.513806
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.523019
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.514502
Train Epoch: 63 - Loss: 0.5157, Accuracy: 99.70%

Test set: Average loss: 0.5258, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.523201
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.514029
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.512511
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.514562
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.531871
Train Epoch: 64 - Loss: 0.5154, Accuracy: 99.70%

Test set: Average loss: 0.5258, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.514768
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.507859
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.537932
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.516377
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506969
Train Epoch: 65 - Loss: 0.5150, Accuracy: 99.70%

Test set: Average loss: 0.5253, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.524756
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.523840
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.529171
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.505069
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.506519
Train Epoch: 66 - Loss: 0.5148, Accuracy: 99.70%

Test set: Average loss: 0.5256, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.525121
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.530050
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.510008
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.510161
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.533939
Train Epoch: 67 - Loss: 0.5146, Accuracy: 99.71%

Test set: Average loss: 0.5255, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.511310
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510453
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.509097
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.507675
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.521345
Train Epoch: 68 - Loss: 0.5143, Accuracy: 99.71%

Test set: Average loss: 0.5257, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.509325
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.509927
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.508335
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.505419
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510192
Train Epoch: 69 - Loss: 0.5142, Accuracy: 99.72%

Test set: Average loss: 0.5252, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512115
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.518977
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510754
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512795
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.509819
Train Epoch: 70 - Loss: 0.5140, Accuracy: 99.73%

Test set: Average loss: 0.5250, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.507719
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.510330
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.514380
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.507835
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.509882
Train Epoch: 71 - Loss: 0.5139, Accuracy: 99.74%

Test set: Average loss: 0.5248, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.510606
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.522771
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509658
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.523324
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.525528
Train Epoch: 72 - Loss: 0.5138, Accuracy: 99.74%

Test set: Average loss: 0.5250, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.504508
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508245
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.511871
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.508642
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.533382
Train Epoch: 73 - Loss: 0.5138, Accuracy: 99.73%

Test set: Average loss: 0.5248, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.514280
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.515460
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.506175
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.512064
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.516143
Train Epoch: 74 - Loss: 0.5137, Accuracy: 99.73%

Test set: Average loss: 0.5248, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.509721
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.510751
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.505760
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.506901
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.514626
Train Epoch: 75 - Loss: 0.5137, Accuracy: 99.74%

Test set: Average loss: 0.5248, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.519706
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.509000
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.507290
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.513822
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.513908
Train Epoch: 76 - Loss: 0.5155, Accuracy: 99.67%

Test set: Average loss: 0.5257, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.516905
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.504805
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.518515
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.535434
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.520865
Train Epoch: 77 - Loss: 0.5150, Accuracy: 99.67%

Test set: Average loss: 0.5252, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.510797
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.525452
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.509365
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.505996
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.506486
Train Epoch: 78 - Loss: 0.5146, Accuracy: 99.70%

Test set: Average loss: 0.5257, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.510687
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.506064
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.509960
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507889
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.541529
Train Epoch: 79 - Loss: 0.5143, Accuracy: 99.69%

Test set: Average loss: 0.5247, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.510637
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.522660
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.507132
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.540712
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.514428
Train Epoch: 80 - Loss: 0.5137, Accuracy: 99.73%

Test set: Average loss: 0.5255, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.509067
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.510995
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.521296
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.510251
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.522297
Train Epoch: 81 - Loss: 0.5132, Accuracy: 99.74%

Test set: Average loss: 0.5247, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508342
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.509838
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.512704
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.509208
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.506935
Train Epoch: 82 - Loss: 0.5130, Accuracy: 99.75%

Test set: Average loss: 0.5245, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.506319
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512078
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.509488
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.551526
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.511026
Train Epoch: 83 - Loss: 0.5126, Accuracy: 99.76%

Test set: Average loss: 0.5242, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.508791
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.523957
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.506894
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.522919
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.505917
Train Epoch: 84 - Loss: 0.5122, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.517772
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.516860
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.506578
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.508019
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509678
Train Epoch: 85 - Loss: 0.5120, Accuracy: 99.78%

Test set: Average loss: 0.5236, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.508193
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.511881
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.505949
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.512495
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.515275
Train Epoch: 86 - Loss: 0.5118, Accuracy: 99.80%

Test set: Average loss: 0.5248, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.512008
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.515182
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511290
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.507608
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.531991
Train Epoch: 87 - Loss: 0.5115, Accuracy: 99.81%

Test set: Average loss: 0.5236, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.506258
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.508319
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.506787
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509899
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.505719
Train Epoch: 88 - Loss: 0.5112, Accuracy: 99.82%

Test set: Average loss: 0.5235, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.512619
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.509023
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.505290
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.506160
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.511726
Train Epoch: 89 - Loss: 0.5109, Accuracy: 99.83%

Test set: Average loss: 0.5231, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.525944
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.505425
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.509871
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508351
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509002
Train Epoch: 90 - Loss: 0.5107, Accuracy: 99.83%

Test set: Average loss: 0.5230, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509666
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.534836
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.508902
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.510359
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.513620
Train Epoch: 91 - Loss: 0.5106, Accuracy: 99.84%

Test set: Average loss: 0.5232, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.520930
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.507970
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.506717
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.504360
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.504520
Train Epoch: 92 - Loss: 0.5104, Accuracy: 99.82%

Test set: Average loss: 0.5229, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506520
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.509746
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.504975
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.508448
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.526041
Train Epoch: 93 - Loss: 0.5103, Accuracy: 99.84%

Test set: Average loss: 0.5229, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.514208
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504326
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.505188
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508999
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.513799
Train Epoch: 94 - Loss: 0.5102, Accuracy: 99.85%

Test set: Average loss: 0.5231, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.515202
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.516069
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.509901
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508603
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.515916
Train Epoch: 95 - Loss: 0.5101, Accuracy: 99.85%

Test set: Average loss: 0.5229, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.508163
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.507832
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.504081
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.512685
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.507704
Train Epoch: 96 - Loss: 0.5100, Accuracy: 99.86%

Test set: Average loss: 0.5229, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506832
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.521670
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.506124
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512889
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507545
Train Epoch: 97 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5228, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506537
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.508649
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507367
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508539
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.507440
Train Epoch: 98 - Loss: 0.5099, Accuracy: 99.86%

Test set: Average loss: 0.5229, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.515089
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.505416
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.507860
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.508914
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.510507
Train Epoch: 99 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5229, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.524229
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.509813
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.505525
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.505061
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.505552
Train Epoch: 100 - Loss: 0.5098, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9922/10000 (99.22%)


Training completed. Best Accuracy: 99.23%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.302266
Train Epoch: 1 [12800/60000 (21%)]	Loss: 2.289020
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.112177
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.905458
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.700069
Train Epoch: 1 - Loss: 1.3487, Accuracy: 64.28%

Test set: Average loss: 0.7526, Accuracy: 9206/10000 (92.06%)

New best model saved with accuracy: 92.06%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.780612
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.721868
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.695228
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.756382
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.631529
Train Epoch: 2 - Loss: 0.7010, Accuracy: 93.73%

Test set: Average loss: 0.6449, Accuracy: 9574/10000 (95.74%)

New best model saved with accuracy: 95.74%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.659085
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.639067
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.600694
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.607708
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.613243
Train Epoch: 3 - Loss: 0.6320, Accuracy: 96.11%

Test set: Average loss: 0.6087, Accuracy: 9683/10000 (96.83%)

New best model saved with accuracy: 96.83%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.635292
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.628660
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.604654
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.598561
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.577284
Train Epoch: 4 - Loss: 0.6015, Accuracy: 97.08%

Test set: Average loss: 0.5900, Accuracy: 9733/10000 (97.33%)

New best model saved with accuracy: 97.33%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.583083
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.576605
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.562302
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.590575
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.567554
Train Epoch: 5 - Loss: 0.5848, Accuracy: 97.58%

Test set: Average loss: 0.5765, Accuracy: 9772/10000 (97.72%)

New best model saved with accuracy: 97.72%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.569857
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.559765
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.542678
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.583733
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.536237
Train Epoch: 6 - Loss: 0.5737, Accuracy: 97.89%

Test set: Average loss: 0.5681, Accuracy: 9798/10000 (97.98%)

New best model saved with accuracy: 97.98%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.531460
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.580683
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.577240
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.556387
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.579043
Train Epoch: 7 - Loss: 0.5653, Accuracy: 98.16%

Test set: Average loss: 0.5650, Accuracy: 9813/10000 (98.13%)

New best model saved with accuracy: 98.13%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.551846
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.529999
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.543010
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.577781
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.541757
Train Epoch: 8 - Loss: 0.5593, Accuracy: 98.33%

Test set: Average loss: 0.5579, Accuracy: 9813/10000 (98.13%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.581748
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.540947
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.554493
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.544808
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.577336
Train Epoch: 9 - Loss: 0.5546, Accuracy: 98.47%

Test set: Average loss: 0.5539, Accuracy: 9835/10000 (98.35%)

New best model saved with accuracy: 98.35%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.549752
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.529714
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.553216
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.570282
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.561129
Train Epoch: 10 - Loss: 0.5513, Accuracy: 98.57%

Test set: Average loss: 0.5512, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.570068
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.526506
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.548795
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.538692
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.554604
Train Epoch: 11 - Loss: 0.5479, Accuracy: 98.67%

Test set: Average loss: 0.5489, Accuracy: 9843/10000 (98.43%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.550329
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.529702
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.570389
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.523591
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.534733
Train Epoch: 12 - Loss: 0.5455, Accuracy: 98.78%

Test set: Average loss: 0.5474, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.575920
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.570524
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.563321
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.545702
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.547057
Train Epoch: 13 - Loss: 0.5435, Accuracy: 98.84%

Test set: Average loss: 0.5459, Accuracy: 9850/10000 (98.50%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.545877
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.537971
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.563146
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.556453
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.553077
Train Epoch: 14 - Loss: 0.5417, Accuracy: 98.87%

Test set: Average loss: 0.5442, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.539667
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.564772
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.548632
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.565403
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.532477
Train Epoch: 15 - Loss: 0.5403, Accuracy: 98.95%

Test set: Average loss: 0.5443, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.549740
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.530960
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.526448
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.530161
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.557576
Train Epoch: 16 - Loss: 0.5391, Accuracy: 98.94%

Test set: Average loss: 0.5426, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.531122
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.538296
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.549995
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.527534
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.546225
Train Epoch: 17 - Loss: 0.5380, Accuracy: 98.98%

Test set: Average loss: 0.5418, Accuracy: 9869/10000 (98.69%)

New best model saved with accuracy: 98.69%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.529209
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.568439
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.538020
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.526611
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.550300
Train Epoch: 18 - Loss: 0.5373, Accuracy: 99.05%

Test set: Average loss: 0.5418, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.545340
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.556751
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.520895
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.548252
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.520113
Train Epoch: 19 - Loss: 0.5366, Accuracy: 99.07%

Test set: Average loss: 0.5412, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.519069
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.524912
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.530978
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.538987
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.522616
Train Epoch: 20 - Loss: 0.5361, Accuracy: 99.09%

Test set: Average loss: 0.5405, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.537150
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.532862
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.536334
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.530455
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.540211
Train Epoch: 21 - Loss: 0.5357, Accuracy: 99.08%

Test set: Average loss: 0.5404, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.536049
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.513870
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.512572
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.536677
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.557592
Train Epoch: 22 - Loss: 0.5355, Accuracy: 99.09%

Test set: Average loss: 0.5400, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.544774
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.542313
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.517020
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.554641
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.544872
Train Epoch: 23 - Loss: 0.5353, Accuracy: 99.11%

Test set: Average loss: 0.5399, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.542566
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.529160
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.521417
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.531660
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.525253
Train Epoch: 24 - Loss: 0.5351, Accuracy: 99.10%

Test set: Average loss: 0.5399, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.521481
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.549750
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.518254
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.520763
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.517393
Train Epoch: 25 - Loss: 0.5350, Accuracy: 99.11%

Test set: Average loss: 0.5399, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.524118
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.553004
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.520377
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.534878
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.541110
Train Epoch: 26 - Loss: 0.5382, Accuracy: 99.00%

Test set: Average loss: 0.5440, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.555537
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.523978
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.516221
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.530646
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.545009
Train Epoch: 27 - Loss: 0.5360, Accuracy: 99.06%

Test set: Average loss: 0.5400, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.516882
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.549402
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.511525
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.511555
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.528858
Train Epoch: 28 - Loss: 0.5344, Accuracy: 99.08%

Test set: Average loss: 0.5372, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.521094
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.515531
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.524791
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.512391
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.522828
Train Epoch: 29 - Loss: 0.5323, Accuracy: 99.18%

Test set: Average loss: 0.5364, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.580615
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.517712
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.520266
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.564980
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.528860
Train Epoch: 30 - Loss: 0.5309, Accuracy: 99.19%

Test set: Average loss: 0.5352, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.522910
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.536907
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.526355
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.523250
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.519901
Train Epoch: 31 - Loss: 0.5295, Accuracy: 99.19%

Test set: Average loss: 0.5345, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.545203
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.518198
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.517900
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.543979
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.557820
Train Epoch: 32 - Loss: 0.5282, Accuracy: 99.27%

Test set: Average loss: 0.5335, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.523887
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.544262
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.525235
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.515788
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.516761
Train Epoch: 33 - Loss: 0.5271, Accuracy: 99.32%

Test set: Average loss: 0.5337, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.535410
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.539492
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.532176
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.537676
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.546686
Train Epoch: 34 - Loss: 0.5261, Accuracy: 99.35%

Test set: Average loss: 0.5321, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.515636
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.517569
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.513277
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.523398
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.515540
Train Epoch: 35 - Loss: 0.5254, Accuracy: 99.35%

Test set: Average loss: 0.5322, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.514729
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.528865
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.537366
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.533793
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.520272
Train Epoch: 36 - Loss: 0.5244, Accuracy: 99.40%

Test set: Average loss: 0.5315, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.515513
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.517172
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.523780
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.517896
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.516482
Train Epoch: 37 - Loss: 0.5237, Accuracy: 99.42%

Test set: Average loss: 0.5306, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.512164
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.520613
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.518371
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.512161
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.521288
Train Epoch: 38 - Loss: 0.5230, Accuracy: 99.46%

Test set: Average loss: 0.5313, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.525498
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.515234
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.547963
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.516888
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.520177
Train Epoch: 39 - Loss: 0.5224, Accuracy: 99.48%

Test set: Average loss: 0.5311, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.513551
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.506986
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.509156
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.537623
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.509408
Train Epoch: 40 - Loss: 0.5219, Accuracy: 99.48%

Test set: Average loss: 0.5293, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.510839
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.516861
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.537636
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.514358
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.520683
Train Epoch: 41 - Loss: 0.5215, Accuracy: 99.49%

Test set: Average loss: 0.5293, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.523915
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.533556
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.520328
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.510433
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.511942
Train Epoch: 42 - Loss: 0.5211, Accuracy: 99.50%

Test set: Average loss: 0.5289, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.515066
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.515917
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.526549
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.516135
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.545290
Train Epoch: 43 - Loss: 0.5207, Accuracy: 99.52%

Test set: Average loss: 0.5291, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.513909
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.530127
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.528446
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.507730
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.515878
Train Epoch: 44 - Loss: 0.5205, Accuracy: 99.53%

Test set: Average loss: 0.5288, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.527058
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.538891
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.517946
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.521143
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.547529
Train Epoch: 45 - Loss: 0.5202, Accuracy: 99.53%

Test set: Average loss: 0.5288, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515833
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.511087
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.510290
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.512410
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.509717
Train Epoch: 46 - Loss: 0.5200, Accuracy: 99.55%

Test set: Average loss: 0.5287, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.509306
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.513673
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.511310
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.551113
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.506783
Train Epoch: 47 - Loss: 0.5199, Accuracy: 99.55%

Test set: Average loss: 0.5286, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.519335
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.515944
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513801
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.514063
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.513109
Train Epoch: 48 - Loss: 0.5198, Accuracy: 99.56%

Test set: Average loss: 0.5287, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.512308
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.523531
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.538837
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.532736
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.536786
Train Epoch: 49 - Loss: 0.5197, Accuracy: 99.55%

Test set: Average loss: 0.5286, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513357
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.514228
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.529224
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.513973
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.512740
Train Epoch: 50 - Loss: 0.5197, Accuracy: 99.56%

Test set: Average loss: 0.5286, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.509884
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.516946
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.527914
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.520472
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.516555
Train Epoch: 51 - Loss: 0.5221, Accuracy: 99.46%

Test set: Average loss: 0.5291, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.507910
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.516766
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.519044
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.520550
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.515124
Train Epoch: 52 - Loss: 0.5213, Accuracy: 99.49%

Test set: Average loss: 0.5289, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.507084
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.527231
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.511746
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.512912
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.522922
Train Epoch: 53 - Loss: 0.5206, Accuracy: 99.50%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.520000
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.511151
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.548117
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.529669
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.515146
Train Epoch: 54 - Loss: 0.5197, Accuracy: 99.54%

Test set: Average loss: 0.5299, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.520229
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.527705
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.514010
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.521790
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.518321
Train Epoch: 55 - Loss: 0.5192, Accuracy: 99.54%

Test set: Average loss: 0.5293, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.514724
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.515521
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.533776
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.510286
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.538610
Train Epoch: 56 - Loss: 0.5185, Accuracy: 99.57%

Test set: Average loss: 0.5288, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.521920
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.517786
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.506534
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.509100
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.524437
Train Epoch: 57 - Loss: 0.5177, Accuracy: 99.59%

Test set: Average loss: 0.5273, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.509427
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.517049
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.523499
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.506880
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.511042
Train Epoch: 58 - Loss: 0.5173, Accuracy: 99.62%

Test set: Average loss: 0.5269, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.505719
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.531689
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.512605
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.505342
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.508522
Train Epoch: 59 - Loss: 0.5167, Accuracy: 99.63%

Test set: Average loss: 0.5278, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.511160
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.508688
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.531758
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.515381
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515998
Train Epoch: 60 - Loss: 0.5164, Accuracy: 99.63%

Test set: Average loss: 0.5262, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.507565
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.508993
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.509319
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.512590
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.510882
Train Epoch: 61 - Loss: 0.5159, Accuracy: 99.65%

Test set: Average loss: 0.5278, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.525874
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.509141
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.506744
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.530894
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.517460
Train Epoch: 62 - Loss: 0.5154, Accuracy: 99.67%

Test set: Average loss: 0.5260, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.509230
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.508021
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.508547
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.541986
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.504737
Train Epoch: 63 - Loss: 0.5150, Accuracy: 99.69%

Test set: Average loss: 0.5263, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.506217
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.512783
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.511566
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508091
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.514692
Train Epoch: 64 - Loss: 0.5147, Accuracy: 99.71%

Test set: Average loss: 0.5257, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.517272
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.531718
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.506353
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.515285
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.506225
Train Epoch: 65 - Loss: 0.5144, Accuracy: 99.71%

Test set: Average loss: 0.5252, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.507114
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.513601
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.507343
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.519943
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.534119
Train Epoch: 66 - Loss: 0.5142, Accuracy: 99.70%

Test set: Average loss: 0.5251, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.509916
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.509797
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.534315
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.507334
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513952
Train Epoch: 67 - Loss: 0.5139, Accuracy: 99.73%

Test set: Average loss: 0.5253, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.508763
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.508711
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.509926
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.526391
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.506222
Train Epoch: 68 - Loss: 0.5137, Accuracy: 99.73%

Test set: Average loss: 0.5252, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.522810
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.522465
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.508874
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.533006
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.508739
Train Epoch: 69 - Loss: 0.5136, Accuracy: 99.74%

Test set: Average loss: 0.5252, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.509235
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.515819
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.508593
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.523907
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.518787
Train Epoch: 70 - Loss: 0.5134, Accuracy: 99.75%

Test set: Average loss: 0.5250, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.533942
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.510369
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.508585
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.515677
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.509337
Train Epoch: 71 - Loss: 0.5133, Accuracy: 99.75%

Test set: Average loss: 0.5250, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.513052
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509551
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508037
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512197
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.512583
Train Epoch: 72 - Loss: 0.5132, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.508067
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.507689
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508763
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.510753
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.531800
Train Epoch: 73 - Loss: 0.5132, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.516096
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.515311
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.518414
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.512138
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.522340
Train Epoch: 74 - Loss: 0.5131, Accuracy: 99.76%

Test set: Average loss: 0.5249, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.529668
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.505877
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.532285
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.510225
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.519909
Train Epoch: 75 - Loss: 0.5131, Accuracy: 99.76%

Test set: Average loss: 0.5249, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.509456
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.507731
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.515287
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.515513
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.508256
Train Epoch: 76 - Loss: 0.5146, Accuracy: 99.69%

Test set: Average loss: 0.5263, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.514107
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.510938
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.539390
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.505074
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.522464
Train Epoch: 77 - Loss: 0.5143, Accuracy: 99.70%

Test set: Average loss: 0.5306, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.507564
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.520931
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.510742
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.531769
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.507714
Train Epoch: 78 - Loss: 0.5140, Accuracy: 99.72%

Test set: Average loss: 0.5257, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.515871
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.507057
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.517464
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.508477
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.506227
Train Epoch: 79 - Loss: 0.5135, Accuracy: 99.74%

Test set: Average loss: 0.5263, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.504917
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.505721
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.508930
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.514743
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.509244
Train Epoch: 80 - Loss: 0.5134, Accuracy: 99.73%

Test set: Average loss: 0.5251, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508936
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.516208
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.504747
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.535994
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.505642
Train Epoch: 81 - Loss: 0.5128, Accuracy: 99.75%

Test set: Average loss: 0.5249, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.513659
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.507440
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.513384
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.516893
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.510154
Train Epoch: 82 - Loss: 0.5124, Accuracy: 99.76%

Test set: Average loss: 0.5244, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.504780
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.509357
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.508440
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.505805
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.515036
Train Epoch: 83 - Loss: 0.5121, Accuracy: 99.76%

Test set: Average loss: 0.5240, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.506270
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.507727
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.509081
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.513224
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.515219
Train Epoch: 84 - Loss: 0.5118, Accuracy: 99.78%

Test set: Average loss: 0.5239, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.512621
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.522316
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.513058
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.509949
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509495
Train Epoch: 85 - Loss: 0.5114, Accuracy: 99.82%

Test set: Average loss: 0.5241, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.504937
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.514395
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.504163
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.505358
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.507592
Train Epoch: 86 - Loss: 0.5111, Accuracy: 99.81%

Test set: Average loss: 0.5239, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.506072
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506626
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.506886
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.521914
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.519086
Train Epoch: 87 - Loss: 0.5109, Accuracy: 99.82%

Test set: Average loss: 0.5240, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.508512
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.510353
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509336
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.503658
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.503987
Train Epoch: 88 - Loss: 0.5106, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.512092
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.516326
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.506582
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.506707
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.530966
Train Epoch: 89 - Loss: 0.5104, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508271
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507043
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507321
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.518782
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.531701
Train Epoch: 90 - Loss: 0.5102, Accuracy: 99.84%

Test set: Average loss: 0.5234, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509684
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.507166
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.505393
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.505066
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.505005
Train Epoch: 91 - Loss: 0.5101, Accuracy: 99.84%

Test set: Average loss: 0.5233, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.510042
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.505491
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507418
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.510479
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.510190
Train Epoch: 92 - Loss: 0.5099, Accuracy: 99.85%

Test set: Average loss: 0.5235, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506717
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.506875
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511919
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.509018
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.509765
Train Epoch: 93 - Loss: 0.5097, Accuracy: 99.87%

Test set: Average loss: 0.5234, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.513930
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.504907
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.519903
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.511468
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.526677
Train Epoch: 94 - Loss: 0.5097, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.513835
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.516493
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.505630
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.504954
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.506580
Train Epoch: 95 - Loss: 0.5095, Accuracy: 99.86%

Test set: Average loss: 0.5233, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507195
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.506698
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.505972
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508986
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.507968
Train Epoch: 96 - Loss: 0.5095, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511374
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509943
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507239
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.519995
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.508699
Train Epoch: 97 - Loss: 0.5094, Accuracy: 99.86%

Test set: Average loss: 0.5233, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.508474
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.504979
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.506468
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.521441
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.504606
Train Epoch: 98 - Loss: 0.5094, Accuracy: 99.86%

Test set: Average loss: 0.5232, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.507869
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.506766
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.504730
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506231
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.503173
Train Epoch: 99 - Loss: 0.5093, Accuracy: 99.87%

Test set: Average loss: 0.5232, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506408
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.505825
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.510153
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.512127
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.504256
Train Epoch: 100 - Loss: 0.5093, Accuracy: 99.87%

Test set: Average loss: 0.5232, Accuracy: 9914/10000 (99.14%)


Training completed. Best Accuracy: 99.16%
