................................................................executing startup script (first run)2018-01-27 15:25:40,688 INFO - root - running container entrypoint2018-01-27 15:25:40,689 INFO - root - starting train task2018-01-27 15:25:41,907 INFO - mxnet_container.train - MXNetTrainingEnvironment: {'enable_cloudwatch_metrics': False, 'available_gpus': 0, 'channels': {u'training': {u'TrainingInputMode': u'File', u'RecordWrapperType': u'None', u'S3DistributionType': u'FullyReplicated'}}, '_ps_verbose': 0, 'resource_config': {u'current_host': u'algo-1', u'hosts': [u'algo-1']}, 'user_script_name': u'planesnet-gluon.py', 'input_config_dir': '/opt/ml/input/config', 'channel_dirs': {u'training': u'/opt/ml/input/data/training'}, 'code_dir': '/opt/ml/code', 'output_data_dir': '/opt/ml/output/data/', 'output_dir': '/opt/ml/output', 'model_dir': '/opt/ml/model', 'hyperparameters': {u'sagemaker_program': u'planesnet-gluon.py', u'learning_rate': 0.001, u'batch_size': 128, u'epochs': 50, u'log_interval': 100, u'sagemaker_region': u'us-east-1', u'sagemaker_enable_cloudwatch_metrics': False, u'sagemaker_job_name': u'sagemaker-mxnet-py2-cpu-2018-01-27-15-20-34-855', u'sagemaker_container_log_level': 20, u'sagemaker_submit_directory': u's3://sagemaker-us-east-1-951232522638/sagemaker-mxnet-py2-cpu-2018-01-27-15-20-34-855/source/sourcedir.tar.gz'}, 'hosts': [u'algo-1'], '_ps_port': 8000, 'user_script_archive': u's3://sagemaker-us-east-1-951232522638/sagemaker-mxnet-py2-cpu-2018-01-27-15-20-34-855/source/sourcedir.tar.gz', '_scheduler_host': u'algo-1', 'sagemaker_region': u'us-east-1', 'input_dir': '/opt/ml/input', '_scheduler_ip': '10.32.0.4', 'current_host': u'algo-1', 'container_log_level': 20, 'available_cpus': 4, 'base_dir': '/opt/ml'}Downloading s3://sagemaker-us-east-1-951232522638/sagemaker-mxnet-py2-cpu-2018-01-27-15-20-34-855/source/sourcedir.tar.gz to /tmp/script.tar.gz2018-01-27 15:25:42,024 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTP connection (1): 169.254.170.22018-01-27 15:25:42,121 INFO - botocore.vendored.requests.packages.urllib3.connectionpool - Starting new HTTPS connection (1): s3.amazonaws.com2018-01-27 15:25:42,243 INFO - mxnet_container.train - Starting distributed training task/opt/ml/input/data/trainingMKL Build:20170720[Epoch 0 Batch 100] Training: accuracy=0.765006, 759.383057 samples/s[Epoch 0] Training: accuracy=0.794805[Epoch 0] Validation: accuracy=0.871719[Epoch 1 Batch 100] Training: accuracy=0.848082, 845.643122 samples/s[Epoch 1] Training: accuracy=0.855625[Epoch 1] Validation: accuracy=0.856250[Epoch 2 Batch 100] Training: accuracy=0.871519, 765.710338 samples/s[Epoch 2] Training: accuracy=0.870508[Epoch 2] Validation: accuracy=0.869531[Epoch 3 Batch 100] Training: accuracy=0.869817, 758.141263 samples/s[Epoch 3] Training: accuracy=0.880703[Epoch 3] Validation: accuracy=0.876875[Epoch 4 Batch 100] Training: accuracy=0.905399, 739.773870 samples/s[Epoch 4] Training: accuracy=0.897422[Epoch 4] Validation: accuracy=0.865156[Epoch 5 Batch 100] Training: accuracy=0.891863, 736.402982 samples/s[Epoch 5] Training: accuracy=0.885039[Epoch 5] Validation: accuracy=0.889687[Epoch 6 Batch 100] Training: accuracy=0.896813, 832.725952 samples/s[Epoch 6] Training: accuracy=0.903828[Epoch 6] Validation: accuracy=0.911250[Epoch 7 Batch 100] Training: accuracy=0.913521, 706.163796 samples/s[Epoch 7] Training: accuracy=0.883164[Epoch 7] Validation: accuracy=0.892188[Epoch 8 Batch 100] Training: accuracy=0.901764, 814.544113 samples/s[Epoch 8] Training: accuracy=0.906250[Epoch 8] Validation: accuracy=0.914531[Epoch 9 Batch 100] Training: accuracy=0.922881, 839.967194 samples/s[Epoch 9] Training: accuracy=0.919961[Epoch 9] Validation: accuracy=0.911875[Epoch 10 Batch 100] Training: accuracy=0.920173, 825.439667 samples/s[Epoch 10] Training: accuracy=0.922930[Epoch 10] Validation: accuracy=0.913906[Epoch 11 Batch 100] Training: accuracy=0.927831, 723.879351 samples/s[Epoch 11] Training: accuracy=0.928633[Epoch 11] Validation: accuracy=0.930937[Epoch 12 Batch 100] Training: accuracy=0.935721, 693.529521 samples/s[Epoch 12] Training: accuracy=0.934258[Epoch 12] Validation: accuracy=0.931719[Epoch 13 Batch 100] Training: accuracy=0.938892, 882.557026 samples/s[Epoch 13] Training: accuracy=0.937070[Epoch 13] Validation: accuracy=0.942344[Epoch 14 Batch 100] Training: accuracy=0.944694, 823.765301 samples/s[Epoch 14] Training: accuracy=0.940312[Epoch 14] Validation: accuracy=0.935156[Epoch 15 Batch 100] Training: accuracy=0.944694, 753.783389 samples/s[Epoch 15] Training: accuracy=0.943242[Epoch 15] Validation: accuracy=0.928438[Epoch 16 Batch 100] Training: accuracy=0.940517, 834.474593 samples/s[Epoch 16] Training: accuracy=0.942891[Epoch 16] Validation: accuracy=0.948125[Epoch 17 Batch 100] Training: accuracy=0.949180, 732.688511 samples/s[Epoch 17] Training: accuracy=0.948750[Epoch 17] Validation: accuracy=0.947031[Epoch 18 Batch 100] Training: accuracy=0.954208, 794.118430 samples/s[Epoch 18] Training: accuracy=0.950586[Epoch 18] Validation: accuracy=0.949375[Epoch 19 Batch 100] Training: accuracy=0.948252, 651.028025 samples/s[Epoch 19] Training: accuracy=0.947617[Epoch 19] Validation: accuracy=0.945156[Epoch 20 Batch 100] Training: accuracy=0.952816, 750.552442 samples/s[Epoch 20] Training: accuracy=0.954023[Epoch 20] Validation: accuracy=0.944219[Epoch 21 Batch 100] Training: accuracy=0.956064, 805.446113 samples/s[Epoch 21] Training: accuracy=0.954766[Epoch 21] Validation: accuracy=0.925312[Epoch 22 Batch 100] Training: accuracy=0.961015, 728.287424 samples/s[Epoch 22] Training: accuracy=0.956914[Epoch 22] Validation: accuracy=0.950469[Epoch 23 Batch 100] Training: accuracy=0.956374, 714.090263 samples/s[Epoch 23] Training: accuracy=0.959883[Epoch 23] Validation: accuracy=0.953125[Epoch 24 Batch 100] Training: accuracy=0.961247, 831.400282 samples/s[Epoch 24] Training: accuracy=0.960352[Epoch 24] Validation: accuracy=0.951094[Epoch 25 Batch 100] Training: accuracy=0.962252, 799.470632 samples/s[Epoch 25] Training: accuracy=0.961172[Epoch 25] Validation: accuracy=0.935781[Epoch 26 Batch 100] Training: accuracy=0.954053, 761.868079 samples/s[Epoch 26] Training: accuracy=0.959102[Epoch 26] Validation: accuracy=0.955156[Epoch 27 Batch 100] Training: accuracy=0.964573, 674.191607 samples/s[Epoch 27] Training: accuracy=0.965938[Epoch 27] Validation: accuracy=0.958438[Epoch 28 Batch 100] Training: accuracy=0.965733, 797.647358 samples/s[Epoch 28] Training: accuracy=0.967930[Epoch 28] Validation: accuracy=0.962656[Epoch 29 Batch 100] Training: accuracy=0.964650, 779.907132 samples/s[Epoch 29] Training: accuracy=0.961406[Epoch 29] Validation: accuracy=0.960625[Epoch 30 Batch 100] Training: accuracy=0.961788, 768.749632 samples/s[Epoch 30] Training: accuracy=0.966055[Epoch 30] Validation: accuracy=0.960156[Epoch 31 Batch 100] Training: accuracy=0.971767, 698.846836 samples/s[Epoch 31] Training: accuracy=0.971484[Epoch 31] Validation: accuracy=0.962031[Epoch 32 Batch 100] Training: accuracy=0.970220, 747.096355 samples/s[Epoch 32] Training: accuracy=0.970625[Epoch 32] Validation: accuracy=0.962344[Epoch 33 Batch 100] Training: accuracy=0.976253, 764.946636 samples/s[Epoch 33] Training: accuracy=0.974883[Epoch 33] Validation: accuracy=0.963125[Epoch 34 Batch 100] Training: accuracy=0.972153, 836.164171 samples/s[Epoch 34] Training: accuracy=0.970469[Epoch 34] Validation: accuracy=0.966250[Epoch 35 Batch 100] Training: accuracy=0.974706, 768.242508 samples/s[Epoch 35] Training: accuracy=0.972695[Epoch 35] Validation: accuracy=0.955000[Epoch 36 Batch 100] Training: accuracy=0.977027, 760.524409 samples/s[Epoch 36] Training: accuracy=0.973594[Epoch 36] Validation: accuracy=0.959844[Epoch 37 Batch 100] Training: accuracy=0.977413, 747.528056 samples/s[Epoch 37] Training: accuracy=0.977695[Epoch 37] Validation: accuracy=0.965000[Epoch 38 Batch 100] Training: accuracy=0.981745, 717.674699 samples/s[Epoch 38] Training: accuracy=0.980547[Epoch 38] Validation: accuracy=0.967812[Epoch 39 Batch 100] Training: accuracy=0.969910, 786.163292 samples/s[Epoch 39] Training: accuracy=0.974219[Epoch 39] Validation: accuracy=0.965313[Epoch 40 Batch 100] Training: accuracy=0.973004, 797.308566 samples/s[Epoch 40] Training: accuracy=0.976211[Epoch 40] Validation: accuracy=0.965000[Epoch 41 Batch 100] Training: accuracy=0.979038, 687.090110 samples/s[Epoch 41] Training: accuracy=0.979688[Epoch 41] Validation: accuracy=0.967500[Epoch 42 Batch 100] Training: accuracy=0.980043, 652.801230 samples/s[Epoch 42] Training: accuracy=0.979609[Epoch 42] Validation: accuracy=0.965313[Epoch 43 Batch 100] Training: accuracy=0.984530, 848.940644 samples/s[Epoch 43] Training: accuracy=0.982187[Epoch 43] Validation: accuracy=0.965156[Epoch 44 Batch 100] Training: accuracy=0.979425, 705.016687 samples/s[Epoch 44] Training: accuracy=0.980078[Epoch 44] Validation: accuracy=0.969531[Epoch 45 Batch 100] Training: accuracy=0.983060, 663.906828 samples/s[Epoch 45] Training: accuracy=0.981523[Epoch 45] Validation: accuracy=0.970469[Epoch 46 Batch 100] Training: accuracy=0.980972, 711.672663 samples/s[Epoch 46] Training: accuracy=0.982422[Epoch 46] Validation: accuracy=0.965781[Epoch 47 Batch 100] Training: accuracy=0.984220, 654.936274 samples/s[Epoch 47] Training: accuracy=0.983164[Epoch 47] Validation: accuracy=0.969531[Epoch 48 Batch 100] Training: accuracy=0.987314, 860.672114 samples/s[Epoch 48] Training: accuracy=0.984062[Epoch 48] Validation: accuracy=0.961875[Epoch 49 Batch 100] Training: accuracy=0.984530, 808.596547 samples/s[Epoch 49] Training: accuracy=0.980664[Epoch 49] Validation: accuracy=0.970625===== Job Complete ===== 
