Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
Select experiment view:  
Dimensions Raw JSON files
# Platform Test Dataset Dataset file Successful
compile?		 Successful
run?		 Output
correct?		 Total test time
(sec.)		 Setup time
(sec.)		 Test time
(sec.)		 Environment All Features Flat
features		 Point UID Reproduce
1 Amazon EC2 (c5.9xlarge) caffe None False True False 7.241000 ± 1.0% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=24
CK_CAFFE_ITERATIONS=20
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-QV8Zvw.prototxt
CK_CAFFE_MODEL_FILE=tmp-QV8Zvw.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 0b448d45cd8d4697
2 Amazon EC2 (c5.9xlarge) caffe None False True False 9.212000 ± 2.0% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=32
CK_CAFFE_ITERATIONS=15
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-zmrMDU.prototxt
CK_CAFFE_MODEL_FILE=tmp-zmrMDU.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 11b8cfe418653791
3 Amazon EC2 (c5.9xlarge) caffe None False True False 13.304000 ± 0.6% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=48
CK_CAFFE_ITERATIONS=10
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-05bK0L.prototxt
CK_CAFFE_MODEL_FILE=tmp-05bK0L.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 9ce3e8c5d344259b
4 Amazon EC2 (c5.9xlarge) caffe None False True False 3.050000 ± 1.6% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=8
CK_CAFFE_ITERATIONS=62
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-Geubii.prototxt
CK_CAFFE_MODEL_FILE=tmp-Geubii.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View abbddee7b544f09a
5 Amazon EC2 (c5.9xlarge) caffe None False True False 0.857000 ± 5.4% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=1
CK_CAFFE_ITERATIONS=500
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-Fha9hx.prototxt
CK_CAFFE_MODEL_FILE=tmp-Fha9hx.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View ac690453983c9e9f
6 Amazon EC2 (c5.9xlarge) caffe None False True False 15.438000 ± 0.2% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=56
CK_CAFFE_ITERATIONS=8
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-rhK1i7.prototxt
CK_CAFFE_MODEL_FILE=tmp-rhK1i7.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View c0b113e34dff49f5
7 Amazon EC2 (c5.9xlarge) caffe None False True False 17.350000 ± 1.1% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=64
CK_CAFFE_ITERATIONS=7
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-wcCc4K.prototxt
CK_CAFFE_MODEL_FILE=tmp-wcCc4K.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View d7d41bc82485e3a9
8 Amazon EC2 (c5.9xlarge) caffe None False True False 11.298000 ± 0.7% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=40
CK_CAFFE_ITERATIONS=12
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-WzWuVG.prototxt
CK_CAFFE_MODEL_FILE=tmp-WzWuVG.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View e085de45fdc4d73a
9 Amazon EC2 (c5.9xlarge) caffe None False True False 5.168000 ± 2.0% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=16
CK_CAFFE_ITERATIONS=31
CK_CAFFE_MODEL=/home/ubuntu/CK/ck-caffe/program/caffe/tmp/tmp-jkzkmX.prototxt
CK_CAFFE_MODEL_FILE=tmp-jkzkmX.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/ubuntu/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View f6294a9eb4e52efd