Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
Select experiment view:  
Dimensions Raw JSON files
# Platform Test Dataset Dataset file Successful
compile?		 Successful
run?		 Output
correct?		 Total test time
(sec.)		 Setup time
(sec.)		 Test time
(sec.)		 Environment All Features Flat
features		 Point UID Reproduce
1 Google (Google Compute Engine) caffe None False True False 13.245000 ± 3.5% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=24
CK_CAFFE_ITERATIONS=20
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-1U7oN6.prototxt
CK_CAFFE_MODEL_FILE=tmp-1U7oN6.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 18ab074014b44b26
2 Google (Google Compute Engine) caffe None False True False 21.657000 ± 30.7% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=40
CK_CAFFE_ITERATIONS=12
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-Po9Poj.prototxt
CK_CAFFE_MODEL_FILE=tmp-Po9Poj.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 1cd1397054b56a59
3 Google (Google Compute Engine) caffe None False True False 8.869000 ± 2.8% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=16
CK_CAFFE_ITERATIONS=31
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-LY3Hzl.prototxt
CK_CAFFE_MODEL_FILE=tmp-LY3Hzl.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 343fd21543f0afe7
4 Google (Google Compute Engine) caffe None False False False CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=64
CK_CAFFE_ITERATIONS=7
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-YxSzFb.prototxt
CK_CAFFE_MODEL_FILE=tmp-YxSzFb.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 3728cba59176bc21
5 Google (Google Compute Engine) caffe None False True False 0.688000 ± 14.1% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=1
CK_CAFFE_ITERATIONS=500
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-8_NpwU.prototxt
CK_CAFFE_MODEL_FILE=tmp-8_NpwU.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 4588be6458292e12
6 Google (Google Compute Engine) caffe None False False False CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=56
CK_CAFFE_ITERATIONS=8
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-JQuwSO.prototxt
CK_CAFFE_MODEL_FILE=tmp-JQuwSO.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 4bae6a7e56bb419a
7 Google (Google Compute Engine) caffe None False True False 25.993000 ± 6.7% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=48
CK_CAFFE_ITERATIONS=10
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-0WrdJr.prototxt
CK_CAFFE_MODEL_FILE=tmp-0WrdJr.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View 62b85e1e4acdd9de
8 Google (Google Compute Engine) caffe None False True False 4.495000 ± 8.3% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=8
CK_CAFFE_ITERATIONS=62
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-JjbrKE.prototxt
CK_CAFFE_MODEL_FILE=tmp-JjbrKE.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View c2b09403eaed17a6
9 Google (Google Compute Engine) caffe None False True False 18.048000 ± 11.8% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=32
CK_CAFFE_ITERATIONS=15
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-Or7ay2.prototxt
CK_CAFFE_MODEL_FILE=tmp-Or7ay2.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-resnet50-fp32/ResNet-50-model.caffemodel
CK_GPU_FREQUENCY=max
 View View View dc29f1bd1741f0b0