Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
Select experiment view:  
Dimensions Raw JSON files
# Platform Test Dataset Dataset file Successful
compile?		 Successful
run?		 Output
correct?		 Total test time
(sec.)		 Setup time
(sec.)		 Test time
(sec.)		 Environment All Features Flat
features		 Point UID Reproduce
1 Google (Google Compute Engine) caffe None False True False 32.626000 ± 8.7% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=56
CK_CAFFE_ITERATIONS=8
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-i06p3Z.prototxt
CK_CAFFE_MODEL_FILE=tmp-i06p3Z.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View 345d0bf026a4383c
2 Google (Google Compute Engine) caffe None False True False 0.747000 ± 12.7% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=1
CK_CAFFE_ITERATIONS=500
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-MbVcGe.prototxt
CK_CAFFE_MODEL_FILE=tmp-MbVcGe.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View 3a616daa24f71a2d
3 Google (Google Compute Engine) caffe None False True False 9.513000 ± 9.5% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=16
CK_CAFFE_ITERATIONS=31
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-DE2ebz.prototxt
CK_CAFFE_MODEL_FILE=tmp-DE2ebz.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View 418d4935a1d4ea1b
4 Google (Google Compute Engine) caffe None False True False 28.444000 ± 4.4% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=48
CK_CAFFE_ITERATIONS=10
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-W2BJ7u.prototxt
CK_CAFFE_MODEL_FILE=tmp-W2BJ7u.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View 4fbe1974a7c434f8
5 Google (Google Compute Engine) caffe None False False False CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=64
CK_CAFFE_ITERATIONS=7
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-CBVRjk.prototxt
CK_CAFFE_MODEL_FILE=tmp-CBVRjk.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View 520ee35c042759c1
6 Google (Google Compute Engine) caffe None False True False 18.548000 ± 6.0% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=32
CK_CAFFE_ITERATIONS=15
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-HAqTcD.prototxt
CK_CAFFE_MODEL_FILE=tmp-HAqTcD.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View a58b4dab66db271a
7 Google (Google Compute Engine) caffe None False True False 23.260000 ± 6.5% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=40
CK_CAFFE_ITERATIONS=12
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-1kvW8s.prototxt
CK_CAFFE_MODEL_FILE=tmp-1kvW8s.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View b89510b34c114eaa
8 Google (Google Compute Engine) caffe None False True False 14.346000 ± 7.2% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=24
CK_CAFFE_ITERATIONS=20
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-_MOAkm.prototxt
CK_CAFFE_MODEL_FILE=tmp-_MOAkm.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View f5b43fe4c4b37801
9 Google (Google Compute Engine) caffe None False True False 4.743000 ± 8.4% (5) CAFFE_COMPUTE_DEVICE_ID=0
CK_CAFFE_BATCH_SIZE=8
CK_CAFFE_ITERATIONS=62
CK_CAFFE_MODEL=/home/gfursin/CK/ck-caffe/program/caffe/tmp/tmp-KiVeNA.prototxt
CK_CAFFE_MODEL_FILE=tmp-KiVeNA.prototxt
CK_CAFFE_MODEL_MEAN_BIN=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3_mean.binaryproto
CK_CAFFE_MODEL_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_CPU_FREQUENCY=max
CK_ENV_MODEL_CAFFE_WEIGHTS=/home/gfursin/CK-TOOLS/caffemodel-inception-v3-fp32/inceptionv3.caffemodel
CK_GPU_FREQUENCY=max
 View View View fb88801a4c2bba51