Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
script:mlperf-inference-v0.7.image-classification (v3.0.0)
Share
Copyright: See copyright in the source repository
License: See license in the source repository
Creation date: 2020-06-18
Source: GitHub
cID: 84e27ad9dd12e734:dba09db8e07735a9

Don't hesitate to get in touch if you encounter any issues or would like to discuss this community project!
Please report if this CK component works: 1 ▲ or fails: 0 ▲
Sign up to be notified when artifacts are shared or updated!

ReadMe  

MLPerf Inference v0.7 - Image Classification

Table of Contents

  1. Prerequisites
    1. CK
    2. Inference engines
    3. Preprocessed datasets
    4. LoadGen config files
  2. Usage

Prerequisites

Collective Knowledge

$ export CK_PYTHON=python3
$ ${CK_PYTHON} -m pip install ck
$ ck pull repo:ck-mlperf
$ ck pull repo --url=https://github.com/arm-software/armnn-mlperf

Inference engines

The run.sh script assumes two inference engine: TFLite (tflite) and ArmNN (armnn). Their latest versions are specified in the script:

$ grep inference_engine_version= * -B1 -n
run.sh-223-  if [ "${inference_engine}" == "tflite" ]; then
run.sh:224:    inference_engine_version="v2.1.1" # "v2.2.0
--
run.sh-227-  elif [ "${inference_engine}" == "armnn" ]; then
run.sh:228:    inference_engine_version="rel.20.05"

The inference engines should be installed using the above versions as follows.

TFLite

$ ck install package --tags=lib,tflite,v2.1.1

ArmNN

To parse TFLite models, ArmNN should be built with the TFLite frontend.

OpenCL backend

If your board has an Arm Mali GPU (e.g. Linaro HiKey960 or Firefly RK3399), build ArmNN with the OpenCL backend:

$ ck install package --tags=lib,armnn,rel.20.05,tflite,neon,opencl

Neon backend

Otherwise, your board should support Arm Neon vector extensions, so build ArmNN with the Neon backend only:

$ ck install package --tags=lib,armnn,rel.20.05,tflite,neon

Preprocessed datasets

Preprocess the ImageNet validation dataset on an x86 machine and copy to your Arm board.

224

$ ck detect soft:dataset.imagenet.preprocessed \
--full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224/ILSVRC2012_val_00000001.rgb8 \
--extra_tags=using-opencv,crop.875,full,inter.linear,side.224,universal

192

$ ck detect soft:dataset.imagenet.preprocessed \
--full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.192/ILSVRC2012_val_00000001.rgb8 \
--extra_tags=using-opencv,crop.875,full,inter.linear,side.192,universal

160

$ ck detect soft:dataset.imagenet.preprocessed \
--full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.160/ILSVRC2012_val_00000001.rgb8 \
--extra_tags=using-opencv,crop.875,full,inter.linear,side.160,universal

128

$ ck detect soft:dataset.imagenet.preprocessed \
--full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.128/ILSVRC2012_val_00000001.rgb8 \
--extra_tags=using-opencv,crop.875,full,inter.linear,side.128,universal

96

$ ck detect soft:dataset.imagenet.preprocessed \
--full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.96/ILSVRC2012_val_00000001.rgb8 \
--extra_tags=using-opencv,crop.875,full,inter.linear,side.96,universal

LoadGen config files

TFLite

$ ck detect soft --tags=config,loadgen,image-classification-tflite

ArmNN

$ ck detect soft --tags=config,loadgen,image-classification-armnn-tflite

Usage

$ cd `ck find ck-mlperf:script:mlperf-inference-v0.7.image-classification`
Parameter Values Default Comment
CK_DIVISION closed, open closed Workload selection.
CK_MODE performance, accuracy performance Execution mode selection.
CK_DATASET_SIZE positive integer 50000 Number of samples in the accuracy mode.
CK_USE_LOADGEN YES, NO YES Use MLPerf LoadGen API.
CK_DRY_RUN YES, NO NO Print commands but do not execute.

Performance

$ CK_DIVISION=open CK_MODE=performance ./run.sh

Accuracy

50,000 images

$ CK_DIVISION=open CK_MODE=accuracy CK_DATASET_SIZE=50000 ./run.sh

500 images

$ CK_DIVISION=open CK_MODE=accuracy CK_DATASET_SIZE=500 ./run.sh

Versions  

Files  

Comments  

Please log in to add your comments!
If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!