Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
program:image-classification-tflite-loadgen (v3.0.0)
Copyright: See copyright in the source repository
License: See license in the source repository
Creation date: 2019-07-09
Source: GitHub
cID: b0ac08fe1d3c2615:45ee0d34a73b7a2c

Don't hesitate to get in touch if you encounter any issues or would like to discuss this community project!
Please report if this CK component works: 1  or fails: 0 
Sign up to be notified when artifacts are shared or updated!

Description  

This portable workflow is our attempt to provide a common CLI with Python JSON API and a JSON meta description to automatically detect or install required components (models, data sets, libraries, frameworks, tools), and then build, run, validate, benchmark and auto-tune the associated method (program) across diverse models, datasets, compilers, platforms and environments. Our on-going project is to make the onboarding process as simple as possible via this platform. Please check this CK white paper and don't hesitate to contact us if you have suggestions or feedback!
  • Automation framework: CK
  • Development repository: ck-ml
  • Source: GitHub
  • Available command lines:
    • ck run program:image-classification-tflite-loadgen --cmd_key=default (META)
  • Support for host OS: any
  • Support for target OS: android, linux
  • Tags: image-classification,tflite,loadgen,lang-cpp
  • Template: Image Classification via TFLite (C++)
  • How to get the stable version via the client:
    pip install cbench
    cb download program:image-classification-tflite-loadgen --version=3.0.0 --all
    ck run program:image-classification-tflite-loadgen
  • How to get the development version:
    pip install ck
    ck pull repo:ck-ml
    ck run program:image-classification-tflite-loadgen

  • CLI and Python API: module:program
  • Dependencies    

    ReadMe  

    MLPerf Inference - Image Classification - TFLite

    This C++ implementation runs TFLite models for Image Classification using TFLite.

    Prerequisites

    Preprocess ImageNet on an x86 machine

    model-tflite-mlperf-resnet*, model-tflite-mlperf-efficientnet-lite0, model-tf-and-tflite-mlperf-mobilenet* (resolution 224)

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.224,full --ask
    

    model-tf-and-tflite-mlperf-mobilenet* (resolution 192)

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.192,full --ask
    

    model-tf-and-tflite-mlperf-mobilenet* (resolution 160)

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.160,full --ask
    

    model-tf-and-tflite-mlperf-mobilenet* (resolution 128)

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.128,full --ask
    

    model-tf-and-tflite-mlperf-mobilenet* (resolution 96)

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.96,full --ask
    

    model-tflite-mlperf-efficientnet-lite1

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.240,full --ask
    

    model-tflite-mlperf-efficientnet-lite2

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.260,full --ask
    

    model-tflite-mlperf-efficientnet-lite3

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.280,full --ask
    

    model-tflite-mlperf-efficientnet-lite4

    $ ck install package --tags=dataset,imagenet,preprocessed,using-opencv,side.300,full --ask
    

    Detect ImageNet on a dev board

    Copy a preprocessed ImageNet dataset onto a dev board e.g. under /datasets and register it with CK according to its resolution e.g.:

    $ echo opencv-side.240 | ck detect soft --tags=dataset,imagenet,preprocessed,rgb8 \
    --extra_tags=using-opencv,crop.875,full,inter.linear,side.240 \
    --full_path=/datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.240/ILSVRC2012_val_00000001.rgb8
    

    Run once (classical CK interface)

    Running this program is similar to running ck-tensorflow:program:image-classification-tflite, as described in the MLPerf Inference repo.

    firefly $ ck benchmark program:image-classification-tflite-loadgen \
    --speed --repetitions=1 \
    --env.CK_VERBOSE=1 \
    --env.CK_LOADGEN_SCENARIO=SingleStream \
    --env.CK_LOADGEN_MODE=PerformanceOnly \
    --env.CK_LOADGEN_DATASET_SIZE=1024 \
    --env.CK_LOADGEN_BUFFER_SIZE=1024 \
    --dep_add_tags.weights=model,tflite,resnet \
    --dep_add_tags.library=tflite,v1.15 \
    --dep_add_tags.compiler=gcc,v7 \
    --dep_add_tags.images=side.224,preprocessed \
    --dep_add_tags.loadgen-config-file=image-classification-tflite \
    --dep_add_tags.python=v3 \
    --skip_print_timers
    ...
    ------------------------------------------------------------
    |            LATENCIES (in nanoseconds and fps)            |
    ------------------------------------------------------------
    Number of queries run: 1024
    Min latency:                      397952762ns  (2.51286 fps)
    Median latency:                   426440993ns  (2.34499 fps)
    Average latency:                  433287227ns  (2.30794 fps)
    90 percentile latency:            460194271ns  (2.173 fps)
    Max latency:                      679467557ns  (1.47174 fps)
    ------------------------------------------------------------
    

    Explore different models

    TODO

    Versions  

    Files  

    Comments  

    Please log in to add your comments!
    If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!