Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.
program:object-detection-tflite-benchmark (v1.0.0)
License: BSD 3-clause (code) and CC BY-SA 4.0 (data)
Creation date: 2019-12-30
Source: GitHub
cID: b0ac08fe1d3c2615:d06491783fbc9b8f

Don't hesitate to get in touch if you encounter any issues or would like to discuss this community project!
Please report if this CK component works: 1  or fails: 0 
Sign up to be notified when artifacts are shared or updated!

Description  

This portable workflow is our attempt to provide a common CLI with Python JSON API and a JSON meta description to automatically detect or install required components (models, data sets, libraries, frameworks, tools), and then build, run, validate, benchmark and auto-tune the associated method (program) across diverse models, datasets, compilers, platforms and environments. Our on-going project is to make the onboarding process as simple as possible via this platform. Please check this CK white paper and don't hesitate to contact us if you have suggestions or feedback!
  • Automation framework: CK
  • Development repository: ck-tensorflow-codereef
  • Source: GitHub
  • Available command lines:
    • ck run program:object-detection-tflite-benchmark --cmd_key=default (META)
  • Support for host OS: any
  • Support for target OS: android, linux
  • Tags: tensorflow-lite,object-detection,tflite,lang-cpp
  • Template: Object Detection via TFLite (C++)
  • How to get the stable version via the client:
    pip install cbench
    cb download program:object-detection-tflite-benchmark --version=1.0.0 --all
    ck run program:object-detection-tflite-benchmark
  • How to get the development version:
    pip install ck
    ck pull repo --url=https://github.com/code-reef/ck-tensorflow-codereef
    ck run program:object-detection-tflite-benchmark

  • CLI and Python API: module:program
  • Dependencies    

    ReadMe  

    TensorFlow Lite (C++) object detection program

    This program uses a statically linked TensorFlow Lite (C++) library and TensorFlow Lite SSD MobileNet models.

    Note: It uses TensorFlow Lite realisation of Non Max Suppresion for custom operator.

    Prerequisites

    Repositories

    $ ck pull repo:ck-tensorflow
    $ ck pull repo:ck-mlperf
    

    TensorFlow Lite library

    $ ck install package:lib-tflite-1.13.1-src-static
    

    Flatbuffers library

    $ ck install package --tags=lib,flatbuffers
    

    xOpenme library

    $ ck install package --tags=lib,xopenme
    

    TensorFlow models API

    $ ck install ck-tensorflow:package:tensorflowmodel-api
    

    or

    $ ck install package --tags=tensorflowmodel,api
    

    Python libraries

    Numpy

    $ ck install package --tags=lib,python-package,numpy
    

    Pillow

    $ ck install package --tags=lib,python-package,pillow
    

    Matplotlib

    $ ck install package --tags=lib,python-package,matplotlib
    

    Python API for COCO-dataset

    $ ck install package --tags=tool,coco
    

    TensorFlow models

    Install a TensorFlow model SSD MobileNet via:

    $ ck install ck-tensorflow:package:-object-detection-ssd-mobilenet-v1-coco
    

    or:

    $ ck install ck-mlperf:package:model-tflite-mlperf-ssd-mobilenet
    

    Datasets

    $ ck install package --tags=dataset,object-detection,coco
    

    NB: If you have previously installed the coco datasets, you should probably renew them:

    $ ck refresh env:{dataset-env-uoa}
    

    where dataset-env-uoa is one of the env identifiers returned by:

    $ ck show env --tags=dataset,coco
    

    Compiling

    $ ck compile ck-tensorflow:program:object-detection-tflite
    

    Running

    $ ck run ck-tensorflow:program:object-detection-tflite
    

    Program parameters

    CK_BATCH_COUNT

    The number of images to be processed.

    Default: 1

    $ ck run ck-tensorflow:program:object-detection-tflite --env.CK_BATCH_COUNT=100
    

    CK_HOST_CPU_NUMBER_OF_PROCESSORS

    The number of threads used by TF Lite library for job.

    Default: 1

    $ ck run ck-tensorflow:program:object-detection-tflite --env.CK_HOST_CPU_NUMBER_OF_PROCESSORS=2
    

    FULL_REPORT or VERBOSE

    Show additional info (FULL_REPORT) or complete additional info (VERBOSE)

    Default: no

    $ ck run ck-tensorflow:program:object-detection-tflite --env.FULL_REPORT=yes
    

    or:

    $ ck run ck-tensorflow:program:object-detection-tflite --env.VERBOSE=yes
    

    USE_CUSTOM_NMS_SETTINGS

    Enable tuning model settings (just for custom operator use).

    Note: Are you sure you know to do?

    Default: no

    Available settings are:

    MAX_CLASSES_PER_DETECTION (integer, >0)
    MAX_DETECTIONS (integer, >0)
    DETECTIONS_PER_CLASS (integer, >0)
    
    NMS_SCORE_THRESHOLD (float, >=0.0)
    NMS_IOU_THRESHOLD (float, >=0.0)
    SCALE_H (float, >0.0)
    SCALE_W (float, >0.0)
    SCALE_X (float, >0.0)
    SCALE_Y (float, >0.0)
    

    Usage example:

    $ ck run ck-tensorflow:program:object-detection-tflite \
        --env.CK_BATCH_COUNT=10 \
        --env.FULL_REPORT=yes \
        --env.USE_CUSTOM_NMS_SETTINGS=yes \
        --env.MAX_DETECTIONS=100 \
        --env.NMS_SCORE_THRESHOLD=0.25 \
        --env.NMS_IOU_THRESHOLD=0.7
    

    Versions  

    Files  

    Comments  

    Please log in to add your comments!
    If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!