{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# [PUBLIC] MLPerf Inference - Image Classification - TFLite\n",
"\n",
"# Table of Contents\n",
"\n",
"1. [Platforms](#platforms)\n",
" 1. [`diviniti`](#platforms_diviniti)\n",
" 1. [`rising`](#platforms_rising)\n",
" 1. [`xavier`](#platforms_xavier)\n",
" 1. [`rpi4coral`](#platforms_rpi4coral)\n",
"\n",
"1. [Installation](#install)\n",
" 1. [Install system-wide prerequisites](#install_system)\n",
" 1. [Ubuntu 20.04](#install_system_ubuntu)\n",
" 1. [Install CK](#install_ck)\n",
" 1. [Detect Python](#detect_python)\n",
" 1. [Detect GCC](#detect_gcc)\n",
" 1. [Install Python dependencies](#install_python_deps)\n",
" 1. [Install the MLPerf Inference repo](#install_inference_repo)\n",
" 1. [Install a small dataset](#install_small_dataset)\n",
" 1. [Preprocess the dataset](#preprocess_small_dataset)\n",
" 1. [Install CMake](#install_cmake)\n",
" 1. [Install TFLite](#install_tflite)\n",
" 1. [Install ResNet50](#install_resnet50)\n",
" 1. [Quick test](#install_test)\n",
" \n",
"1. [Benchmarking accuracy](#benchmarking_accuracy)\n",
" 1. [Single Stream](#benchmarking_accuracy_singlestream)\n",
" 1. [Offline](#benchmarking_accuracy_offline)\n",
" \n",
"1. [Benchmarking performance](#benchmarking_performance)\n",
" 1. [Single Stream](#benchmarking_performance_singlestream)\n",
" 1. [`diviniti`](#benchmarking_performance_singlestream_diviniti)\n",
" 1. [`rising`](#benchmarking_performance_singlestream_rising)\n",
" 1. [`xavier`](#benchmarking_performance_singlestream_xavier)\n",
" 1. [`rpi4coral`](#benchmarking_performance_singlestream_rpi4coral)\n",
" 1. [Offline](#benchmarking_performance_offline)\n",
" 1. [`diviniti`](#benchmarking_performance_offline_diviniti) \n",
" 1. [`xavier`](#benchmarking_performance_offline_xavier) \n",
" 1. [`rpi4coral`](#benchmarking_performance_offline_rpi4coral) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# Platforms"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## `diviniti` (HP ZBook Studio G3)\n",
"yvan@diviniti:~$ uname -a\n",
"Linux diviniti 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\n",
"\n",
"\n",
"yvan@diviniti:~$ cat /etc/lsb-release\n",
"DISTRIB_ID=Ubuntu\n",
"DISTRIB_RELEASE=20.04\n",
"DISTRIB_CODENAME=focal\n",
"DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"\n",
"\n",
"\n",
"yvan@diviniti:~$ lscpu\n",
"Architecture: x86_64\n",
"CPU op-mode(s): 32-bit, 64-bit\n",
"Byte Order: Little Endian\n",
"Address sizes: 39 bits physical, 48 bits virtual\n",
"CPU(s): 8\n",
"On-line CPU(s) list: 0-7\n",
"Thread(s) per core: 2\n",
"Core(s) per socket: 4\n",
"Socket(s): 1\n",
"NUMA node(s): 1\n",
"Vendor ID: GenuineIntel\n",
"CPU family: 6\n",
"Model: 94\n",
"Model name: Intel(R) Xeon(R) CPU E3-1505M v5 @ 2.80GHz\n",
"Stepping: 3\n",
"CPU MHz: 900.013\n",
"CPU max MHz: 3700.0000\n",
"CPU min MHz: 800.0000\n",
"BogoMIPS: 5599.85\n",
"Virtualisation: VT-x\n",
"L1d cache: 128 KiB\n",
"L1i cache: 128 KiB\n",
"L2 cache: 1 MiB\n",
"L3 cache: 8 MiB\n",
"NUMA node0 CPU(s): 0-7\n",
"Vulnerability Itlb multihit: KVM: Vulnerable\n",
"Vulnerability L1tf: Mitigation; PTE Inversion\n",
"Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable\n",
"Vulnerability Meltdown: Mitigation; PTI\n",
"Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp\n",
"Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\n",
"Vulnerability Spectre v2: Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling\n",
"Vulnerability Srbds: Mitigation; Microcode\n",
"Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable\n",
"Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon\n",
" pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe p\n",
" opcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad \n",
" fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act\n",
" _window hwp_epp md_clear flush_l1d\n",
"\n",
"\n",
"yvan@diviniti:~$ nvidia-smi\n",
"Sun Dec 27 20:06:35 2020 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 455.38 Driver Version: 455.38 CUDA Version: 11.1 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|===============================+======================+======================|\n",
"| 0 Quadro M1000M Off | 00000000:01:00.0 On | N/A |\n",
"| N/A 45C P0 N/A / N/A | 911MiB / 4042MiB | 0% Default |\n",
"| | | N/A |\n",
"+-------------------------------+----------------------+----------------------+\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## `rising` (AMD Ryzen 5 5600x)\n",
"yvan@rising:~$ uname -a\n",
"Linux rising 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux\n",
"\n",
"\n",
"yvan@rising:~$ cat /etc/lsb-release\n",
"DISTRIB_ID=Ubuntu\n",
"DISTRIB_RELEASE=20.04\n",
"DISTRIB_CODENAME=focal\n",
"DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"\n",
"\n",
"\n",
"yvan@rising:~/projects$ lscpu\n",
"Architecture: x86_64\n",
"CPU op-mode(s): 32-bit, 64-bit\n",
"Byte Order: Little Endian\n",
"Address sizes: 48 bits physical, 48 bits virtual\n",
"CPU(s): 12\n",
"On-line CPU(s) list: 0-11\n",
"Thread(s) per core: 2\n",
"Core(s) per socket: 6\n",
"Socket(s): 1\n",
"NUMA node(s): 1\n",
"Vendor ID: AuthenticAMD\n",
"CPU family: 25\n",
"Model: 33\n",
"Model name: AMD Ryzen 5 5600X 6-Core Processor\n",
"Stepping: 0\n",
"Frequency boost: enabled\n",
"CPU MHz: 2826.831\n",
"CPU max MHz: 3700.0000\n",
"CPU min MHz: 2200.0000\n",
"BogoMIPS: 7385.91\n",
"Virtualisation: AMD-V\n",
"L1d cache: 192 KiB\n",
"L1i cache: 192 KiB\n",
"L2 cache: 3 MiB\n",
"L3 cache: 32 MiB\n",
"NUMA node0 CPU(s): 0-11\n",
"Vulnerability Itlb multihit: Not affected\n",
"Vulnerability L1tf: Not affected\n",
"Vulnerability Mds: Not affected\n",
"Vulnerability Meltdown: Not affected\n",
"Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp\n",
"Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization\n",
"Vulnerability Spectre v2: Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP always-on, RSB filling\n",
"Vulnerability Srbds: Not affected\n",
"Vulnerability Tsx async abort: Not affected\n",
"Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good no\n",
" pl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_l\n",
" egacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb \n",
" stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_tota\n",
" l cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vm\n",
" load vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca\n",
"\n",
"\n",
"yvan@rising:~$ nvidia-smi\n",
"Sun Jan 3 16:47:52 2021 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 460.27.04 Driver Version: 460.27.04 CUDA Version: 11.2 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|===============================+======================+======================|\n",
"| 0 GeForce GTX 1080 Off | 00000000:0A:00.0 On | N/A |\n",
"| 27% 32C P8 8W / 180W | 196MiB / 8118MiB | 1% Default |\n",
"| | | N/A |\n",
"+-------------------------------+----------------------+----------------------+\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## `xavier` ([NVIDIA Jetson AGX Xavier](https://www.nvidia.com/en-gb/autonomous-machines/embedded-systems/jetson-agx-xavier/))\n",
"arjun@xavier:~$ uname -a\n",
"Linux xavier 4.9.140-tegra #1 SMP PREEMPT Mon Dec 9 22:52:02 PST 2019 aarch64 aarch64 aarch64 GNU/Linux \n",
"\n",
"arjun@xavier:~$ cat /etc/lsb-release\n",
"DISTRIB_ID=Ubuntu\n",
"DISTRIB_RELEASE=18.04\n",
"DISTRIB_CODENAME=bionic\n",
"DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"\n",
"arjun@xavier:~$ lscpu\n",
"Architecture: aarch64\n",
"Byte Order: Little Endian\n",
"CPU(s): 8\n",
"On-line CPU(s) list: 0-7\n",
"Thread(s) per core: 1\n",
"Core(s) per socket: 2\n",
"Socket(s): 4\n",
"Vendor ID: Nvidia\n",
"Model: 0\n",
"Model name: ARMv8 Processor rev 0 (v8l)\n",
"Stepping: 0x0\n",
"CPU max MHz: 2265.6001\n",
"CPU min MHz: 115.2000\n",
"BogoMIPS: 62.50\n",
"L1d cache: 64K\n",
"L1i cache: 128K\n",
"L2 cache: 2048K\n",
"L3 cache: 4096K\n",
"Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp\n",
"arjun@xavier:~$ sudo jetson_clocks\n",
"arjun@xavier:~$ sudo jetson_clocks --show\n",
"SOC family:tegra194 Machine:Jetson-AGX\n",
"Online CPUs: 0-7\n",
"CPU Cluster Switching: Disabled\n",
"cpu0: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu1: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu2: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu3: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu4: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu5: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu6: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"cpu7: Online=1 Governor=schedutil MinFreq=2265600 MaxFreq=2265600 CurrentFreq=2265600 IdleStates: C1=0 c6=0 \n",
"GPU MinFreq=1377000000 MaxFreq=1377000000 CurrentFreq=1377000000\n",
"EMC MinFreq=204000000 MaxFreq=2133000000 CurrentFreq=2133000000 FreqOverride=1\n",
"Fan: speed=255\n",
"NV Power Mode: MAXN\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## `rpi4coral` ([Raspberry Pi 4](https://www.raspberrypi.org/products/raspberry-pi-4-model-b/specifications/))\n",
"arjun@rpi4coral:~$ uname -a\n",
"Linux rpi4coral 5.4.0-1026-raspi #29-Ubuntu SMP PREEMPT Mon Dec 14 17:01:16 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux\n",
"\n",
"arjun@rpi4coral:~$ cat /etc/lsb-release\n",
"DISTRIB_ID=Ubuntu\n",
"DISTRIB_RELEASE=20.04\n",
"DISTRIB_CODENAME=focal\n",
"DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"\n",
"\n",
"arjun@rpi4coral:~$ lscpu\n",
"Architecture: aarch64\n",
"CPU op-mode(s): 32-bit, 64-bit\n",
"Byte Order: Little Endian\n",
"CPU(s): 4\n",
"On-line CPU(s) list: 0-3\n",
"Thread(s) per core: 1\n",
"Core(s) per socket: 4\n",
"Socket(s): 1\n",
"Vendor ID: ARM\n",
"Model: 3\n",
"Model name: Cortex-A72\n",
"Stepping: r0p3\n",
"CPU max MHz: 1500.0000\n",
"CPU min MHz: 600.0000\n",
"BogoMIPS: 108.00\n",
"Vulnerability Itlb multihit: Not affected\n",
"Vulnerability L1tf: Not affected\n",
"Vulnerability Mds: Not affected\n",
"Vulnerability Meltdown: Not affected\n",
"Vulnerability Spec store bypass: Vulnerable\n",
"Vulnerability Spectre v1: Mitigation; __user pointer sanitization\n",
"Vulnerability Spectre v2: Vulnerable\n",
"Vulnerability Srbds: Not affected\n",
"Vulnerability Tsx async abort: Not affected\n",
"Flags: fp asimd evtstrm crc32 cpuid\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# Installation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install system-wide prerequisites\n",
"\n",
"**NB:** Run the below commands for your Linux system with `sudo` or as superuser.\n",
"\n",
"\n",
"#### Ubuntu 20.04 or similar\n",
"\n",
"\n",
"$ sudo apt update -y\n",
"$ sudo apt install -y apt-utils\n",
"$ sudo apt upgrade -y\n",
"$ sudo apt install -y\\\n",
" python3 python3-pip gcc g++\\\n",
" autoconf autogen libtool make cmake patch\\\n",
" git curl wget zip libz-dev libssl-dev vim\n",
"$ sudo apt clean\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install [Collective Knowledge](http://cknowledge.org/) (CK)\n",
"\n",
"yvan@diviniti:~$ export CK_PYTHON=/usr/bin/python3\n",
"yvan@diviniti:~$ $CK_PYTHON -m pip install --ignore-installed pip setuptools testresources --user\n",
"yvan@diviniti:~$ $CK_PYTHON -m pip install ck\n",
"yvan@diviniti:~$ echo 'export PATH=$HOME/.local/bin:$PATH' >> $HOME/.bashrc\n",
"yvan@diviniti:~$ source $HOME/.bashrc\n",
"yvan@diviniti:~$ ck version\n",
"V1.55.2\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install CK repositories\n",
"\n",
"### Install public repositories\n",
"\n",
"\n",
"yvan@diviniti:~$ ck pull repo --url=https://github.com/krai/ck-mlperf\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use generic Linux settings with dummy frequency setting scripts\n",
"\n",
"yvan@diviniti:~$ ck detect platform.os --platform_init_uoa=generic-linux-dummy\n",
"\n",
"OS CK UOA: linux-64 (4258b5fe54828a50)\n",
"\n",
"OS name: Ubuntu 20.04.1 LTS\n",
"Short OS name: Linux 5.4.0\n",
"Long OS name: Linux-5.4.0-58-generic-x86_64-with-glibc2.29\n",
"OS bits: 64\n",
"OS ABI: x86_64\n",
"\n",
"Platform init UOA: -\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Detect (system) Python\n",
"\n",
"\n",
"yvan@diviniti:~$ export CK_PYTHON=/usr/bin/python3\n",
"yvan@diviniti:~$ ck detect soft:compiler.python --full_path=$CK_PYTHON\n",
"yvan@diviniti:~$ ck show env --tags=compiler,python\n",
"Env UID: Target OS: Bits: Name: Version: Tags:\n",
"\n",
"6dd55d431ea19241 linux-64 64 python 3.8.5 64bits,compiler,host-os-linux-64,lang-python,python,target-os-linux-64,v3,v3.8,v3.8.5\n",
"\n",
"\n",
"**NB:** CK can normally detect available Python interpreters automatically, but we are playing safe here."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Detect (system) GCC\n",
"\n",
"\n",
"yvan@diviniti:~$ export CK_CC=/usr/bin/gcc\n",
"yvan@diviniti:~$ ck detect soft:compiler.gcc --full_path=$CK_CC\n",
"yvan@diviniti:~$ ck show env --tags=compiler,gcc\n",
"Env UID: Target OS: Bits: Name: Version: Tags:\n",
"\n",
"ebd4a04ea0ae7003 linux-64 64 GNU C compiler 9.3.0 64bits,compiler,gcc,host-os-linux-64,lang-c,lang-cpp,target-os-linux-64,v9,v9.3,v9.3.0\n",
"\n",
"**NB:** CK can normally detect compilers automatically, but we are playing safe here."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install Python dependencies (in userspace)\n",
"\n",
"#### Install implicit dependencies via pip\n",
"\n",
"**NB:** These dependencies are _implicit_, i.e. CK will not try to satisfy them. If they are not installed, however, the workflow will fail.\n",
"\n",
"\n",
"$ export CK_PYTHON=/usr/bin/python3\n",
"$ $CK_PYTHON -m pip install --user --upgrade \\\n",
" ...\n",
"...\n",
"Successfully installed...\n",
"\n",
"\n",
"#### Install explicit dependencies via CK (also via `pip`, but register with CK at the same time)\n",
"\n",
"**NB:** These dependencies are _explicit_, i.e. CK will try to satisfy them automatically. On a machine with multiple versions of Python, things can get messy, so we are playing safe here.\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=python-package,numpy
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install the MLPerf Inference repo and build LoadGen\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=mlperf,inference,source\n",
"yvan@diviniti:~$ ck install package --tags=python-package,mlperf,loadgen\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install a small dataset\n",
"yvan@diviniti:~$ ck install package:imagenet-2012-val-min
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Preprocess the small dataset \n",
"\n",
"\n",
"### Using OpenCV\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=dataset,imagenet,full,preprocessed,using-opencv\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Detect (system) CMake or install CMake from source\n",
"\n",
"### Detect\n",
"\n",
"Try to detect CMake on your system:\n",
"\n",
"yvan@diviniti:~$ ck detect soft --tags=tool,cmake\n",
"yvan@diviniti:~$ ck show env --tags=tool,cmake\n",
"Env UID: Target OS: Bits: Name: Version: Tags:\n",
"\n",
"bed4e78b070155ee linux-64 64 cmake 3.16.3 64bits,cmake,host-os-linux-64,target-os-linux-64,tool,v3,v3.16,v3.16.3\n",
"\n",
"\n",
"### Install\n",
"\n",
"If this fails, try to install it from source:\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=tool,cmake,from.source\n",
"yvan@diviniti:~$ ck show env --tags=tool,cmake,from.source\n",
"Env UID: Target OS: Bits: Name: Version: Tags:\n",
"\n",
"d4559b7302a3cfda linux-64 64 cmake 3.18.2 64bits,cmake,compiled,compiled-by-gcc,compiled-by-gcc-9.3.0,from.source,host-os-linux-64,source,target-os-linux-64,tool,v3,v3.18,v3.18.2\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install the TFLite library (inference engine)\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=lib,tflite,via-cmake\n",
"\n",
"### `xavier`\n",
"\n",
"arjun@xavier:~$ ck install package --tags=lib,via-cmake,with.ruy\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Install the ResNet50 model\n",
"\n",
"yvan@diviniti:~$ ck install package --tags=model,tflite,resnet50,no-argmax\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Quick test\n",
"\n",
"yvan@diviniti:~$ ck benchmark program:image-classification-tflite \\\n",
"--speed --repetitions=1 --skip_print_timers \\\n",
"--dep_add_tags.images=preprocessed,using-opencv \\\n",
"--dep_add_tags.weights=resnet \\\n",
"--dep_add_tags.library=tflite,via-cmake \\\n",
"--env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=10\n",
"...\n",
"Summary:\n",
"-------------------------------\n",
"Graph loaded in 0.000649s\n",
"All images loaded in 0.007372s\n",
"All images classified in 1.482165s\n",
"Average classification time: 0.134372s\n",
"Accuracy top 1: 0.8 (8 of 10)\n",
"Accuracy top 5: 1.0 (10 of 10)\n",
"--------------------------------\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# Benchmarking accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Single Stream\n",
"\n",
"### 50 samples\n",
"\n",
"yvan@diviniti:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=AccuracyOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=50 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 50\n",
"\n",
"Loading graph...\n",
"Input tensor dimensions (NHWC): 1*224*224*3\n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: AccuracyOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"U\n",
" (post processing via CK (/home/yvan/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"accuracy=80.000%, good=40, total=50\n",
"\n",
"--------------------------------\n",
"\n",
"\n",
" (reading fine grain timers from tmp-ck-timer.json ...)\n",
"\n",
"\n",
"Execution time: 0.000 sec.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 500 samples\n",
"\n",
"yvan@diviniti:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=AccuracyOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph...\n",
"Input tensor dimensions (NHWC): 1*224*224*3\n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: AccuracyOnly \n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"U\n",
" (post processing via CK (/home/yvan/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"accuracy=75.200%, good=376, total=500\n",
"\n",
"--------------------------------\n",
"\n",
"\n",
" (reading fine grain timers from tmp-ck-timer.json ...)\n",
"\n",
"\n",
"Execution time: 0.000 sec.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Offline\n",
"\n",
"### 50 samples\n",
"\n",
"yvan@diviniti:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=AccuracyOnly --env.CK_LOADGEN_SCENARIO=Offline \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=50 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 50\n",
"\n",
"Loading graph...\n",
"Input tensor dimensions (NHWC): 1*224*224*3\n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: Offline\n",
"LoadGen Mode: AccuracyOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"U\n",
" (post processing via CK (/home/yvan/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"accuracy=80.000%, good=40, total=50\n",
"\n",
"--------------------------------\n",
"\n",
"\n",
" (reading fine grain timers from tmp-ck-timer.json ...)\n",
"\n",
"\n",
"Execution time: 0.000 sec.\n",
"\n",
"\n",
"### 500 samples\n",
"\n",
"arjun@rpi4coral:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=AccuracyOnly --env.CK_LOADGEN_SCENARIO=Offline \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/arjun/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/arjun/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph...\n",
"Input tensor dimensions (NHWC): 1*224*224*3\n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/arjun/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: Offline\n",
"LoadGen Mode: AccuracyOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"Qpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp\n",
"U\n",
" (post processing via CK (/home/arjun/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"accuracy=75.200%, good=376, total=500\n",
"\n",
"--------------------------------\n",
"\n",
"\n",
" (reading fine grain timers from tmp-ck-timer.json ...)\n",
"\n",
"\n",
"Execution time: 0.000 sec.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# Benchmarking performance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Single Stream\n",
"\n",
"A valid SingleStream performance run [must reach](https://github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc#3-scenarios) a) the minimum duration of 60 seconds (**NB: increased to 600 seconds for v1.0**), _and_ b) the minimum of 1,024 queries. Increasing the _expected_ SingleStream target latency in `user.conf` from 10 milliseconds to above ~60 milliseconds decreases the number of queries that LoadGen issues from 6,000 (actually, 12,000 to account for variability) to 1,024. Note that it does not matter whether the expected latency is, say, 100 ms or 1000 ms, as long as it is above ~60 ms."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`diviniti`](#platforms_diviniti)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@xavier:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_TARGET_LATENCY=75 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
" \n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 1024\n",
"Min latency: 129675251ns (7.71157 fps)\n",
"Median latency: 132842137ns (7.52773 fps)\n",
"Average latency: 134556470ns (7.43182 fps)\n",
"90 percentile latency: 141582128ns (7.06304 fps)\n",
"Max latency: 161307411ns (6.19934 fps)\n",
"------------------------------------------------------------ \n",
"U\n",
" (post processing via CK (/home/yvan/CK/ck-mlperf/script/image-classification, loadgen_postprocess)\n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"yvan@diviniti:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt
\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Single Stream\n",
"Mode : Performance\n",
"90th percentile latency (ns) : 141582128\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"QPS w/ loadgen overhead : 7.43\n",
"QPS w/o loadgen overhead : 7.43\n",
"\n",
"Min latency (ns) : 129675251\n",
"Max latency (ns) : 161307411\n",
"Mean latency (ns) : 134556470\n",
"50.00 percentile latency (ns) : 132842137\n",
"90.00 percentile latency (ns) : 141582128\n",
"95.00 percentile latency (ns) : 145700184\n",
"97.00 percentile latency (ns) : 148383642\n",
"99.00 percentile latency (ns) : 153034511\n",
"99.90 percentile latency (ns) : 159634704\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 1\n",
"target_qps : 13.3333\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1024\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"No warnings encountered during test.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`rising`](#platforms_rising)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"yvan@rising:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_TARGET_LATENCY=45 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 1399\n",
"Min latency: 41049908ns (24.3606 fps)\n",
"Median latency: 42518497ns (23.5192 fps)\n",
"Average latency: 42918601ns (23.2999 fps)\n",
"90 percentile latency: 44370188ns (22.5377 fps)\n",
"Max latency: 58469132ns (17.103 fps)\n",
"------------------------------------------------------------\n",
"U\n",
" (post processing via CK (/home/yvan/CK/ck-mlperf/script/image-classification, loadgen_postprocess)\n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"yvan@rising:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Single Stream\n",
"Mode : Performance\n",
"90th percentile latency (ns) : 44370188\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"QPS w/ loadgen overhead : 23.30\n",
"QPS w/o loadgen overhead : 23.30\n",
"\n",
"Min latency (ns) : 41049908\n",
"Max latency (ns) : 58469132\n",
"Mean latency (ns) : 42918601\n",
"50.00 percentile latency (ns) : 42518497\n",
"90.00 percentile latency (ns) : 44370188\n",
"95.00 percentile latency (ns) : 46449159\n",
"97.00 percentile latency (ns) : 47475951\n",
"99.00 percentile latency (ns) : 49663623\n",
"99.90 percentile latency (ns) : 56845302\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 1\n",
"target_qps : 22.2222\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1024\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"No warnings encountered during test.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`xavier`](#platforms_xavier)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@xavier:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_TARGET_LATENCY=600 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/arjun/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/arjun/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 1024\n",
"Min latency: 74525667ns (13.4182 fps)\n",
"Median latency: 76175177ns (13.1276 fps)\n",
"Average latency: 77628573ns (12.8819 fps)\n",
"90 percentile latency: 81057888ns (12.3369 fps)\n",
"Max latency: 136391571ns (7.33183 fps)\n",
"------------------------------------------------------------\n",
"U\n",
" (post processing via CK (/home/arjun/CK/ck-mlperf/script/image-classification, loadgen_postprocess)\n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"arjun@xavier:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Single Stream\n",
"Mode : Performance\n",
"90th percentile latency (ns) : 81057888\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"QPS w/ loadgen overhead : 12.88\n",
"QPS w/o loadgen overhead : 12.88\n",
"\n",
"Min latency (ns) : 74525667\n",
"Max latency (ns) : 136391571\n",
"Mean latency (ns) : 77628573\n",
"50.00 percentile latency (ns) : 76175177\n",
"90.00 percentile latency (ns) : 81057888\n",
"95.00 percentile latency (ns) : 88910582\n",
"97.00 percentile latency (ns) : 90713765\n",
"99.00 percentile latency (ns) : 93834349\n",
"99.90 percentile latency (ns) : 111686370\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 1\n",
"target_qps : 1.66667\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1024\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"No warnings encountered during test.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`rpi4coral`](#platforms_rpi4coral)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@rpi4coral:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=SingleStream \\\n",
"--env.CK_LOADGEN_TARGET_LATENCY=600 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/arjun/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/arjun/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/arjun/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: SingleStream\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"QpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQpQp\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 1024\n",
"Min latency: 470279185ns (2.1264 fps)\n",
"Median latency: 510943336ns (1.95716 fps)\n",
"Average latency: 518517909ns (1.92857 fps)\n",
"90 percentile latency: 565778889ns (1.76747 fps)\n",
"Max latency: 693473624ns (1.44202 fps)\n",
"------------------------------------------------------------\n",
"U\n",
" (post processing via CK (/home/arjun/CK/ck-mlperf/script/image-classification, loadgen_postprocess)\n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"arjun@rpi4coral:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Single Stream\n",
"Mode : Performance\n",
"90th percentile latency (ns) : 565778889\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"QPS w/ loadgen overhead : 1.93\n",
"QPS w/o loadgen overhead : 1.93\n",
"\n",
"Min latency (ns) : 470279185\n",
"Max latency (ns) : 693473624\n",
"Mean latency (ns) : 518517909\n",
"50.00 percentile latency (ns) : 510943336\n",
"90.00 percentile latency (ns) : 565778889\n",
"95.00 percentile latency (ns) : 578424410\n",
"97.00 percentile latency (ns) : 583799548\n",
"99.00 percentile latency (ns) : 610083186\n",
"99.90 percentile latency (ns) : 667977252\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 1\n",
"target_qps : 1.66667\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1024\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"No warnings encountered during test.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Offline\n",
"\n",
"A valid Offline performance run [must reach](https://github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc#3-scenarios) a) the minimum duration of 60 seconds (**NB: increased to 600 seconds for v1.0**), _and_ b) the minimum of 24,576 samples."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`diviniti`](#platforms_diviniti)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"yvan@diviniti:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=Offline \\\n",
"--env.CK_LOADGEN_TARGET_QPS=8 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/yvan/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/yvan/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/yvan/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: Offline\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllQpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp...\n",
"\n",
"...\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 24576 \n",
"Min latency: 3204754749738ns (0.000312036 fps)\n",
"Median latency: 3204754749738ns (0.000312036 fps)\n",
"Average latency: 3204754749738ns (0.000312036 fps)\n",
"90 percentile latency: 3204754749738ns (0.000312036 fps)\n",
"Max latency: 3204754749738ns (0.000312036 fps)\n",
"------------------------------------------------------------ \n",
"U\n",
" (post processing via CK (/home/anton/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
"\n",
"\n",
"real 53m35.422s\n",
"user 363m38.614s\n",
"sys 1m50.982s\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"yvan@diviniti:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Offline\n",
"Mode : Performance\n",
"Samples per second : 7.66861\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"Min latency (ns) : 3204754749738\n",
"Max latency (ns) : 3204754749738\n",
"Mean latency (ns) : 3204754749738\n",
"50.00 percentile latency (ns) : 3204754749738\n",
"90.00 percentile latency (ns) : 3204754749738\n",
"95.00 percentile latency (ns) : 3204754749738\n",
"97.00 percentile latency (ns) : 3204754749738\n",
"99.00 percentile latency (ns) : 3204754749738\n",
"99.90 percentile latency (ns) : 3204754749738\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 24576\n",
"target_qps : 8\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"1 warning encountered. See detailed log.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"yvan@diviniti:~/CK$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_detail.txt | grep -i warning\n",
""pid": 63432, "tid": 63432, "ts": 3204804686273ns : WARNING : Logging allocation detected: "tid" : 63432, "reserved_entries" : 1024, "max_entries" : 24580\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`xavier`](#platforms_xavier)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@xavier:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=Offline \\\n",
"--env.CK_LOADGEN_TARGET_QPS=13 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/arjun/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /datasets/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/arjun/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: Offline\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"Qppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp........\n",
"\n",
"...\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 24576 \n",
"Min latency: 1877627079157ns (0.000532587 fps)\n",
"Median latency: 1877627079157ns (0.000532587 fps)\n",
"Average latency: 1877627079157ns (0.000532587 fps)\n",
"90 percentile latency: 1877627079157ns (0.000532587 fps)\n",
"Max latency: 1877627079157ns (0.000532587 fps)\n",
"------------------------------------------------------------ \n",
"U\n",
" (post processing via CK (/home/arjun/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
"\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"arjun@xavier:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Offline\n",
"Mode : Performance\n",
"Samples per second : 13.0889\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"Min latency (ns) : 1877627079157\n",
"Max latency (ns) : 1877627079157\n",
"Mean latency (ns) : 1877627079157\n",
"50.00 percentile latency (ns) : 1877627079157\n",
"90.00 percentile latency (ns) : 1877627079157\n",
"95.00 percentile latency (ns) : 1877627079157\n",
"97.00 percentile latency (ns) : 1877627079157\n",
"99.00 percentile latency (ns) : 1877627079157\n",
"99.90 percentile latency (ns) : 1877627079157\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 24576\n",
"target_qps : 13\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"1 warning encountered. See detailed log.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@xavier:~/CK$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_detail.txt | grep -i warning\n",
""pid": 21838, "tid": 21838, "ts": 1877743914633ns : WARNING : Logging allocation detected: "tid" : 21838, "reserved_entries" : 1024, "max_entries" : 24537\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### [`rpi4coral`](#platforms_rpi4coral)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@rpi4coral:~$ ck benchmark program:image-classification-tflite-loadgen \\\n",
"--dep_add_tags.weights=resnet50 --dep_add_tags.images=preprocessed,using-opencv \\\n",
"--env.CK_LOADGEN_MODE=PerformanceOnly --env.CK_LOADGEN_SCENARIO=Offline \\\n",
"--env.CK_LOADGEN_TARGET_QPS=2 \\\n",
"--env.CK_LOADGEN_DATASET_SIZE=500 --env.CK_LOADGEN_BUFFER_SIZE=1024 \\\n",
"--repetitions=1 --skip_print_timers\n",
"...\n",
"Graph file: /home/arjun/CK-TOOLS/model-tflite-mlperf-resnet-no-argmax-downloaded/resnet50_v1.no-argmax.tflite\n",
"Image dir: /home/arjun/CK-TOOLS/dataset-imagenet-preprocessed-using-opencv-crop.875-full-inter.linear-side.224-universal-unmutilated\n",
"Image list: image_list.txt\n",
"Image size: 224\n",
"Image channels: 3\n",
"Prediction classes: 1000\n",
"Result dir: predictions\n",
"How many images fit in memory buffer: 1024\n",
"Normalize: 0\n",
"Subtract mean: 1\n",
"Per-channel means to subtract: 123.68, 116.78, 103.94\n",
"Number of available imagefiles: 500\n",
"\n",
"Loading graph... \n",
"Input tensor dimensions (NHWC): 1*224*224*3 \n",
"Output tensor dimensions: 1*1001\n",
"Path to mlperf.conf : /home/arjun/CK-TOOLS/mlperf-inference-r0.7/inference/mlperf.conf\n",
"Path to user.conf : user.conf\n",
"Model Name: resnet50\n",
"LoadGen Scenario: Offline\n",
"LoadGen Mode: PerformanceOnly\n",
"CBllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll\n",
"Qppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp........\n",
"\n",
"...\n",
"\n",
"------------------------------------------------------------\n",
"| LATENCIES (in nanoseconds and fps) |\n",
"------------------------------------------------------------\n",
"Number of queries run: 24576 \n",
"Min latency: 14316963274310ns (6.98472e-05 fps)\n",
"Median latency: 14316963274310ns (6.98472e-05 fps)\n",
"Average latency: 14316963274310ns (6.98472e-05 fps)\n",
"90 percentile latency: 14316963274310ns (6.98472e-05 fps)\n",
"Max latency: 14316963274310ns (6.98472e-05 fps)\n",
"------------------------------------------------------------ \n",
"U\n",
" (post processing via CK (/home/arjun/CK/ck-mlperf/script/image-classification, loadgen_postprocess) \n",
"\n",
"\n",
"--------------------------------\n",
"--------------------------------\n",
"\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"arjun@rpi4coral:~$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_summary.txt\n",
"\n",
"\n",
"================================================\n",
"MLPerf Results Summary\n",
"================================================\n",
"SUT name : TFLite_SUT\n",
"Scenario : Offline\n",
"Mode : Performance\n",
"Samples per second : 1.71657\n",
"Result is : VALID\n",
" Min duration satisfied : Yes\n",
" Min queries satisfied : Yes\n",
"\n",
"================================================\n",
"Additional Stats\n",
"================================================\n",
"Min latency (ns) : 14316963274310\n",
"Max latency (ns) : 14316963274310\n",
"Mean latency (ns) : 14316963274310\n",
"50.00 percentile latency (ns) : 14316963274310\n",
"90.00 percentile latency (ns) : 14316963274310\n",
"95.00 percentile latency (ns) : 14316963274310\n",
"97.00 percentile latency (ns) : 14316963274310\n",
"99.00 percentile latency (ns) : 14316963274310\n",
"99.90 percentile latency (ns) : 14316963274310\n",
"\n",
"================================================\n",
"Test Parameters Used\n",
"================================================\n",
"samples_per_query : 24576\n",
"target_qps : 2\n",
"target_latency (ns): 0\n",
"max_async_queries : 1\n",
"min_duration (ms): 60000\n",
"max_duration (ms): 0\n",
"min_query_count : 1\n",
"max_query_count : 0\n",
"qsl_rng_seed : 12786827339337101903\n",
"sample_index_rng_seed : 12640797754436136668\n",
"schedule_rng_seed : 3135815929913719677\n",
"accuracy_log_rng_seed : 0\n",
"accuracy_log_probability : 0\n",
"accuracy_log_sampling_target : 0\n",
"print_timestamps : false\n",
"performance_issue_unique : false\n",
"performance_issue_same : false\n",
"performance_issue_same_index : 0\n",
"performance_sample_count : 1024\n",
"\n",
"1 warning encountered. See detailed log.\n",
"\n",
"No errors encountered during test.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"arjun@rpi4coral:~/CK$ cat $(ck find program:image-classification-tflite-loadgen)/tmp/mlperf_log_detail.txt | grep -i warning\n",
""pid": 1969535, "tid": 1969535, "ts": 14317167368970ns : WARNING : Logging allocation detected: "tid" : 1969535, "reserved_entries" : 1024, "max_entries" : 20077\n",
""
]
}
],
"metadata": {
"celltoolbar": "Raw Cell Format",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}