Shared results from collaborative experiments

Live CK scoreboards are connected with portable CK workflows and adaptive containers to help the community participate in collaborative AI/ML/SW/HW benchmarking and DSE. See this paper and ACM TechTalk for more details.

Benchmark results (performance): P_*_SS - Single Stream in milliseconds, P_*_MS - MultiStream in no. streams, P_*_S - Server in QPS, P_*_O - Offline in inputs/second.
Benchmark results (accuracy): A_IC* - Top-1, A_OD* - mAP, A_NMT* - BLEU.
cK components: packages, software detection plugins.
Image Classification: IC1 - ImageNet, MobileNet-v1, IC2 - ImageNet, ResNet-50 v1.5.
Object detection: OD1 - COCO, SSD w/ MobileNet-v1, OD2 - COCO 1200x1200, SSD w/ ResNet-34.
Translation: NMT - WMT E-G, NMT.
Form Factor: FF_M - Mobile/Handheld, FF_D - Desktop/Workstation, FF_S - Server, FF_E - Edge/Embedded.
These are not official results but a snapshot to collaboratively reproduce results and add portable workflows!
MLPerf name and logo are trademarks. See www.mlperf.org for more information.

Reproduced paper: MLPerf Inference Benchmark
Related reusable solutions: 1 2 3 4 5 6 7

SOTA: MLPerf inference benchmark v0.5 results snapshot (open, Research, Development, and Other) for collaborative validation

Versions

1.0.0 (2020-04-12)

Files

.cm/desc.json (3 bytes)
.cm/info.json (577 bytes)
.cm/meta.json (2435 bytes)
.cm/updates.json (868 bytes)
desc.json (4974 bytes)
result-1.json (1421 bytes)
result-10.json (1543 bytes)
result-11.json (1559 bytes)
result-12.json (1556 bytes)
result-13.json (1540 bytes)
result-14.json (1543 bytes)
result-15.json (1531 bytes)
result-2.json (1432 bytes)
result-3.json (1433 bytes)
result-4.json (1562 bytes)
result-5.json (1541 bytes)
result-6.json (1547 bytes)
result-7.json (1551 bytes)
result-8.json (1550 bytes)
result-9.json (1538 bytes)

Comments

Please log in to add your comments!

If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!

SOTA: MLPerf inference benchmark v0.5 results snapshot (open, Research, Development, and Other) for collaborative validation

Versions Hide

Files Hide

Comments Hide

Versions

Files

Comments