Go directly to the demo of a portable ML solution (object detection, TF, MobileNets, COCO)

MLPerf crowd-benchmarking    Webcam-based live test    ML/SW/HW autotuning scoreboard

Our mission is to help researchers and practitioners find reproduced papers with portable, reusable and open-source components (models, data sets, frameworks, libraries, tools) for emerging digital technologies (AI, ML, quantum, IoT), assemble functional solutions, understand how to use them, and participate in collaborative benchmarking and optimization of novel techniques using live scoreboards. Here you can find and try several real use cases.

Note that this is an ongoing community project and there is still a lot to be done - please get in touch if you have suggestions or encounter any issues. Thank you!

Participate in collaborative ML&systems benchmarking

Preparing, submitting and reproducing ML benchmarking results is a very tedious, ad-hoc and time consuming process. Check these CK solutions to learn how to automatically install and run real applications from the MLPerf benchmark on your platform in a few relatively simple steps: You can then participate in collaborative benchmarking to validate MLPerf results and submit the new ones using these live CK scoreboards. You can also see all dependencies on reusable components from this portal required to assemble this portable solution.

Test the above object detection solution in practice

Besides looking at benchmarking results, we also want to test research techniques in practice on real data sets. You can test how the above object detection solution works in practice in your browser.

See other reproduced results (AI, ML, quantum, IoT)

Follow this link to find reproduced results from open competitions, reproducible hackathons and collaborative benchmarking efforts we helped to organize since 2015.

Check our concept of a live research paper

See the live paper with reusable workflows to apply machine learning to compilers (our collaboration with the Raspberry Pi foundation).

Create your own dashboard for crowd-benchmarking

Please follow this documentaiton about how to create your own customized dashboard for crowd-benchmarking and live research papers.

Create your own portable CK solution for crowd-benchmarking

Preliminary documentation (MLPerf benchmark automation example)