cK-powered report

Check the preview of 2nd version of this platform being developed by the open MLCommons taskforce on automation and reproducibility as a free, open-source and technology-agnostic on-prem platform.

Pamela project - reproducing slambench multi-objective autotuning (FPS vs accuracy vs other characteristics)

Demo of the parametric, multi-objective autotuning of slambench via CK

Graph:

Graph params (to customize/reproduce):

{
  "axis_x_desc": "Accuracy (m.)",
  "axis_y_desc": "FPS (frames per second)",
  "data_uoa_list": [
    "algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-i40",
    "algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-i40-pareto-fps",
    "algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-default"
  ],
  "display_x_error_bar": "no",
  "display_y_error_bar": "yes",
  "experiment_module_uoa": "experiment",
  "flat_keys_list": [
    "##characteristics#run#accuracy_ate_max#max",
    "##characteristics#run#fps#center",
    "##characteristics#run#fps#halfrange"
  ],
  "ignore_point_if_empty_string": "yes",
  "plot_type": "d3_2d_scatter",
  "point_style": {
    "0": {
      "color": "#3366cc",
      "size": 4
    },
    "1": {
      "color": "#dc3912",
      "size": 5,
      "type": "rect"
    },
    "2": {
      "color": "#00df00",
      "size": 5,
      "type": "rect"
    }
  },
  "save_info_table_to_json_file": "2d_points_fps_vs_accuracy_with_pareto_interactive.json",
  "save_table_to_csv_file": "2d_points_fps_vs_accuracy_with_pareto_interactive.csv",
  "save_table_to_json_file": "2d_points_fps_vs_accuracy_with_pareto_interactive.json",
  "size_x": 700,
  "size_y": 400,
  "title": "Powered by Collective Knowledge"
}

[ ] [ ] Auto-replot graph: , seconds:

Reproducing graph:

Experiment entries:	algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-i40 algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-i40-pareto-fps algorithm-exploration-slambench-opencl-samsung-chromebook2-plat0-dev0-default
Scripts to rebuild:	ck find script:64fbdfa1d9c28942 View in CK viewer

Obtain shared CK repo with all artifacts:	ck pull repo:reproduce-pamela-project
Original experiment table:	Download in JSON; Download in CSV