" Input: { data_uoa - experiment data UOA (can have wildcards) (repo_uoa) - experiment repo UOA (can have wildcards) (experiment_repo_uoa) - use it, if repository is remote (remote_repo_uoa) - if repo above is remote, use this repo on remote machine (module_uoa) (tags) - search by tags (point) - point (or skip, if there is only one), can be of format UID-(subpoint) - subpoint (or skip, if there is only one) (repetitions) - statistical repetitions (default=4) (pipeline_update) - customize pipeline with this dict (useful to update already prepared pipeline from file) (dims_to_check) or (dims) - list of dimensions to check, can have wildcards (if empty, check all); alternatively can use a string (useful for CMD) (end_of_dims_to_check) - list of endings of dimensions to compare ... (threshold_to_compare) - % threshold to decide whether dim is different or not (record_original_flat_json) - file to record flat dict with original results (record_reproduced_flat_json) - file to record flag dict with reproduced results (skip) - if 'yes', do not perform comparison ============================== If one would like to compare results with another point (for example, to check speedups rather than just reproducing a given point) (compare_data_uoa) (compare_repo_uoa) (compare_point) (compare_subpoint) ============================== Prunning (reducing complexity) (prune) - if 'yes', replay and prune choices! (reduce) - the same as above (reduce_bug) - reduce choices to localize bug (pipeline fail) (prune_md5) - if 'yes' and compilation is used in a pipeline (workflow), check if binary MD5 doesn't change (prune_invert) - if 'yes', prune all (switch off even unused - useful for collaborative machine learning) (prune_print_keys) - list of keys from flat dict to print during pruning (to monitor characteristics, for example) (prune_invert_do_not_remove_key) - if 'yes', keep both on and off keys (to know exact solution) (prune_conditions) - conditions on results (see "ck check math.conditions --help") (condition_objective) - which objective to use for characteristics (#min, #max, #exp, ...) (solutions) - prune first solution ============================== (record_uoa) - (data UOA or CID where module_uoa ignored!) explicitly record to this entry (record_repo) - (repo UOA) explicitly select this repo to record ============================== Some productivity keys specifically for autotuning pipeline (ck-autotuning repo): (local_platform) or (local) - if 'yes', use parameters of a local platform (to retarget experiment) (skip_target) - do not select target machines (to customize via host_os/target_os/device_id) (no_deps) - if 'yes', do not reuse deps (skip_clean_after) - if 'yes', do not clean program pipeline after execution (keeping low level scripts in tmp directory for low-level debugging) (all) - print all comparisons of keys (not only when there is a difference) (skip_scenario_keys) - skip scenario detection (otherwise get keys from the detected module) } Output: { return - return code = 0, if successful > 0, if error (error) - error text if return > 0 output of a given pipeline different_dims - list of different dimensions } "