@@ -8,8 +8,9 @@ a small set of example designs through our complete RTL-to-GDS flow.
88To keep track of the quality of the results, every CI run uploads its metrics
99to the [ QoR dashboard] ( https://dashboard.precisioninno.com ) . The dashboard
1010stores one rule per tracked metric and design (a comparison direction and a
11- tolerance), and judges a run against a real baseline build: the latest
12- ` master ` build by default. That check is the QoR gate in ` make metadata ` .
11+ tolerance), and judges a run against a real baseline build: the ` master `
12+ build at the run's merge base by default. That check is the QoR gate in
13+ ` make metadata ` .
1314
1415The values a run is measured against are collected by
1516` flow/util/genMetrics.py ` into ` $(REPORTS_DIR)/metadata.json `
@@ -47,12 +48,52 @@ check did not run. Read the `[WARN]` lines to see which. Pass `--strict` to
4748` checkQorMetrics.py ` to get distinct exit codes instead (2 inconclusive,
48493 unreachable).
4950
50- Two environment variables shape the comparison:
51-
52- - ` DASHBOARD_API_KEY ` grants access to private platforms. Without it, a
53- private design reports as inconclusive, not as an authentication error.
54- - ` DASHBOARD_JOB_NAME ` names the Jenkins pipeline whose ` master ` builds are the
55- baseline. The default is ` OpenROAD-flow-scripts-Public ` .
51+ The check runs as the commit ` metadata.json ` records. The dashboard searches
52+ its default baseline pipelines in order and takes a ` master ` build that has the
53+ design: the one at the commit's merge base with ` master ` . A commit on ` master `
54+ is its own merge base, so a run of a ` master ` commit, such as a downloaded CI
55+ artifact, is compared against that commit's own build.
56+
57+ The dashboard finds the merge base through GitHub, so it cannot find one for a
58+ commit that was never pushed. The check handles that commit itself: when no
59+ remote branch in your checkout contains the commit, it computes
60+ ` git merge-base <commit> origin/master ` locally and pins the baseline to that
61+ build. A commit that is on a remote, or that is not in your checkout at all, is
62+ left to the dashboard.
63+
64+ When the dashboard has no ` master ` build at the merge base, or no merge base,
65+ it compares against the latest ` master ` build instead (the latest one before
66+ the commit, if it knows the commit's date). That build may be newer than the
67+ commit's base, so changes merged since then can show up as regressions or hide
68+ real ones, and the result moves with ` master ` . The report warns whenever this
69+ happens; set ` DASHBOARD_BASE_COMMIT ` to compare against an exact build instead.
70+
71+ These environment variables shape the comparison:
72+
73+ - ` DASHBOARD_API_KEY ` grants access to private platforms. With it, a private
74+ design is resolved like a public one: the dashboard's default baseline
75+ pipelines include the private ones, and only a caller with a key can see
76+ their builds. Without it, a private design has no baseline and reports as
77+ inconclusive, not as an authentication error.
78+ - ` DASHBOARD_JOB_NAME ` checks the run as a build of that CI pipeline instead,
79+ with the pipeline's rules for the branch checked out in this repository: the
80+ previous build on a trunk branch such as ` master ` , the merge base on any
81+ other branch. It is meant for CI jobs, which also set ` DASHBOARD_BRANCH ` and
82+ ` DASHBOARD_COMMIT ` (or pass ` --branch ` and ` --commit ` ) since their checkout
83+ has no branch. Earlier versions of the check used it to pick the pipeline
84+ that holds a private platform's baseline; that is no longer needed, so leave
85+ it unset for a local run, private or not. With it set, an unpushed commit is
86+ not pinned to its local merge base.
87+ - ` DASHBOARD_MERGE_BASE ` (` --merge-base ` ) names the ref an unpushed commit's
88+ merge base is computed against, ` origin/master ` by default. Set it to your
89+ upstream ref if ` origin ` is a fork with an outdated ` master ` , or to ` none ` to
90+ leave unpushed commits to the dashboard. It does not apply together with
91+ ` DASHBOARD_BASE_COMMIT ` or ` DASHBOARD_JOB_NAME ` .
92+ - ` DASHBOARD_BASE_COMMIT ` (` --base-commit ` ) pins the baseline to the build of
93+ that commit. Nothing else is tried: if that commit has no build of the
94+ design, the check is inconclusive.
95+ - ` DASHBOARD_API_URL ` (` --api-url ` ) points the check at another deployment and
96+ takes precedence over ` --beta ` .
5697
5798If you only changed synthesis, you can gate a synthesis-only run, without
5899running the rest of the flow. Only the ` synth__ ` and ` constraints__ ` metrics
@@ -62,8 +103,7 @@ are sent, so only their rules are evaluated:
62103make metadata-synth
63104```
64105
65- To check every run under ` reports/ ` at once, or to pin the baseline to a
66- specific commit, call the script directly:
106+ To check every run under ` reports/ ` at once, call the script directly:
67107
68108``` shell
69109cd OpenROAD-flow-scripts/flow
0 commit comments