Record the state of linked merge requests and show it on the page (schema 7) (#33)

Closes #30.

Author: lesnik512

README.md

@@ -93,6 +93,10 @@ The report is versioned by `schema_version`; the models live in
 [`release_scope/_report.py`](https://github.com/modern-python/release-scope/blob/main/release_scope/_report.py).
 Top-level `jira` is `null` without a Jira token; otherwise it holds `issues` by key (summary, status, status category,
 issue type, linked GitLab changes), the `missing` keys Jira did not return, and an `error` if a Jira request failed.
+A linked merge request in a collected service has its `state` (`opened`, `closed`, `locked`, or `merged`) and
+`target_branch`. When a row holds it, it is merged into the default branch and costs no request; otherwise it is read
+from GitLab, and a merged one is cached. The state stays `null` for a commit, for a project outside the report, and when
+the merge request cannot be read.
 Warnings and errors, on services and in `jira`, are messages: a `code`, its `params`, and the English `text`.
 Each service lists its `candidates`: the tags a release could ship, newest first, each with its pipeline, the number of
 rows it ships, the in-scope Jira keys of those rows, and the compare link from production, or the tag's commit history
@@ -151,12 +155,15 @@ reload starts from the release tags again.
 **Issues**, at the top of the Release column when the report has Jira issues, picks tags from issues instead. It lists
 every issue of the report, Done first and then the rest, by Jira's status category, and filters them by key or
 summary or shows only the chosen ones. Under each issue are the services it touches: the tag that ships it, `no tag
-yet`, `in prod or not merged` when it links to a merge request or commit outside the range, `not in this report`, or
-`failed to collect`. Ticking an issue picks, in every service it touches, the tag a `--jira` run would make its
+yet`, `not in this report`, `failed to collect`, or, when it links only to changes outside the range, why: `not
+merged` while a linked merge request is open, `in prod` when one was merged into the default branch (`older than the
+range` when the range was cut), `merged into <branch>`, `closed`, or `in prod or not merged` for a commit or a
+merge request that could not be read. Ticking an issue picks, in every service it touches, the tag a `--jira` run would make its
 release tag: the nearest tag at or above the newest row the issue links to or names. When several chosen issues touch
 a service, the newest of their tags wins; when one of them has no tag yet, the service gets no pick and its line
 links to create the next tag. Each touched service line names the chosen issues it is picked for and the other Jira
-keys its picked tag also ships; in the list those issues are marked `ships anyway`. Services no chosen issue
+keys its picked tag also ships; in the list those issues are marked `ships anyway`. It also says why a chosen issue
+that links only to changes outside the range has nothing to ship there, such as `SHOP-150 not merged`. Services no chosen issue
 touches are dimmed, and projects the issues link to that are not in the report are listed under the services.
 Unticking an issue clears the picks only it needed. A pick made by hand stays until the chosen issues change, and the
 line says which chosen issues it does not ship. The address keeps the issues before the picks, as
@@ -188,7 +195,7 @@ release-report:
   rules:
     - if: $CI_PIPELINE_SOURCE == "schedule"
   script:
-    - uvx --from 'release-scope>=0.8,<0.9' release-scope collect --group team/backend --output public || [ $? -eq 1 ]
+    - uvx --from 'release-scope>=0.9,<0.10' release-scope collect --group team/backend --output public || [ $? -eq 1 ]
   pages: true
 ```
 
@@ -227,4 +234,4 @@ npx skills add modern-python/release-scope
 ```
 
 The agent reads the same environment variables as the CLI, so set them first as described under Configuration.
-The skill runs `release-scope>=0.8,<0.9`, the range whose flags and report schema it describes.
+The skill runs `release-scope>=0.9,<0.10`, the range whose flags and report schema it describes.

release_scope/_report.py

@@ -5,7 +5,7 @@
 import pydantic
 
 
-SCHEMA_VERSION: typing.Final = 6
+SCHEMA_VERSION: typing.Final = 7
 
 
 class MessageCode(enum.StrEnum):
@@ -140,6 +140,8 @@ class LinkedChange(pydantic.BaseModel):
     url: str
     iid: int | None = None
     sha: str | None = None
+    state: str | None = None
+    target_branch: str | None = None
 
 
 class JiraIssue(pydantic.BaseModel):
@@ -159,7 +161,7 @@ class JiraState(pydantic.BaseModel):
 
 
 class Report(pydantic.BaseModel):
-    schema_version: typing.Literal[6] = SCHEMA_VERSION
+    schema_version: typing.Literal[7] = SCHEMA_VERSION
     collected_at: dt.datetime
     production_environment: str
     services: list[Service]

release_scope/_static/index.html

@@ -105,7 +105,9 @@
   .cell { padding: 0 6px; border-radius: 4px; font-size: 12px; white-space: nowrap; }
   .cell-tag { background: color-mix(in srgb, var(--accent) 14%, transparent); color: var(--accent); }
   .cell-untagged { background: color-mix(in srgb, var(--warn) 16%, transparent); color: var(--warn); }
-  .cell-elsewhere { background: color-mix(in srgb, var(--good) 14%, transparent); color: var(--good); }
+  .cell-production, .cell-older { background: color-mix(in srgb, var(--good) 14%, transparent); color: var(--good); }
+  .cell-open, .cell-branch { background: color-mix(in srgb, var(--warn) 16%, transparent); color: var(--warn); }
+  .cell-closed, .cell-elsewhere { background: var(--panel); color: var(--muted); }
   .cell-outside, .cell-failed { background: color-mix(in srgb, var(--bad) 12%, transparent); color: var(--bad); }
   .issue-tools { display: flex; flex-wrap: wrap; gap: 8px; margin: 6px 0; }
   .issue-tools input { flex: 1; min-width: 140px; font: inherit; padding: 2px 8px; border: 1px solid var(--border); border-radius: 6px;
@@ -205,8 +207,8 @@ <h2 x-text="t('services')"></h2>
                       x-text="service.untagged && service.untagged.next_tag ? t('createTag', { name: service.untagged.next_tag }) : t('createAnyTag')"></a>
                   </span>
                 </template>
-                <template x-if="planOf(service).elsewhere.length && !planOf(service).best && !planOf(service).untagged.length">
-                  <span class="muted" x-text="t('planElsewhere', { keys: planOf(service).elsewhere.join(', ') })"></span>
+                <template x-if="planOf(service).elsewhere.length">
+                  <span class="muted" x-text="outOfRangeText(planOf(service))"></span>
                 </template>
                 <template x-if="missedBy(service).length">
                   <span class="warn" x-text="t('planMissed', { tag: picks[service.project], keys: missedBy(service).join(', ') })"></span>
@@ -535,12 +537,17 @@ <h2 x-text="t('release')"></h2>
       shipsAnywayTitle: "Not chosen, but a picked tag ships it",
       reachUntagged: "no tag yet",
       reachElsewhere: "in prod or not merged",
+      reachProduction: "in prod",
+      reachOlder: "older than the range",
+      reachOpen: "not merged",
+      reachClosed: "closed",
+      reachBranch: "merged into {branch}",
       notInReport: "not in this report",
       linkedTo: "linked to",
       planFor: "for",
       planAlso: "also ships",
       planNoTag: "{keys}: no tag yet.",
-      planElsewhere: "{keys}: in production or not merged, nothing to pick",
+      planElsewhere: "{items}, nothing to pick",
       planMissed: "{tag} does not ship {keys}",
     },
     ru: {
@@ -631,12 +638,17 @@ <h2 x-text="t('release')"></h2>
       shipsAnywayTitle: "Задача не выбрана, но выбранный тег её содержит",
       reachUntagged: "нет тега",
       reachElsewhere: "на проде или не влита",
+      reachProduction: "на проде",
+      reachOlder: "старше диапазона",
+      reachOpen: "не влита",
+      reachClosed: "закрыта",
+      reachBranch: "влита в {branch}",
       notInReport: "нет в этом отчёте",
       linkedTo: "связан с",
       planFor: "для",
       planAlso: "также войдут",
       planNoTag: "{keys}: тега ещё нет.",
-      planElsewhere: "{keys}: уже на проде или не влиты, выбирать нечего",
+      planElsewhere: "{items}, выбирать нечего",
       planMissed: "{tag} не содержит {keys}",
     },
   };
@@ -703,6 +715,19 @@ <h2 x-text="t('release')"></h2>
 
   const byKey = (a, b) => a.localeCompare(b, undefined, { numeric: true });
 
+  const OUT_OF_RANGE = new Set(["production", "older", "open", "branch", "closed", "elsewhere"]);
+
+  const outOfRange = (service, links) => {
+    if (links.some((link) => link.state === "opened" || link.state === "locked")) return { state: "open" };
+    if (links.some((link) => link.state === "merged" && link.target_branch === service.default_branch)) {
+      return { state: service.truncated ? "older" : "production" };
+    }
+    const merged = links.find((link) => link.state === "merged");
+    if (merged) return { state: "branch", branch: merged.target_branch };
+    if (links.some((link) => link.state === "closed")) return { state: "closed" };
+    return { state: "elsewhere" };
+  };
+
   const reachOf = (report) => {
     const issues = report.jira ? report.jira.issues : {};
     const reach = Object.fromEntries(Object.keys(issues).map((key) => [key, []]));
@@ -726,7 +751,7 @@ <h2 x-text="t('release')"></h2>
           continue;
         }
         if (!indexes.length) {
-          if (links.length) reach[key].push({ project, state: "elsewhere" });
+          if (links.length) reach[key].push({ project, ...outOfRange(service, links) });
           continue;
         }
         const index = Math.min(...indexes);
@@ -916,7 +941,7 @@ <h2 x-text="t('release')"></h2>
             entry.keys.push(key);
             if (item.state === "tag" && (!entry.best || item.tagIndex < entry.best.tagIndex)) entry.best = item;
             if (item.state === "untagged") entry.untagged.push(key);
-            if (item.state === "elsewhere") entry.elsewhere.push(key);
+            if (OUT_OF_RANGE.has(item.state)) entry.elsewhere.push({ key, ...item });
           }
         }
         return plan;
@@ -941,10 +966,27 @@ <h2 x-text="t('release')"></h2>
       shipsAnyway(key) {
         return !this.chosen.includes(key) && this.picked.some(({ candidate }) => candidate.jira_keys.some((item) => item.key === key));
       },
+      reachLabel(item) {
+        if (item.state === "tag") return item.tag;
+        if (item.state === "branch") return this.t("reachBranch", { branch: item.branch });
+        const states = {
+          untagged: "reachUntagged",
+          production: "reachProduction",
+          older: "reachOlder",
+          open: "reachOpen",
+          closed: "reachClosed",
+          elsewhere: "reachElsewhere",
+          outside: "notInReport",
+          failed: "failedToCollect",
+        };
+        return this.t(states[item.state]);
+      },
+      outOfRangeText(entry) {
+        const items = entry.elsewhere.map((item) => `${item.key} ${this.reachLabel(item)}`).join(", ");
+        return entry.best || entry.untagged.length ? items : this.t("planElsewhere", { items });
+      },
       reachText(item) {
-        const name = item.project.split("/").pop();
-        const states = { untagged: "reachUntagged", elsewhere: "reachElsewhere", outside: "notInReport", failed: "failedToCollect" };
-        return `${name} ${item.state === "tag" ? item.tag : this.t(states[item.state])}`;
+        return `${item.project.split("/").pop()} ${this.reachLabel(item)}`;
       },
       showIssues(view) {
         this.issueView = view;

release_scope/_use_case.py

@@ -154,6 +154,9 @@ def __call__(
         state: typing.Final = self._read_issues(self.jira, _row_keys(services), exclude) if self.jira else None
         if state is not None:
             self._join_linked_issues(services, state)
+            self._record_link_states(
+                services, state, {project.path_with_namespace: project for project in resolved}, cache
+            )
         return Report(
             collected_at=dt.datetime.now(dt.UTC),
             production_environment=self.settings.production_environment,
@@ -169,14 +172,15 @@ def for_issues(
             raise ConfigError(msg)
         state: typing.Final = self._read_issues(self.jira, keys, exclude)
         services: typing.Final[list[Service]] = []
+        projects: typing.Final[dict[str, Project]] = {}
         if state.error is None:
             links_by_project: dict[str, list[LinkedChange]] = {}
             for key in keys:
                 issue = state.issues.get(key)
                 for link in issue.links if issue else []:
                     links_by_project.setdefault(link.project, []).append(link)
             services.extend(
-                self._scoped_service(path, links_by_project[path], cache) for path in sorted(links_by_project)
+                self._scoped_service(path, links_by_project[path], cache, projects) for path in sorted(links_by_project)
             )
             row_state: typing.Final = self._read_issues(
                 self.jira, sorted(set(_row_keys(services)) - set(keys)), exclude
@@ -185,6 +189,7 @@ def for_issues(
             state.missing.extend(row_state.missing)
             state.error = row_state.error
             self._join_linked_issues(services, state)
+            self._record_link_states(services, state, projects, cache)
         return Report(
             collected_at=dt.datetime.now(dt.UTC),
             production_environment=self.settings.production_environment,
@@ -193,12 +198,15 @@ def for_issues(
             jira_scope=list(keys),
         )
 
-    def _scoped_service(self, path: str, links: list[LinkedChange], cache: Cache) -> Service:
+    def _scoped_service(
+        self, path: str, links: list[LinkedChange], cache: Cache, projects: dict[str, Project]
+    ) -> Service:
         try:
             project: typing.Final = self.api.get_project(path)
         except GitLabError as exc:
             project_url: typing.Final = links[0].project_url
             return Service(project=path, project_url=project_url, error=explain_failure(path, project_url, exc))
+        projects[path] = project
         return self._collect_or_explain(project, cache, links=links)
 
     def _collect_or_explain(self, project: Project, cache: Cache, *, links: list[LinkedChange] | None) -> Service:
@@ -233,6 +241,36 @@ def _join_linked_issues(self, services: list[Service], state: JiraState) -> None
             )
             service.candidates = build_candidates(service, production)
 
+    def _record_link_states(
+        self, services: list[Service], state: JiraState, projects: dict[str, Project], cache: Cache
+    ) -> None:
+        collected: typing.Final = {
+            service.project: service
+            for service in services
+            if service.error is None and service.default_branch and skip_reason(projects[service.project]) is None
+        }
+        links: typing.Final = [
+            (link, link.iid)
+            for issue in state.issues.values()
+            for link in issue.links
+            if link.iid is not None and link.project in collected
+        ]
+        known: typing.Final = {
+            (link.project, iid): (link.state, link.target_branch) for link, iid in links if link.state is not None
+        }
+        for link, iid in links:
+            service = collected[link.project]
+            if (link.project, iid) not in known:
+                if any(item.iid == iid for row in service.rows for item in row.merge_requests):
+                    known[link.project, iid] = ("merged", service.default_branch)
+                else:
+                    try:
+                        merge_request = self._linked_merge_request(projects[link.project], iid, cache)
+                    except GitLabError:
+                        continue
+                    known[link.project, iid] = (merge_request.state, merge_request.target_branch)
+            link.state, link.target_branch = known[link.project, iid]
+
     def _read_issues(
         self, jira: JiraApi, keys: collections.abc.Sequence[str], exclude: collections.abc.Sequence[str]
     ) -> JiraState:
@@ -408,6 +446,7 @@ def _link_target(
             if link.iid is None:
                 continue
             merge_request = self._linked_merge_request(project, link.iid, cache)
+            link.state, link.target_branch = merge_request.state, merge_request.target_branch
             if merge_request.state != "merged":
                 pending.append(_merge_request_ref(merge_request))
             elif merge_request.target_branch != default_branch:

skills/release-scope/SKILL.md

@@ -14,7 +14,7 @@ description: >
 through uvx with this version range; it matches the flags and report schema described here:
 
 ```bash
-uvx --from 'release-scope>=0.8,<0.9' release-scope --help
+uvx --from 'release-scope>=0.9,<0.10' release-scope --help
 ```
 
 ## Check the settings
@@ -58,7 +58,7 @@ run and only saves requests:
 ```bash
 out="${XDG_CACHE_HOME:-$HOME/.cache}/release-scope"
 mkdir -p "$out"
-uvx --from 'release-scope>=0.8,<0.9' release-scope collect --project team/backend/shop \
+uvx --from 'release-scope>=0.9,<0.10' release-scope collect --project team/backend/shop \
   --output "$out/site" --cache "$out/cache.json"
 ```
 
@@ -97,7 +97,9 @@ Read `$out/site/report.json` and answer the user's question from it, briefly.
   `compare_url`.
   To answer which issues a set of tags releases, merge the `jira_keys` of the picked candidates.
 - `jira.issues` by key: `summary`, `status`, `status_category` (anything but `done` is not done), `links` to GitLab
-  merge requests and commits. `jira.missing` lists keys Jira did not return; `jira.error` a failed request, as a message. `jira` is
+  merge requests and commits. A merge request link into a collected service has `state` (`opened`, `closed`, `locked`,
+  `merged`) and `target_branch`; `null` for commits, other projects, and unreadable merge requests. A merged link that
+  no row holds was merged before production, or below a cut range, or into another branch. `jira.missing` lists keys Jira did not return; `jira.error` a failed request, as a message. `jira` is
   `null` without a Jira token.
 - In a `--jira` report, `jira_scope` lists the issues, rows with `linked: true` are the ones the issues point to,
   rows with `in_scope: false` sit above the latest linked change and do not ship with the issues, and each service

tests/test_cli.py

@@ -49,7 +49,7 @@ def test_collect_writes_report_and_cache(gitlab: respx.Router, tmp_path: pathlib
         "report.json",
     ]
     report: typing.Final = json.loads((output / "report.json").read_text())
-    assert report["schema_version"] == 6
+    assert report["schema_version"] == 7
     assert report["jira"] is None
     assert report["production_environment"] == "production"
     assert [len(item["rows"]) for item in report["services"]] == [5]

tests/test_scope.py

@@ -225,6 +225,23 @@ def test_missing_scope_issue_yields_no_services(scoped: respx.Router) -> None:
     assert scoped["jira_search"].call_count == 1
 
 
+@pytest.mark.httpx2(assert_all_called=False)
+def test_scope_links_record_merge_request_states_without_reading_them_twice(scoped: respx.Router) -> None:
+    scoped["remote_links:SHOP-12"].respond(
+        json=_links(f"{ENDPOINT}/team/svc/-/merge_requests/12", f"{ENDPOINT}/team/svc/-/merge_requests/13")
+    )
+    scoped.get(f"{SERVICE_API}/merge_requests/13", name="mr:13").respond(json=merge_request(13, "WIP", state="opened"))
+
+    report: typing.Final = _scope()
+
+    assert report.jira is not None
+    assert [(item.iid, item.state, item.target_branch) for item in report.jira.issues["SHOP-12"].links] == [
+        (12, "merged", "main"),
+        (13, "opened", "main"),
+    ]
+    assert (scoped["mr:12"].call_count, scoped["mr:13"].call_count) == (1, 1)
+
+
 @pytest.mark.httpx2(assert_all_called=False)
 def test_scope_stops_when_jira_fails(scoped: respx.Router) -> None:
     scoped["jira_search"].respond(401)