Skip to content

Commit 2b9a50b

Browse files
feat: pull request mirroring for GitLab, Bitbucket, Azure DevOps, and CodeCommit (#46)
* refactor(vcs): split reopen from close, add BaseSHA to review comments Bitbucket Cloud has no reopen endpoint and CodeCommit forbids the CLOSED to OPEN transition, so reopening cannot be part of the writer contract every destination satisfies. PullRequestWriter now closes only, and reopening is an optional PullRequestReopener a destination may implement. A destination without it records the divergence and leaves the pull request closed rather than opening a second one for the same work. GitLab requires base, head, and start SHAs to anchor an inline comment, and CodeCommit requires a before and after commit id, so ReviewComment carries BaseSHA alongside CommitSHA. GitHub and Bitbucket ignore it. * feat(gitlab): mirror merge requests, notes, and approvals * feat(bitbucket): mirror pull requests, comments, and participant verdicts * feat(azuredevops): mirror pull requests, threads, and reviewer votes * feat(codecommit): mirror pull requests, comments, and approvals * feat(gitsync): pull request mirroring for every provider type docs: what each provider cannot represent * fix(prsync): stop updating a closed destination pull request A destination that cannot reopen, which is Bitbucket and CodeCommit, fell through to UpdatePullRequest on a pull request it had just decided to leave closed. Both providers reject an update to a closed pull request, so the call errored before the divergence was ever recorded: the warning fired every run, a failure was appended every run, and the mirror never settled. The same path ran for a merged source whose destination was already closed whenever new activity advanced its timestamp. An update now runs only while the destination is open, and an unfixable divergence records itself and returns. A recorded divergence is also re-checked each run rather than trusted, so a destination somebody reopened by hand is picked back up; previously the marker made statesAgree return true forever and the pull request could never be closed again. A source that reports no update timestamp, which Azure DevOps does not, is no longer watermarked. Skipping on a zero time mirrored such a pull request once and then froze it: no title, body, comment, or verdict change ever reached the destination again. * fix(vcs): correct provider defects found in review of #46 CodeCommit's pull request cache outlived the run that filled it. A provider is built once and reused for every cron tick, so a CodeCommit source served the first run's snapshot forever: later pull requests were never mirrored, closed ones never closed, and the repository was skipped as unchanged because the same stale set fed the fingerprint. The listing now re-enumerates and the cache only serves the finds that follow it. CodeCommit's FindPullRequest searched open pull requests only, so a destination pull request somebody had closed was invisible and a lost state file produced a second one for the same branch. A missed lookup now enumerates the closed ones too. CodeCommit treated every failure to post a located comment as a rejected anchor, so throttling or an expired credential permanently downgraded the comment to a discussion comment. Only genuine location errors downgrade. CodeCommit's ListReviews answered "no reviews" when the underlying call failed, and the caller deletes mirrored comments it no longer sees at the source, so one throttled call deleted every mirrored verdict. It returns the error now. Azure DevOps stood the creation date in for a missing update timestamp, which froze the engine's watermark and mirrored each pull request exactly once. It reports no timestamp instead, and the engine reconciles every run. Azure DevOps added labels and never removed them, re-POSTing every label on every run and failing the pull request when one already existed. Labels are reconciled against the source, and truncation counts characters rather than bytes so a non-ASCII description is neither cut to a third of the limit nor split through a rune. GitLab decided a comment was on the left side by testing an empty path, but GitLab populates new_path on both sides of a text diff, so every deleted line comment anchored to line 0 on the right. It tests the line instead. GitLab's transient locked state is no longer read as closed, which used to close the destination and post "closed without merging" moments before the source became merged. Bitbucket omitted draft on update, so a pull request that left draft at the source stayed a draft at the destination, and its lookup missed superseded pull requests. GitHub never reported a base SHA, so every inline comment mirrored to GitLab or CodeCommit, which both require one, silently downgraded to a discussion comment. Turning comments or reviews off deleted everything previously mirrored in that category, because nothing in it appeared in the seen set. Deletion is now scoped to categories the run actually listed. --------- Co-authored-by: Anmol Nagpal <ianmolnagpal@gmail.com>
1 parent 3825739 commit 2b9a50b

25 files changed

Lines changed: 4735 additions & 71 deletions

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- Pull request mirroring now works for every provider type, in any direction: GitLab, Bitbucket Cloud, Azure DevOps, and AWS CodeCommit join GitHub. Each maps onto the same model, and the runbook carries a table of what each provider cannot represent
12+
- CodeCommit is in scope after all. It was excluded when the work was planned on the grounds that it had no pull request API worth mirroring, which was wrong: it has a full one, already present in the pinned SDK, and AWS returned the service to general availability on 2025-11-24 after closing it to new customers in 2024
13+
14+
### Changed
15+
- `vcs.PullRequestSpec` carries `SyncLabels`, so a provider can tell "the source has no labels", where a destination label should be removed, from "label mirroring is off", where a label somebody added at the destination must be left alone. Labels are now reconciled rather than only added: one removed at the source is removed at the destination
16+
- `vcs.PullRequestWriter` closes a pull request but no longer reopens one. Bitbucket Cloud has no reopen endpoint and CodeCommit permits only the `OPEN` to `CLOSED` transition, so reopening is now an optional `vcs.PullRequestReopener` capability. A destination that lacks it leaves the pull request closed, warns once, and records the divergence rather than opening a second pull request for the same work or retrying on every run
17+
- `vcs.ReviewComment` carries `BaseSHA` alongside `CommitSHA`: GitLab refuses to anchor an inline comment without base, head, and start SHAs, and CodeCommit needs a before and after commit id. A source that cannot supply them downgrades the comment to a discussion comment rather than dropping it
18+
1019
## [0.2.0] - 2026-08-19
1120

1221
### Added

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -380,7 +380,7 @@ export SYNCERD_GIT_GL_TOKEN=your-gitlab-token
380380

381381
**Pull request heads (opt in):** set `pull_requests.enabled: true` on a mirror and the commits behind each open pull request opened from a **fork** are pushed to the destination as a branch, default `syncerd/pr/<number>`. Without it those commits reach the destination nowhere, since a fork pull request has no branch in the source repository. A pull request opened from a source branch already mirrors under its own name and gets no second copy. The pull request itself is not recreated at the destination yet. Off by default because a fork head is third-party code, and a destination that builds on branch push would run it: read the security note in [docs/git-sync-runbook.md](docs/git-sync-runbook.md#mirroring-pull-request-heads) before enabling it.
382382

383-
**Mirrored pull requests (opt in):** add `pull_requests.mirror_objects: true` and each open source pull request is recreated at the destination as a real pull request, kept in step on every run, with its discussion, inline review comments, and review verdicts. The source is the single authority; nothing written at the destination flows back. A merged source pull request is closed at the destination with a note naming the merge commit, never merged there, because a destination merge would diverge from the mirrored base branch. Review verdicts are mirrored as attributed text rather than real approvals. Mentions and issue references in mirrored text are neutralised so a mirror never notifies or cross links anyone. GitHub on both ends for now.
383+
**Mirrored pull requests (opt in):** add `pull_requests.mirror_objects: true` and each open source pull request is recreated at the destination as a real pull request, kept in step on every run, with its discussion, inline review comments, and review verdicts. The source is the single authority; nothing written at the destination flows back. A merged source pull request is closed at the destination with a note naming the merge commit, never merged there, because a destination merge would diverge from the mirrored base branch. Review verdicts are mirrored as attributed text rather than real approvals. Mentions and issue references in mirrored text are neutralised so a mirror never notifies or cross links anyone. All five provider types work, in any direction; the runbook has a table of what each one cannot represent (Bitbucket and CodeCommit cannot reopen, neither has labels, CodeCommit has no drafts).
384384

385385
Run with `--dry-run` to print the ref changes each mirror would make, per repository, without creating, pushing, or deleting anything.
386386

docs/git-sync-runbook.md

Lines changed: 49 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -671,8 +671,55 @@ person at the destination is left alone, even though the source is the
671671
authority: the source is authority over what it published, not over what
672672
somebody else said.
673673

674-
Today this needs GitHub on both ends. Any other destination is rejected at
675-
startup.
674+
### What survives the crossing
675+
676+
Every provider type can mirror pull requests, in any direction. They do not
677+
agree on what a pull request has, so some things are approximated and some
678+
are lost. This table is the whole of it:
679+
680+
| | GitHub | GitLab | Bitbucket | Azure DevOps | CodeCommit |
681+
|---|---|---|---|---|---|
682+
| Reopen a closed mirror | yes | yes | **no** | yes | **no** |
683+
| Draft | yes | via a `Draft:` title prefix | yes | on create only | **no** |
684+
| Update watermark | yes | yes | yes | **none reported, so every run reconciles** | yes |
685+
| Labels | yes | yes | **no** | yes | **no** |
686+
| Inline comment anchors | yes | needs the source to report commit SHAs | yes | yes | yes |
687+
| Comment deletion | removed | removed | removed | **tombstone** | **content blanked** |
688+
689+
Nothing here fails a run. A capability the destination lacks is logged and
690+
skipped, never faked:
691+
692+
- **Reopen.** Bitbucket has no reopen endpoint and CodeCommit permits only
693+
`OPEN` to `CLOSED`. A destination pull request closed by hand while its
694+
source is still open stays closed, and SyncerD warns once and records the
695+
divergence rather than opening a second pull request for the same work.
696+
Nothing else is attempted on it, because both providers reject an update
697+
to a closed pull request. Reopen it by hand and the next run picks it up
698+
again: the recorded divergence is re-checked, never trusted.
699+
- **Draft.** GitLab has no draft field, so a draft is expressed by prefixing
700+
the title with `Draft:`, and the prefix is stripped when the source leaves
701+
draft. This is the one place SyncerD does not copy a title verbatim.
702+
CodeCommit ignores the flag.
703+
- **Labels.** Ignored on Bitbucket and CodeCommit, which have none.
704+
- **Inline comments.** GitLab and CodeCommit refuse to anchor a comment
705+
without commit SHAs. When the source did not report them, the comment is
706+
posted as a discussion comment naming the file and line instead of being
707+
dropped.
708+
- **Deletion.** Azure DevOps and CodeCommit delete softly, so a comment
709+
deleted at the source leaves a tombstone or an empty comment rather than
710+
disappearing.
711+
712+
CodeCommit is also the expensive one: its API returns pull request ids only,
713+
so each listing costs one extra call per pull request, and it cannot search
714+
by branch at all, so SyncerD enumerates once per repository per run and
715+
matches locally. A lookup that misses among the open pull requests
716+
enumerates the closed ones too, which is what stops a lost state file from
717+
producing a duplicate.
718+
719+
Azure DevOps reports no update timestamp on a pull request, so there is
720+
nothing to watermark against and every pull request is reconciled on every
721+
run. That is slower than the other four and it is the only way to notice an
722+
edit there at all.
676723

677724
**Read this before enabling it.** A fork pull request head is code written by
678725
anyone who can open a pull request against the source. Turning this on pushes

go.mod

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@ module github.com/clouddrove/syncerd
33
go 1.25.0
44

55
require (
6+
github.com/aws/aws-sdk-go-v2 v1.43.6
67
github.com/aws/aws-sdk-go-v2/config v1.32.36
78
github.com/aws/aws-sdk-go-v2/service/codecommit v1.37.0
89
github.com/aws/aws-sdk-go-v2/service/ecr v1.60.6
10+
github.com/aws/smithy-go v1.27.8
911
github.com/google/go-containerregistry v0.19.0
1012
github.com/robfig/cron/v3 v3.0.1
1113
github.com/spf13/cobra v1.8.0
@@ -14,7 +16,6 @@ require (
1416
)
1517

1618
require (
17-
github.com/aws/aws-sdk-go-v2 v1.43.6 // indirect
1819
github.com/aws/aws-sdk-go-v2/credentials v1.19.35 // indirect
1920
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.36 // indirect
2021
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.37 // indirect
@@ -26,7 +27,6 @@ require (
2627
github.com/aws/aws-sdk-go-v2/service/sso v1.33.5 // indirect
2728
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.5 // indirect
2829
github.com/aws/aws-sdk-go-v2/service/sts v1.45.5 // indirect
29-
github.com/aws/smithy-go v1.27.8 // indirect
3030
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
3131
github.com/docker/cli v29.6.1+incompatible // indirect
3232
github.com/docker/distribution v2.8.2+incompatible // indirect

internal/gitsync/build.go

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ func BuildMirrors(cfg *config.GitConfig) ([]Mirror, *Redactor, error) {
144144
if mc.PullRequests.Enabled {
145145
sourcePRs, ok = src.(vcs.PullRequestLister)
146146
if !ok {
147-
return nil, nil, fmt.Errorf("mirror %q: pull_requests is enabled but source provider %q (type %q) cannot list pull requests; GitHub is supported today and the remaining provider types arrive in a later release", mc.Name, mc.Source, src.Type())
147+
return nil, nil, fmt.Errorf("mirror %q: pull_requests is enabled but source provider %q (type %q) cannot list pull requests", mc.Name, mc.Source, src.Type())
148148
}
149149
}
150150

@@ -157,16 +157,16 @@ func BuildMirrors(cfg *config.GitConfig) ([]Mirror, *Redactor, error) {
157157
if mc.PullRequests.MirrorObjects {
158158
destPRs, ok = dst.(vcs.PullRequestWriter)
159159
if !ok {
160-
return nil, nil, fmt.Errorf("mirror %q: pull_requests.mirror_objects is set but destination provider %q (type %q) cannot write pull requests; GitHub is supported today and the remaining provider types arrive in a later release", mc.Name, mc.Destination, dst.Type())
160+
return nil, nil, fmt.Errorf("mirror %q: pull_requests.mirror_objects is set but destination provider %q (type %q) cannot write pull requests", mc.Name, mc.Destination, dst.Type())
161161
}
162162
if mc.PullRequests.CommentsOrDefault() || mc.PullRequests.ReviewsOrDefault() {
163163
sourceConv, ok = src.(vcs.PullRequestConversation)
164164
if !ok {
165-
return nil, nil, fmt.Errorf("mirror %q: pull request comments are enabled but source provider %q (type %q) cannot read them; set comments: false and reviews: false, or wait for that provider", mc.Name, mc.Source, src.Type())
165+
return nil, nil, fmt.Errorf("mirror %q: pull request comments are enabled but source provider %q (type %q) cannot read them; set comments: false and reviews: false", mc.Name, mc.Source, src.Type())
166166
}
167167
destConv, ok = dst.(vcs.PullRequestConversation)
168168
if !ok {
169-
return nil, nil, fmt.Errorf("mirror %q: pull request comments are enabled but destination provider %q (type %q) cannot write them; set comments: false and reviews: false, or wait for that provider", mc.Name, mc.Destination, dst.Type())
169+
return nil, nil, fmt.Errorf("mirror %q: pull request comments are enabled but destination provider %q (type %q) cannot write them; set comments: false and reviews: false", mc.Name, mc.Destination, dst.Type())
170170
}
171171
}
172172
}

internal/gitsync/build_test.go

Lines changed: 76 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import (
66
"testing"
77

88
"github.com/clouddrove/syncerd/internal/config"
9+
"github.com/clouddrove/syncerd/internal/vcs"
910
)
1011

1112
func buildableConfig() *config.GitConfig {
@@ -289,15 +290,82 @@ func TestBuildMirrorsWiresPullRequests(t *testing.T) {
289290
}
290291
}
291292

292-
func TestBuildMirrorsRejectsAPullRequestSourceThatCannotList(t *testing.T) {
293-
cfg := buildableConfig()
294-
cfg.Mirrors[0].Source = "gl"
295-
cfg.Mirrors[0].Destination = "gh"
296-
cfg.Mirrors[0].PullRequests = config.PullRequestsConfig{Enabled: true}
293+
// prCapableProviders is every provider type that can mirror pull requests.
294+
// All five can, since P3; the table exists so a regression in any one of
295+
// them fails here rather than at run time in somebody's mirror.
296+
var prCapableProviders = []config.GitProviderConfig{
297+
{Name: "p-github", Type: "github", Owner: "acme", Token: "ghp_token_value"},
298+
{Name: "p-gitlab", Type: "gitlab", Owner: "acme", Token: "glpat_token_value"},
299+
{Name: "p-bitbucket", Type: "bitbucket", Owner: "acme", Email: "svc@acme.com", Token: "bb_token_value"},
300+
{Name: "p-azuredevops", Type: "azuredevops", Owner: "acme", Project: "proj", Token: "ado_token_value"},
301+
{Name: "p-codecommit", Type: "codecommit", Region: "us-east-1", GitUsername: "u", GitPassword: "p"},
302+
}
303+
304+
func TestEveryProviderTypeCanMirrorPullRequests(t *testing.T) {
305+
for _, src := range prCapableProviders {
306+
for _, dst := range prCapableProviders {
307+
if src.Name == dst.Name {
308+
continue
309+
}
310+
t.Run(src.Type+"-to-"+dst.Type, func(t *testing.T) {
311+
cfg := &config.GitConfig{
312+
Providers: prCapableProviders,
313+
Mirrors: []config.MirrorConfig{{
314+
Name: "m", Source: src.Name, Destination: dst.Name,
315+
PullRequests: config.PullRequestsConfig{Enabled: true, MirrorObjects: true},
316+
}},
317+
}
318+
cfg.ApplyDefaults()
319+
320+
mirrors, _, err := BuildMirrors(cfg)
321+
if err != nil {
322+
t.Fatalf("build: %v", err)
323+
}
324+
m := mirrors[0]
325+
if m.SourcePRs == nil {
326+
t.Error("source lister not wired")
327+
}
328+
if m.DestPRs == nil {
329+
t.Error("destination writer not wired")
330+
}
331+
if m.SourceConv == nil || m.DestConv == nil {
332+
t.Error("conversation endpoints not wired")
333+
}
334+
})
335+
}
336+
}
337+
}
297338

298-
_, _, err := BuildMirrors(cfg)
299-
if err == nil || !strings.Contains(err.Error(), "pull_requests") {
300-
t.Fatalf("expected an error naming pull_requests, got %v", err)
339+
func TestOnlySomeDestinationsCanReopen(t *testing.T) {
340+
// Bitbucket has no reopen endpoint and CodeCommit forbids the
341+
// transition, so the engine must see the capability absent on exactly
342+
// those two and present on the rest.
343+
want := map[string]bool{
344+
"github": true, "gitlab": true, "azuredevops": true,
345+
"bitbucket": false, "codecommit": false,
346+
}
347+
348+
for _, dst := range prCapableProviders {
349+
cfg := &config.GitConfig{
350+
Providers: prCapableProviders,
351+
Mirrors: []config.MirrorConfig{{
352+
Name: "m", Source: "p-github", Destination: dst.Name,
353+
PullRequests: config.PullRequestsConfig{Enabled: true, MirrorObjects: true},
354+
}},
355+
}
356+
if dst.Name == "p-github" {
357+
cfg.Mirrors[0].Source = "p-gitlab"
358+
}
359+
cfg.ApplyDefaults()
360+
361+
mirrors, _, err := BuildMirrors(cfg)
362+
if err != nil {
363+
t.Fatalf("%s: build: %v", dst.Type, err)
364+
}
365+
_, canReopen := mirrors[0].DestPRs.(vcs.PullRequestReopener)
366+
if canReopen != want[dst.Type] {
367+
t.Errorf("%s reopen capability = %v, want %v", dst.Type, canReopen, want[dst.Type])
368+
}
301369
}
302370
}
303371

@@ -316,7 +384,6 @@ func TestBuildMirrorsLeavesPullRequestsOffByDefault(t *testing.T) {
316384

317385
func TestBuildMirrorsWiresPullRequestObjects(t *testing.T) {
318386
cfg := buildableConfig()
319-
// GitHub on both ends: the only pair that can write pull requests today.
320387
cfg.Mirrors[0].Destination = "gh2"
321388
cfg.Providers = append(cfg.Providers, config.GitProviderConfig{
322389
Name: "gh2", Type: "github", Owner: "mirror", Token: "ghp_token_value_two",
@@ -341,13 +408,3 @@ func TestBuildMirrorsWiresPullRequestObjects(t *testing.T) {
341408
t.Errorf("switches = %+v", m.PullRequests)
342409
}
343410
}
344-
345-
func TestBuildMirrorsRejectsADestinationThatCannotWritePullRequests(t *testing.T) {
346-
cfg := buildableConfig()
347-
cfg.Mirrors[0].PullRequests = config.PullRequestsConfig{Enabled: true, MirrorObjects: true}
348-
349-
_, _, err := BuildMirrors(cfg)
350-
if err == nil || !strings.Contains(err.Error(), "mirror_objects") {
351-
t.Fatalf("a GitLab destination cannot write pull requests yet, want that error, got %v", err)
352-
}
353-
}

internal/prsync/comments.go

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"context"
55
"errors"
66
"fmt"
7+
"strings"
78

89
"github.com/clouddrove/syncerd/internal/logging"
910
"github.com/clouddrove/syncerd/internal/state"
@@ -79,11 +80,23 @@ func syncConversation(ctx context.Context, pr vcs.PullRequest, destNumber int, o
7980
}
8081

8182
// A source comment that has since been deleted takes its mirrored copy
82-
// with it, but only the copy SyncerD wrote.
83+
// with it, but only the copy SyncerD wrote, and only in a category this
84+
// run actually listed.
85+
//
86+
// That second condition matters: without it, turning comments off would
87+
// delete every comment previously mirrored, and turning reviews off
88+
// would delete every mirrored verdict, because nothing in those
89+
// categories would appear in seen.
8390
for sourceID, destID := range rec.CommentIDs {
8491
if seen[sourceID] {
8592
continue
8693
}
94+
if isReviewKey(sourceID) && !opts.Reviews {
95+
continue
96+
}
97+
if !isReviewKey(sourceID) && !opts.Comments {
98+
continue
99+
}
87100
if opts.DryRun {
88101
logging.Info(fmt.Sprintf("mirror %s: would delete mirrored comment %s on destination pull request %d", opts.Mirror, destID, destNumber))
89102
continue
@@ -98,6 +111,12 @@ func syncConversation(ctx context.Context, pr vcs.PullRequest, destNumber int, o
98111
return created, downgraded, nil
99112
}
100113

114+
// isReviewKey reports whether a tracked id belongs to a mirrored review
115+
// verdict rather than a comment. ComposeReview prefixes those keys.
116+
func isReviewKey(sourceID string) bool {
117+
return strings.HasPrefix(sourceID, "review-")
118+
}
119+
101120
// upsert posts a comment the first time and rewrites it after that.
102121
func upsert(ctx context.Context, opts Options, destNumber int, rec *state.PRRecord, sourceID, body string) (int, error) {
103122
if destID, ok := rec.CommentIDs[sourceID]; ok {

0 commit comments

Comments
 (0)