Skip to content

Commit f098e0f

Browse files
weidongxu-microsoftweidongxu-microsoft
authored
Add Java dependency version sync script and sync dependencies (#12286)
## Summary - Add a PowerShell script to sync Java dependency versions from Azure SDK for Java, with an allowlist, version pins, and `-WhatIf`. - Apply the sync to generator POMs and `Project.Dependency`. - Update the authentication-test token-cache import for Azure Core 1.60.0. - Fix XML pagination reading stream-backed responses twice by parsing cached bytes, matching JSON behavior. ## Validation - Package build, formatting, lint, and typecheck. - Offline script tests, including V2 enum names. - Both integration-test modules compile. - Rerunning the sync produces no changes. - Original pageable Spector tests and sync/async XML pagination regression tests pass. --------- Copilot-Session: 6dc9e98a-42fc-4fcf-b49c-b0823d177763
1 parent 31457ca commit f098e0f

21 files changed

Lines changed: 417 additions & 45 deletions

File tree

‎.chronus/changes/chore-java-sync-dependency-versions-2026-9-16-10-20-30.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: dependencies
3+
packages:
4+
- "@typespec/http-client-java"
5+
---
6+
7+
Sync Java runtime, test, and Maven plugin dependency versions with Azure SDK for Java.

‎.chronus/changes/chore-java-sync-dependency-versions-2026-9-16-11-27-54.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: fix
3+
packages:
4+
- "@typespec/http-client-java"
5+
---
6+
7+
Fix XML pagination with stream-backed responses by caching response bytes before reading items and the next link.

‎.chronus/changes/http-client-java-sync-dependency-versions.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: internal
3+
packages:
4+
- "@typespec/http-client-java"
5+
---
6+
7+
Add a PowerShell script to sync Java dependency versions with Azure SDK for Java.

‎.github/instructions/http-client-java.instructions.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,10 @@ Steps:
1616
3. Update package versions in `overrides` (keep the semver ranges) in the other two `package.json` files to match the corresponding package versions in the root `package.json`.
1717
4. Save the files and run `npm install` in the root so that `package-lock.json` is updated.
1818
5. Commit the changes to `package.json` and `package-lock.json`.
19-
6. If there is an update to the `http-specs` or `azure-http-specs` libraries, run `Generate.ps1` in `generator/http-client-generator-test` and commit the generated changes in that folder.
20-
7. If there is an update to the `http-specs` library, run `Generate.ps1` in `generator/http-client-generator-clientcore-test` and commit the generated changes in that folder.
21-
8. Call `pnpm change add @typespec/http-client-java --kind=dependencies --message="<change-summary>"`. Commit the new md file in ".chronus" folder of repository root.
19+
6. Run `pwsh eng\versioning\Sync-Versions.ps1` from the package root to also update Java/JAR dependency versions. Commit the Java and POM changes.
20+
7. If there is an update to the `http-specs` or `azure-http-specs` libraries, run `Generate.ps1` in `generator/http-client-generator-test` and commit the generated changes in that folder.
21+
8. If there is an update to the `http-specs` library, run `Generate.ps1` in `generator/http-client-generator-clientcore-test` and commit the generated changes in that folder.
22+
9. Call `pnpm change add @typespec/http-client-java --kind=dependencies --message="<change-summary>"`. Commit the new md file in ".chronus" folder of repository root.
2223

2324
# Prepare for minor/patch release
2425

‎packages/http-client-java/CONTRIBUTING.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,18 @@ When writing / updating Java tests, you can manually
7979
1. Start the test server with `npm run spector-serve` (or `npm run spector-start` to start as a background process, `npm run spector-stop` to stop).
8080
2. Run the Java tests in the module.
8181

82+
## Sync Java Dependency Versions
83+
84+
From this package directory, run `pwsh eng\versioning\Sync-Versions.ps1` to sync
85+
`Project.Dependency`, generator/module POMs, the customization POM, and the generated-POM
86+
template with Azure SDK for Java's version files. Use `-WhatIf` to preview changes.
87+
88+
`eng\versioning\external_dependencies.txt` lists the packages to sync; append `;version`
89+
to pin a package. The script prefers the first (released) version in `version_client.txt`,
90+
then falls back to `external_dependencies.txt`. Unlisted dependencies are unchanged.
91+
92+
Run the script's offline tests with `pwsh eng\versioning\Sync-Versions.Tests.ps1`.
93+
8294
## Format Code
8395

8496
Before creating pull request, the code is required to be well-formatted.

‎packages/http-client-java/eng/versioning/Sync-Versions.Tests.ps1‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
4+
$ErrorActionPreference = 'Stop'
5+
. (Join-Path $PSScriptRoot 'Sync-Versions.ps1')
6+
7+
function Assert-Equal($Expected, $Actual)
8+
{
9+
if ($Expected -cne $Actual)
10+
{
11+
throw "Expected '$Expected', got '$Actual'."
12+
}
13+
}
14+
15+
$clientVersions = Read-Versions @'
16+
# Released version precedes the development version.
17+
com.azure:azure-core;1.2.3;1.3.0-beta.1
18+
com.azure.v2:azure-client-sdk-parent;2.0.0-beta.2;2.0.0-beta.3
19+
io.clientcore:core;1.0.0-beta.11;1.0.0-beta.12
20+
'@
21+
$externalVersions = Read-Versions @'
22+
com.azure:azure-core;0.9.0
23+
testdep_net.bytebuddy:byte-buddy;1.17.7
24+
org.mockito:mockito-core;5.0.0
25+
'@
26+
$packages = Read-Versions @'
27+
com.azure:azure-core
28+
com.azure.v2:azure-client-sdk-parent
29+
io.clientcore:core
30+
net.bytebuddy:byte-buddy
31+
org.mockito:mockito-core;4.11.0
32+
'@
33+
$versions = Resolve-Versions $packages @($clientVersions, $externalVersions)
34+
Assert-Equal '1.2.3' $versions['com.azure:azure-core']
35+
Assert-Equal '1.0.0-beta.11' $versions['io.clientcore:core']
36+
Assert-Equal '1.17.7' $versions['net.bytebuddy:byte-buddy']
37+
Assert-Equal '4.11.0' $versions['org.mockito:mockito-core']
38+
39+
$failed = $false
40+
try
41+
{
42+
Resolve-Versions (Read-Versions 'missing:dependency') @($clientVersions, $externalVersions)
43+
}
44+
catch
45+
{
46+
if ($_.Exception.Message -notlike '*missing:dependency*') { throw }
47+
$failed = $true
48+
}
49+
Assert-Equal $true $failed
50+
51+
$root = Join-Path ([System.IO.Path]::GetTempPath()) "java-version-sync-$([guid]::NewGuid())"
52+
try
53+
{
54+
$generator = Join-Path $root 'generator'
55+
$module = Join-Path $generator 'http-client-generator-core'
56+
$resources = Join-Path $module 'src\main\resources'
57+
$javaDirectory = Join-Path $module 'src\main\java\com\microsoft\typespec\http\client\generator\core\model\projectmodel'
58+
$target = Join-Path $module 'target'
59+
$customization = Join-Path $generator 'http-client-generator-test\customization'
60+
foreach ($directory in @($resources, $javaDirectory, $target, $customization))
61+
{
62+
$null = New-Item -ItemType Directory -Path $directory -Force
63+
}
64+
$pom = @'
65+
<project>
66+
<!-- <groupId>com.azure</groupId><artifactId>azure-core</artifactId><version>comment</version> -->
67+
<parent>
68+
<groupId>com.microsoft.typespec</groupId>
69+
<artifactId>typespec-java-generator</artifactId>
70+
<version>1.0.0-beta.1</version>
71+
</parent>
72+
<dependencies>
73+
<dependency>
74+
<groupId>com.azure</groupId>
75+
<!-- Keep this comment and the original formatting. -->
76+
<artifactId>azure-core</artifactId>
77+
<version>old</version>
78+
<exclusions>
79+
<exclusion><groupId>io.clientcore</groupId><artifactId>core</artifactId></exclusion>
80+
</exclusions>
81+
</dependency>
82+
<dependency>
83+
<groupId>unlisted</groupId><artifactId>library</artifactId><version>old</version>
84+
</dependency>
85+
<dependency>
86+
<groupId>org.mockito</groupId><artifactId>mockito-core</artifactId><version>old</version>
87+
</dependency>
88+
</dependencies>
89+
</project>
90+
'@
91+
$pom = $pom.Replace("`r`n", "`n").Replace("`n", "`r`n")
92+
$pomFiles = @(
93+
(Join-Path $generator 'pom.xml'),
94+
(Join-Path $module 'pom.xml'),
95+
(Join-Path $resources 'pom.xml'),
96+
(Join-Path $customization 'pom.xml')
97+
)
98+
foreach ($file in $pomFiles + (Join-Path $target 'pom.xml'))
99+
{
100+
[System.IO.File]::WriteAllText($file, $pom)
101+
}
102+
$javaFile = Join-Path $javaDirectory 'Project.java'
103+
$java = " AZURE_CORE(`"com.azure`", `"azure-core`", `"old`"),`n" +
104+
" AZURE_CLIENT_SDK_PARENT_V2(`"com.azure.v2`", `"azure-client-sdk-parent`", `"old-v2`"),`n" +
105+
" CLIENTCORE(`"io.clientcore`", `"core`", `"old`");`n"
106+
[System.IO.File]::WriteAllText($javaFile, $java)
107+
108+
Update-VersionFiles -Root $root -Versions $versions -WhatIf
109+
Assert-Equal $pom ([System.IO.File]::ReadAllText($pomFiles[0]))
110+
Assert-Equal $java ([System.IO.File]::ReadAllText($javaFile))
111+
112+
Update-VersionFiles -Root $root -Versions $versions
113+
$expected = $pom.Replace(" <version>old</version>", " <version>1.2.3</version>")
114+
$expected = $expected.Replace('<artifactId>mockito-core</artifactId><version>old</version>',
115+
'<artifactId>mockito-core</artifactId><version>4.11.0</version>')
116+
foreach ($file in $pomFiles)
117+
{
118+
Assert-Equal $expected ([System.IO.File]::ReadAllText($file))
119+
}
120+
Assert-Equal $pom ([System.IO.File]::ReadAllText((Join-Path $target 'pom.xml')))
121+
Assert-Equal ($java.Replace('"old"),', '"1.2.3"),').Replace('"old");', '"1.0.0-beta.11");').Replace('"old-v2"', '"2.0.0-beta.2"')) `
122+
([System.IO.File]::ReadAllText($javaFile))
123+
124+
$modified = (Get-Item $javaFile).LastWriteTimeUtc
125+
Update-VersionFiles -Root $root -Versions $versions
126+
Assert-Equal $modified (Get-Item $javaFile).LastWriteTimeUtc
127+
Write-Host 'Version sync tests passed.'
128+
}
129+
finally
130+
{
131+
if (Test-Path -LiteralPath $root)
132+
{
133+
Remove-Item -LiteralPath $root -Recurse -Force
134+
}
135+
}

‎packages/http-client-java/eng/versioning/Sync-Versions.ps1‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Copyright (c) Microsoft Corporation. All rights reserved.
2+
# Licensed under the MIT License.
3+
4+
[CmdletBinding(SupportsShouldProcess)]
5+
param()
6+
7+
$ErrorActionPreference = 'Stop'
8+
9+
function Read-Versions([string] $Content)
10+
{
11+
$versions = @{}
12+
foreach ($line in $Content -split '\r?\n')
13+
{
14+
$line = $line.Trim()
15+
if (!$line -or $line.StartsWith('#')) { continue }
16+
$fields = $line -split ';'
17+
$package = $fields[0] -replace '^testdep_', ''
18+
if ($package -notmatch '^[\w.-]+:[\w.-]+$')
19+
{
20+
throw "Invalid package coordinate: $line"
21+
}
22+
$versions[$package] = if ($fields.Length -gt 1) { $fields[1].Trim() } else { '' }
23+
}
24+
return $versions
25+
}
26+
27+
function Resolve-Versions([hashtable] $Packages, [hashtable[]] $Sources)
28+
{
29+
$versions = @{}
30+
foreach ($package in $Packages.Keys)
31+
{
32+
$version = $Packages[$package]
33+
if (!$version)
34+
{
35+
foreach ($source in $Sources)
36+
{
37+
if ($source[$package])
38+
{
39+
$version = $source[$package]
40+
break
41+
}
42+
}
43+
}
44+
if (!$version) { throw "Version not found for package $package" }
45+
$versions[$package] = $version
46+
}
47+
return $versions
48+
}
49+
50+
function Update-VersionFiles
51+
{
52+
[CmdletBinding(SupportsShouldProcess)]
53+
param([string] $Root, [hashtable] $Versions)
54+
55+
$generator = Join-Path $Root 'generator'
56+
$core = Join-Path $generator 'http-client-generator-core'
57+
$pomFiles = @((Get-Item (Join-Path $generator 'pom.xml')).FullName)
58+
$pomFiles += Get-ChildItem -Path $generator -Directory | ForEach-Object {
59+
$pom = Join-Path $_.FullName 'pom.xml'
60+
if (Test-Path -LiteralPath $pom) { $pom }
61+
}
62+
$pomFiles += Join-Path $core 'src\main\resources\pom.xml'
63+
$pomFiles += Join-Path $generator 'http-client-generator-test\customization\pom.xml'
64+
$javaFile = Join-Path $core 'src\main\java\com\microsoft\typespec\http\client\generator\core\model\projectmodel\Project.java'
65+
$updates = @{}
66+
foreach ($file in $pomFiles + $javaFile)
67+
{
68+
$content = [System.IO.File]::ReadAllText($file)
69+
if ($file -eq $javaFile)
70+
{
71+
$pattern = '(?m)(?<prefix>^\s*[A-Z][A-Z_0-9]*\("(?<group>[^"]+)",\s*"(?<artifact>[^"]+)",\s*")(?<version>[^"]+)(?<suffix>"\)[,;])'
72+
}
73+
else
74+
{
75+
# Match comments separately so commented-out dependencies are never changed.
76+
$gap = '(?:\s|<!--.*?-->)*'
77+
$pattern = "<!--.*?-->|(?<prefix><groupId>(?<group>[^<]+)</groupId>$gap<artifactId>(?<artifact>[^<]+)</artifactId>$gap<version>)(?<version>[^<]+)(?<suffix></version>)"
78+
$null = [xml] $content
79+
}
80+
$updated = [regex]::Replace($content, $pattern, [System.Text.RegularExpressions.MatchEvaluator] {
81+
param($match)
82+
if (!$match.Groups['prefix'].Success) { return $match.Value }
83+
$package = "$($match.Groups['group'].Value.Trim()):$($match.Groups['artifact'].Value.Trim())"
84+
if (!$Versions.ContainsKey($package)) { return $match.Value }
85+
return $match.Groups['prefix'].Value + $Versions[$package] + $match.Groups['suffix'].Value
86+
}, [System.Text.RegularExpressions.RegexOptions]::Singleline)
87+
if ($updated -cne $content) { $updates[$file] = $updated }
88+
}
89+
90+
# Read and validate all inputs before writing any changes.
91+
foreach ($file in $updates.Keys | Sort-Object)
92+
{
93+
if ($PSCmdlet.ShouldProcess($file, 'Sync dependency versions'))
94+
{
95+
[System.IO.File]::WriteAllText($file, $updates[$file])
96+
Write-Host "Updated $file"
97+
}
98+
}
99+
}
100+
101+
if ($MyInvocation.InvocationName -ne '.')
102+
{
103+
$root = [System.IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..\..'))
104+
$baseUri = 'https://raw.githubusercontent.com/Azure/azure-sdk-for-java/main/eng/versioning'
105+
$clientVersions = Read-Versions (Invoke-WebRequest "$baseUri/version_client.txt").Content
106+
$externalVersions = Read-Versions (Invoke-WebRequest "$baseUri/external_dependencies.txt").Content
107+
$packages = Read-Versions ([System.IO.File]::ReadAllText((Join-Path $PSScriptRoot 'external_dependencies.txt')))
108+
$versions = Resolve-Versions $packages @($clientVersions, $externalVersions)
109+
Update-VersionFiles -Root $root -Versions $versions -WhatIf:$WhatIfPreference
110+
}

‎packages/http-client-java/eng/versioning/external_dependencies.txt‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Packages to sync from azure-sdk-for-java. Add ;version to pin a package.
2+
com.azure:azure-client-sdk-parent
3+
com.azure.v2:azure-client-sdk-parent
4+
com.azure.v2:azure-core
5+
com.azure:azure-core
6+
com.azure:azure-core-experimental
7+
com.azure:azure-core-http-netty
8+
com.azure:azure-core-management
9+
com.azure:azure-core-test
10+
com.azure:azure-identity
11+
com.azure:azure-json
12+
com.azure:azure-xml
13+
14+
io.clientcore:core
15+
io.clientcore:annotation-processor
16+
17+
com.fasterxml.jackson.core:jackson-databind
18+
com.fasterxml.jackson.dataformat:jackson-dataformat-xml
19+
com.fasterxml.jackson.datatype:jackson-datatype-jsr310
20+
org.slf4j:slf4j-api
21+
org.slf4j:slf4j-simple
22+
org.junit.jupiter:junit-jupiter-api
23+
org.junit.jupiter:junit-jupiter-engine
24+
org.junit.jupiter:junit-jupiter-params
25+
io.projectreactor:reactor-test
26+
27+
# Mockito 4.11.0 is the last version supporting Java 8.
28+
org.mockito:mockito-core;4.11.0
29+
net.bytebuddy:byte-buddy
30+
net.bytebuddy:byte-buddy-agent
31+
32+
org.apache.maven.plugins:maven-clean-plugin
33+
org.apache.maven.plugins:maven-compiler-plugin
34+
org.apache.maven.plugins:maven-install-plugin
35+
org.apache.maven.plugins:maven-jar-plugin
36+
org.apache.maven.plugins:maven-javadoc-plugin
37+
org.apache.maven.plugins:maven-resources-plugin
38+
org.apache.maven.plugins:maven-shade-plugin
39+
org.apache.maven.plugins:maven-surefire-plugin

‎packages/http-client-java/generator/http-client-generator-clientcore-test/pom.xml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
<dependency>
3535
<groupId>io.projectreactor</groupId>
3636
<artifactId>reactor-test</artifactId>
37-
<version>3.7.18</version>
37+
<version>3.8.7</version>
3838
</dependency>
3939
<dependency>
4040
<groupId>org.mockito</groupId>
@@ -47,13 +47,13 @@
4747
<dependency>
4848
<groupId>net.bytebuddy</groupId>
4949
<artifactId>byte-buddy</artifactId>
50-
<version>1.17.7</version>
50+
<version>1.18.11</version>
5151
<scope>test</scope>
5252
</dependency>
5353
<dependency>
5454
<groupId>net.bytebuddy</groupId>
5555
<artifactId>byte-buddy-agent</artifactId>
56-
<version>1.17.7</version>
56+
<version>1.18.11</version>
5757
<scope>test</scope>
5858
</dependency>
5959
<dependency>
@@ -80,7 +80,7 @@
8080
<plugin>
8181
<groupId>org.apache.maven.plugins</groupId>
8282
<artifactId>maven-compiler-plugin</artifactId>
83-
<version>3.14.0</version>
83+
<version>3.15.0</version>
8484
<executions>
8585
<execution>
8686
<id>run-annotation-processing</id>

‎packages/http-client-java/generator/http-client-generator-core/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
<dependency>
6060
<groupId>com.azure</groupId>
6161
<artifactId>azure-core</artifactId>
62-
<version>1.58.1</version>
62+
<version>1.60.0</version>
6363
<scope>test</scope>
6464
</dependency>
6565
<dependency>

0 commit comments

Comments
 (0)