Skip to content

Commit 577d253

Browse files
Sync typespec-java core: fix XML serializer for azure-v1 and raw bytes (#5012)
Part A (fix) of the next typespec-java hotfix. Advances `packages/typespec-java/core-commit.json` to microsoft/typespec [43ff0041](microsoft/typespec@43ff004) (#11329): - Gate XML serialization to `azure-v1` data-plane clients. - Skip the XML `ObjectSerializer` for raw `byte[]`/`BinaryData` payloads that aren't structured XML models, fixing a build break from a reference to the non-generated `XmlSerializerProviders` helper. Follow-up fix on top of 0.45.8's XML serialization support. Adds a `fix` changelog entry. Version bump + publish (Part B) follows in a separate `publish/hotfix/*` PR after this merges. Targets `release/july-2026` (NOT main). --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent e734843 commit 577d253

9 files changed

Lines changed: 220 additions & 62 deletions

File tree

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: fix
3+
packages:
4+
- "@azure-tools/typespec-java"
5+
---
6+
7+
Fix XML serialization to only apply for `azure-v1` data-plane clients, and to skip the XML `ObjectSerializer` for raw `byte[]`/`BinaryData` payloads that are not structured XML models. This avoids emitting a reference to a non-generated `XmlSerializerProviders` helper (which caused a build break) for operations that return raw XML bytes.

‎packages/typespec-java/Build-Generator.ps1‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,11 @@
1010

1111
$ErrorActionPreference = "Stop"
1212

13+
# See Copy-Sources.ps1: silence progress so no cmdlet touches the controlling
14+
# terminal from turbo's background process group (which would SIGTTIN/SIGTTOU-stop
15+
# this process and hang the parallel build).
16+
$ProgressPreference = "SilentlyContinue"
17+
1318
$packageRoot = $PSScriptRoot
1419
$generatorRoot = Join-Path $packageRoot "generator"
1520
$generatorPom = Join-Path $generatorRoot "pom.xml"

‎packages/typespec-java/Copy-Sources.ps1‎

Lines changed: 31 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,33 @@
44
# Azure-flavored `src/options.ts` (excluded from the copy).
55
#
66
# The desired core commit is pinned in core-commit.json; CoreCommit.ps1's
7-
# Enter-CoreCommit temporarily checks it out (when newer than the submodule's
8-
# current checkout) to copy the sources from, and Restore-CoreCommit always puts
9-
# the submodule back to its original SHA (repo-wide `pnpm build` runs this and CI
10-
# checks git status).
7+
# Get-CoreSourceRoot returns a directory to read the emitter/generator sources from
8+
# (extracted from the pinned commit via `git archive` when it is newer than the
9+
# submodule's current checkout), without ever mutating the core/ submodule working
10+
# tree. Remove-CoreSourceRoot cleans up any temporary extraction afterwards.
1111
#
1212
# Mirrors the "Copy TypeScript code" step of autorest.java's Build-TypeSpec.ps1.
1313

1414
$ErrorActionPreference = "Stop"
1515

16+
# Disable progress rendering. Under a full parallel `pnpm build`, turbo runs each
17+
# task's process in its own background process group under the build's controlling
18+
# terminal. When a cmdlet here (notably `Copy-Item -Recurse`, which emits a
19+
# Write-Progress) renders progress, PowerShell touches that terminal from a
20+
# background process group and the OS stops the process with SIGTTIN/SIGTTOU -- it
21+
# never resumes, so the emitter build appears to hang forever (intermittently, and
22+
# only for this package since it is the only one that shells out to pwsh). Silencing
23+
# progress removes that terminal interaction entirely.
24+
$ProgressPreference = "SilentlyContinue"
25+
1626
$packageRoot = $PSScriptRoot
1727
$repoRoot = Resolve-Path (Join-Path $packageRoot ".." "..")
1828
$coreRoot = Join-Path $repoRoot "core"
1929

2030
. (Join-Path $packageRoot "CoreCommit.ps1")
21-
$originSha = Enter-CoreCommit -CoreRoot $coreRoot -PackageRoot $packageRoot
31+
$core = Get-CoreSourceRoot -CoreRoot $coreRoot -PackageRoot $packageRoot
2232
try {
23-
$emitterRoot = Resolve-Path (Join-Path $coreRoot "packages" "http-client-java" "emitter")
33+
$emitterRoot = Join-Path $core.Root "emitter"
2434

2535
Copy-Item -Path (Join-Path $emitterRoot "src") -Destination $packageRoot -Exclude "options.ts" -Recurse -Force
2636
Copy-Item -Path (Join-Path $emitterRoot "test") -Destination $packageRoot -Recurse -Force
@@ -29,15 +39,27 @@ try {
2939
# customization patch (core.patch) to the copy, so building emitter.jar
3040
# (Build-Generator.ps1) never mutates the core/ submodule. The patch paths are
3141
# relative to this generator folder.
32-
$srcGenerator = Join-Path $coreRoot "packages" "http-client-java" "generator"
42+
$srcGenerator = Join-Path $core.Root "generator"
3343
$destGenerator = Join-Path $packageRoot "generator"
3444
$patchFile = Join-Path $packageRoot "core.patch"
3545

46+
# The emitter.jar reactor (generator/pom.xml) only builds http-client-generator,
47+
# http-client-generator-mgmt and http-client-generator-core. The two test modules
48+
# below live in a separate (never-activated) Maven `test` profile and hold ~38MB
49+
# of generated Java test sources that are not part of the emitter build, so skip
50+
# copying them -- it is the bulk of the copy time. (The Azure emitter-tests
51+
# project syncs http-client-generator-test straight from core via SyncTests.ps1,
52+
# not from this copy.)
53+
$excludedGeneratorModules = @("http-client-generator-test", "http-client-generator-clientcore-test")
54+
3655
Write-Host "Copy generator sources from core"
3756
if (Test-Path $destGenerator) {
3857
Remove-Item $destGenerator -Recurse -Force
3958
}
40-
Copy-Item -Path $srcGenerator -Destination $destGenerator -Recurse -Force
59+
New-Item -ItemType Directory -Path $destGenerator | Out-Null
60+
Get-ChildItem -Path $srcGenerator -Force |
61+
Where-Object { $excludedGeneratorModules -notcontains $_.Name } |
62+
ForEach-Object { Copy-Item -Path $_.FullName -Destination $destGenerator -Recurse -Force }
4163

4264
Write-Host "Apply Azure customization patch to copied generator"
4365
# Run from the repo root with --directory so git apply resolves the patch's
@@ -50,5 +72,5 @@ try {
5072
}
5173
}
5274
finally {
53-
Restore-CoreCommit -CoreRoot $coreRoot -OriginSha $originSha
75+
Remove-CoreSourceRoot $core
5476
}
Lines changed: 104 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,36 @@
1-
# Shared helpers for running work against the desired `core/` submodule commit.
1+
# Shared helper for reading sources from the desired `core/` submodule commit
2+
# WITHOUT mutating the submodule's working tree.
23
#
3-
# Dot-source this file to import Enter-CoreCommit / Restore-CoreCommit, then wrap
4-
# the work in a try/finally:
4+
# Dot-source this file to import Get-CoreSourceRoot / Remove-CoreSourceRoot, then
5+
# wrap the work in a try/finally so the temporary extraction (if any) is cleaned up:
56
#
67
# . (Join-Path $packageRoot "CoreCommit.ps1")
7-
# $originSha = Enter-CoreCommit -CoreRoot $coreRoot -PackageRoot $packageRoot
8+
# $core = Get-CoreSourceRoot -CoreRoot $coreRoot -PackageRoot $packageRoot
89
# try {
9-
# # copy/sync from the (possibly pinned) core checkout
10+
# # read sources from under $core.Root (e.g. emitter/, generator/)
1011
# }
1112
# finally {
12-
# Restore-CoreCommit -CoreRoot $coreRoot -OriginSha $originSha
13+
# Remove-CoreSourceRoot $core
1314
# }
1415
#
1516
# The desired core commit is pinned in <PackageRoot>/core-commit.json. When that
1617
# file is present and its commit is newer than the submodule's current checkout,
17-
# Enter-CoreCommit temporarily checks it out; Restore-CoreCommit puts the submodule
18-
# back to its original SHA (repo-wide `pnpm build` runs these scripts and CI checks
19-
# git status).
18+
# Get-CoreSourceRoot extracts that commit's `packages/http-client-java` subtree into
19+
# a temporary directory via `git archive`. It never checks out a different SHA or
20+
# otherwise touches the submodule's working tree, so it is safe to run while a
21+
# parallel monorepo build (`pnpm build`, `regen-all-packages-docs`, ...) reads the
22+
# core/ submodule concurrently, and `git status` on core always stays clean.
23+
#
24+
# When core-commit.json is absent, or its commit is not newer than the current
25+
# checkout, the sources are read straight from the submodule's current working tree.
26+
27+
# Path (relative to the core repo root) of the subtree the Java package reads from.
28+
$script:CoreJavaSubtree = "packages/http-client-java"
2029

21-
# Check out the pinned core commit (when newer) and return the submodule's original
22-
# SHA so the caller can restore it in a finally block.
23-
function Enter-CoreCommit {
30+
# Resolve where to read the core Java sources from and return a descriptor:
31+
# @{ Root = <dir containing emitter/, generator/, ...>; TempDir = <temp dir or $null> }
32+
# Pass the descriptor to Remove-CoreSourceRoot in a finally block to clean up.
33+
function Get-CoreSourceRoot {
2434
param(
2535
# Path to the `core/` submodule.
2636
[Parameter(Mandatory = $true)]
@@ -31,59 +41,107 @@ function Enter-CoreCommit {
3141
[string] $PackageRoot
3242
)
3343

34-
# Capture the submodule's current SHA so the caller can restore it.
44+
# Never let any git invocation below block on an interactive credential or
45+
# host-key prompt. During a parallel `pnpm build` the emitter build runs
46+
# headless (turbo captures its stdio / attaches a pseudo-terminal), so a git
47+
# prompt has nowhere to read from and would hang the whole build forever --
48+
# observed as an intermittent, Java-only build hang. Make git fail fast instead.
49+
$env:GIT_TERMINAL_PROMPT = "0"
50+
51+
# Silence progress: a cmdlet rendering Write-Progress (e.g. the archive
52+
# extraction below) touches the controlling terminal, and under turbo's
53+
# background process group that stops this process with SIGTTIN/SIGTTOU and
54+
# hangs the build. Scoped to this function so callers are unaffected.
55+
$ProgressPreference = "SilentlyContinue"
56+
3557
$originSha = (git -C $CoreRoot rev-parse HEAD).Trim()
3658
if ($LASTEXITCODE -ne 0) {
3759
Write-Error "Failed to read the current SHA of the 'core' submodule at: $CoreRoot`nMake sure it is checked out (git submodule update --init --recursive)."
3860
}
3961

62+
$liveRoot = Join-Path $CoreRoot $script:CoreJavaSubtree
63+
4064
# core-commit.json is optional: when present it pins the core commit to use;
4165
# when absent we just use the submodule's current checkout.
4266
$configPath = Join-Path $PackageRoot "core-commit.json"
43-
$targetSha = $null
44-
if (Test-Path $configPath) {
45-
$targetSha = ((Get-Content -Raw $configPath | ConvertFrom-Json).sha).Trim()
46-
if ([string]::IsNullOrWhiteSpace($targetSha)) {
47-
Write-Error "No 'sha' found in $configPath."
48-
}
49-
}
50-
else {
67+
if (-not (Test-Path $configPath)) {
5168
Write-Host "core-commit.json not found; using current core checkout $originSha."
69+
return @{ Root = $liveRoot; TempDir = $null }
70+
}
71+
72+
$targetSha = ((Get-Content -Raw $configPath | ConvertFrom-Json).sha).Trim()
73+
if ([string]::IsNullOrWhiteSpace($targetSha)) {
74+
Write-Error "No 'sha' found in $configPath."
5275
}
5376

54-
if ($targetSha -and $targetSha -ne $originSha) {
55-
# Make sure the target commit is available locally (it may be newer than
56-
# what has been fetched); ignore failure if it is already present.
77+
if ($targetSha -eq $originSha) {
78+
return @{ Root = $liveRoot; TempDir = $null }
79+
}
80+
81+
# Make sure the target commit is available locally (the pin may be newer than
82+
# what has been fetched). Only hit the network when the commit is genuinely
83+
# missing: this code runs on EVERY emitter build and doc regen, and a `git
84+
# fetch` on core/ is the only per-build network operation in the whole repo
85+
# build, so doing it unconditionally turns any transient network/credential
86+
# stall into a Java-only build hang. `git cat-file -e <sha>^{commit}` is a
87+
# cheap, offline existence check.
88+
git -C $CoreRoot cat-file -e "${targetSha}^{commit}" 2>$null
89+
if ($LASTEXITCODE -ne 0) {
5790
git -C $CoreRoot fetch --quiet origin $targetSha 2>$null
91+
}
5892

59-
# Only move forward: check out the target when it is a descendant (newer)
60-
# of the current checkout. `merge-base --is-ancestor A B` exits 0 when A is
61-
# an ancestor of B, i.e. B is newer than A.
62-
git -C $CoreRoot merge-base --is-ancestor $originSha $targetSha
63-
if ($LASTEXITCODE -eq 0) {
64-
Write-Host "Checking out newer core commit $targetSha (was $originSha)"
65-
git -C $CoreRoot checkout --quiet $targetSha
66-
if ($LASTEXITCODE -ne 0) {
67-
Write-Error "Failed to checkout core commit $targetSha."
68-
}
69-
}
70-
else {
71-
Write-Host "core-commit.json SHA $targetSha is not newer than current $originSha; using current checkout."
72-
}
93+
# Only move forward: use the target when it is a descendant (newer) of the
94+
# current checkout. `merge-base --is-ancestor A B` exits 0 when A is an ancestor
95+
# of B, i.e. B is newer than A.
96+
git -C $CoreRoot merge-base --is-ancestor $originSha $targetSha
97+
if ($LASTEXITCODE -ne 0) {
98+
Write-Host "core-commit.json SHA $targetSha is not newer than current $originSha; using current checkout."
99+
return @{ Root = $liveRoot; TempDir = $null }
73100
}
74101

75-
return $originSha
102+
# The target is newer: extract only its packages/http-client-java subtree into a
103+
# temp directory with `git archive`. This never modifies the submodule's working
104+
# tree, so it stays safe under a parallel monorepo build that reads core/.
105+
Write-Host "Reading core sources from pinned commit $targetSha (was $originSha) without checking it out"
106+
$tempDir = Join-Path ([System.IO.Path]::GetTempPath()) ("typespec-java-core-" + [System.Guid]::NewGuid().ToString("N"))
107+
New-Item -ItemType Directory -Path $tempDir | Out-Null
108+
$zipPath = Join-Path $tempDir "core.zip"
109+
$extractDir = Join-Path $tempDir "src"
110+
try {
111+
# Produce a zip (not tar) and expand it with .NET's ZipFile.ExtractToDirectory:
112+
# this keeps the whole flow inside PowerShell/.NET and avoids depending on an
113+
# external `tar`, whose behavior varies by platform (notably Git for Windows'
114+
# GNU tar treats a drive-letter path like C:\...\core.tar as a remote host).
115+
#
116+
# ExtractToDirectory is used instead of Expand-Archive on purpose: the archived
117+
# subtree holds several thousand files, and Expand-Archive emits a per-entry
118+
# Write-Progress that makes it pathologically slow on that many files (worse when
119+
# its output is captured by a parallel `pnpm build`), to the point the emitter
120+
# build looks hung. ZipFile.ExtractToDirectory has no such overhead.
121+
git -C $CoreRoot archive --format=zip -o $zipPath "${targetSha}:$script:CoreJavaSubtree"
122+
if ($LASTEXITCODE -ne 0) {
123+
Write-Error "Failed to archive core commit ${targetSha}:$script:CoreJavaSubtree."
124+
}
125+
Add-Type -AssemblyName System.IO.Compression.FileSystem
126+
[System.IO.Compression.ZipFile]::ExtractToDirectory($zipPath, $extractDir)
127+
}
128+
catch {
129+
Remove-Item $tempDir -Recurse -Force -ErrorAction SilentlyContinue
130+
throw
131+
}
132+
Remove-Item $zipPath -Force
133+
return @{ Root = $extractDir; TempDir = $tempDir }
76134
}
77135

78-
# Restore the submodule to the SHA captured by Enter-CoreCommit.
79-
function Restore-CoreCommit {
136+
# Clean up the temporary extraction created by Get-CoreSourceRoot (no-op when the
137+
# submodule's current checkout was read directly).
138+
function Remove-CoreSourceRoot {
80139
param(
81140
[Parameter(Mandatory = $true)]
82-
[string] $CoreRoot,
83-
84-
[Parameter(Mandatory = $true)]
85-
[string] $OriginSha
141+
$CoreSource
86142
)
87143

88-
git -C $CoreRoot checkout --quiet $OriginSha
144+
if ($CoreSource.TempDir) {
145+
Remove-Item $CoreSource.TempDir -Recurse -Force -ErrorAction SilentlyContinue
146+
}
89147
}
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{ "sha": "d012e83129830c7b71a44999c4f3681dcdd9041c" }
1+
{ "sha": "43ff00415eb571c0c9e2d1274f621a61886e4a9b" }

‎packages/typespec-java/emitter-tests/Setup.ps1‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ if (Test-Path package-lock.json) {
3434
# typespec-tests references typespec-java via a local file path.
3535
# npm ci will fail when the hash of the package is different from the one in package-lock.json.
3636
# To avoid this, we remove package-lock.json and run npm install instead.
37-
npm install --registry=https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-js/npm/registry/
37+
npm install
3838

3939
# delete output
4040
if (Test-Path tsp-output) {

‎packages/typespec-java/emitter-tests/SyncTests.ps1‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,10 @@
1010
# hand-written tests and specs synced here are committed.
1111
#
1212
# The desired core commit is pinned in ../core-commit.json; CoreCommit.ps1's
13-
# Enter-CoreCommit transiently checks it out (when newer) and Restore-CoreCommit
14-
# restores the submodule afterwards.
13+
# Get-CoreSourceRoot returns a directory to read the test sources from (extracted
14+
# from the pinned commit via `git archive` when it is newer than the submodule's
15+
# current checkout), without mutating the core/ submodule working tree.
16+
# Remove-CoreSourceRoot cleans up any temporary extraction afterwards.
1517

1618
$ErrorActionPreference = "Stop"
1719

@@ -20,9 +22,9 @@ $packageRoot = Resolve-Path (Join-Path $scriptRoot "..")
2022
$coreRoot = Resolve-Path (Join-Path $scriptRoot ".." ".." ".." "core")
2123

2224
. (Join-Path $packageRoot "CoreCommit.ps1")
23-
$originSha = Enter-CoreCommit -CoreRoot $coreRoot -PackageRoot $packageRoot
25+
$core = Get-CoreSourceRoot -CoreRoot $coreRoot -PackageRoot $packageRoot
2426
try {
25-
$testRoot = Resolve-Path (Join-Path $coreRoot "packages" "http-client-java" "generator" "http-client-generator-test")
27+
$testRoot = Join-Path $core.Root "generator" "http-client-generator-test"
2628

2729
$localSrc = Join-Path $scriptRoot "src"
2830
$localTsp = Join-Path $scriptRoot "tsp"
@@ -67,7 +69,7 @@ try {
6769
Write-Host "Synced dependency and override versions from $corePackageJsonPath"
6870
}
6971
finally {
70-
Restore-CoreCommit -CoreRoot $coreRoot -OriginSha $originSha
72+
Remove-CoreSourceRoot $core
7173
}
7274

7375
# Point the local @azure-tools/typespec-java dependency at the .tgz produced by
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License.
3+
4+
package tsptest.xmlbytesverify;
5+
6+
import java.lang.reflect.Field;
7+
import java.util.Arrays;
8+
import org.junit.jupiter.api.Assertions;
9+
import org.junit.jupiter.api.Test;
10+
11+
/**
12+
* Regression test for XML serializer gating.
13+
*
14+
* An operation whose response content-type is {@code application/xml} but whose body is raw {@code bytes} (not a
15+
* structured model) must NOT generate the static XML {@code ObjectSerializer XML_SERIALIZER} field, because the
16+
* {@code XmlSerializerProviders} helper it references is only generated for models actually used in XML.
17+
*/
18+
public class XmlBytesVerifyTests {
19+
20+
private static boolean hasSerializerField(Class<?> clazz) {
21+
return Arrays.stream(clazz.getDeclaredFields()).anyMatch(f -> "XML_SERIALIZER".equals(f.getName()));
22+
}
23+
24+
@Test
25+
public void noXmlSerializerFieldForBytesPayload() {
26+
Assertions.assertFalse(hasSerializerField(XmlBytesVerifyClient.class),
27+
"XmlBytesVerifyClient should not declare a static XML_SERIALIZER field for a raw bytes XML payload.");
28+
Assertions.assertFalse(hasSerializerField(XmlBytesVerifyAsyncClient.class),
29+
"XmlBytesVerifyAsyncClient should not declare a static XML_SERIALIZER field for a raw bytes XML payload.");
30+
}
31+
32+
@Test
33+
public void xmlSerializerFieldPresentForModelPayload() throws ClassNotFoundException {
34+
// Positive control: a real XML model client does declare the SERIALIZER field, ensuring the reflection
35+
// check above is meaningful and would catch a regression that emits XML_SERIALIZER unconditionally.
36+
Class<?> modelClient = Class.forName("payload.xml.SimpleModelValueClient");
37+
Field serializer = Arrays.stream(modelClient.getDeclaredFields())
38+
.filter(f -> "XML_SERIALIZER".equals(f.getName()))
39+
.findFirst()
40+
.orElse(null);
41+
Assertions.assertNotNull(serializer,
42+
"payload.xml.SimpleModelValueClient should declare a XML_SERIALIZER field.");
43+
}
44+
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import "@typespec/rest";
2+
import "@typespec/http";
3+
import "@azure-tools/typespec-client-generator-core";
4+
5+
using TypeSpec.Http;
6+
using TypeSpec.Rest;
7+
using Azure.ClientGenerator.Core;
8+
9+
@service(#{ title: "XmlBytesVerify" })
10+
namespace TspTest.XmlBytesVerify;
11+
12+
// Operation that returns a raw XML document as bytes (like planetarycomputer getWmtsCapabilities).
13+
// The response content-type is application/xml but the body is unstructured bytes, so NO XML
14+
// ObjectSerializer (SERIALIZER) should be generated/used.
15+
@route("/xml-bytes")
16+
@get
17+
op getWmtsCapabilities(): OkResponse & {
18+
@header("content-type") contentType: "application/xml";
19+
@body body: bytes;
20+
};

0 commit comments

Comments
 (0)