[Crystal YU (chunyu3)]

Fix Azure/azure-sdk-tools#16915
Fix Azure/azure-sdk-tools#16916

SDK Breaking Change Analysis workflow

• implement a SDK breaking change analysis workflow. This workflow will detect and analyze the SDK breaking changes for a language and surface the result in the comment (see following example)

Example:

SDK Breaking changes for Go

Analyzed commit: 00850ca596599f280dc140fa4cc14dc613e9192b

Typespec Project: specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService

SDK Package: armwebpubsub

ID	Breaking change	Category	Suggested fix
30346390	[Model WebPubSubResource renamed] SDK struct ResourceInfo renamed to Resource, changing the parameters of *Client.BeginCreateOrUpdate and *Client.BeginUpdate and the embedded fields of ClientCreateOrUpdateResponse, ClientGetResponse, and ClientUpdateResponse.	conversion-need resolve	Apply @@clientName(Microsoft.SignalRService.WebPubSubResource, "ResourceInfo", "go") to restore the legacy readme.go.md rename-model directive in TypeSpec client customizations.
18387797	[Model WebPubSubResourceList renamed] SDK struct ResourceInfoList renamed to ResourceList, renaming the embedded fields of ClientListByResourceGroupResponse and ClientListBySubscriptionResponse.	conversion-need resolve	Apply @@clientName(Microsoft.SignalRService.WebPubSubResourceList, "ResourceInfoList", "go") to restore the legacy readme.go.md rename-model directive in TypeSpec client customizations.
57811240	Field `Family`, `Tier` of struct `ResourceSKU` has been removed	spec change

SDK breaking change analysis details

• The workflow will be triggered by SDK Breaking Change Labels workflow
• When SDK Breaking Change Labels workflow completed and find out there are SDK breaking change detected for an language, it will comment '/azsdk sdk-breaking-analysis ' to trigger this 'SDK Breaking Change Analysis' workflow

SDK Breaking Change Mitigation workflow

• implement a SDK breaking change analysis workflow. The workflow will mitigate the SDK breaking changes detected in 'SDK breaking change analysis' workflow and surface the result in the comment (see following example)

Example:

SDK Breaking Change Mitigation for Go

Mitigation upon commit: 00850ca596599f280dc140fa4cc14dc613e9192b

TypeSpec project: specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService

SDK package: armwebpubsub

Breaking Changes:

View breaking changes

Resolvable	ID	Breaking change	Resolution	TypeSpec changes
✔️	82589366	[Model WebPubSubResource renamed] SDK struct ResourceInfo renamed to Resource, changing the parameters of Client.BeginCreateOrUpdate and Client.BeginUpdate and the embedded fields of ClientCreateOrUpdateResponse, ClientGetResponse, and ClientUpdateResponse.	Restore the legacy readme.go.md rename-model directive in client.tsp with @@clientName(Microsoft.SignalRService.WebPubSubResource, "ResourceInfo", "go");.	Added @@clientName(Microsoft.SignalRService.WebPubSubResource, "ResourceInfo", "go"); to client.tsp, restoring the legacy Go model name without affecting other languages.
✔️	00974410	[Model WebPubSubResourceList renamed] SDK struct ResourceInfoList renamed to ResourceList, changing the embedded fields of ClientListByResourceGroupResponse and ClientListBySubscriptionResponse.	Restore the legacy readme.go.md rename-model directive in client.tsp with @@clientName(Microsoft.SignalRService.WebPubSubResourceList, "ResourceInfoList", "go");.	Added Go-only @@clientName(Microsoft.SignalRService.WebPubSubResourceList, "ResourceInfoList", "go") in client.tsp to restore the legacy list-model name without affecting other languages.
❌	57811240	Field `Family`, `Tier` of struct `ResourceSKU` has been removed

Customization Code:

View customization Code

diff --git a/specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService/client.tsp b/specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService/client.tsp
index 7048e05f8c..d414f06b52 100644
--- a/specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService/client.tsp
+++ b/specification/webpubsub/resource-manager/Microsoft.SignalRService/SignalRService/client.tsp
@@ -7,6 +7,9 @@ using Azure.Core;
 using Microsoft.SignalRService;
 using Http;
 
+@@clientName(Microsoft.SignalRService.WebPubSubResource, "ResourceInfo", "go");
+@@clientName(Microsoft.SignalRService.WebPubSubResourceList, "ResourceInfoList", "go");
+
 @@clientName(
   Microsoft.SignalRService,
   "WebPubSubManagementClient",

View customization code in the workflow summary

Next Steps

💡 Apply Mitigation: Apply the customization code to mitigate the breaking changes.

💡 Suppress Breaking Changes: For accepted breaking changes, click the breaking change ID link to view suppression instructions and follow the suppression process.

• The workflow will be triggered by 'SDK Breaking Change Analysis' workflow

Integrate into Next Steps to Merge

When SDK breaking-change labels are present, the Next Steps to Merge comment includes
one summary row per affected language.

An authorized SDK reviewer applies the BreakingChange-<language>-Sdk-Approved label to
approve breaking changes for that language. The language's row is removed from the
Next Steps to Merge summary when it next refreshes.

Example

SDK Breaking Change Detection and Mitigation

Language	Status	Analysis	Mitigation	Next action
Go	Breaking changes detected	View analysis Results	View mitigate Results	Review the mitigation report, apply any proposed customizations, then rerun SDK validation. For accepted breaking changes, follow the suppression process to suppress them.
JavaScript	Breaking changes detected	View analysis Results	View mitigate Results	Review the mitigation report, apply any proposed customizations, then rerun SDK validation. For accepted breaking changes, follow the suppression process to suppress them.

💡 Mitigation reports contain proposed changes; they do not mean those changes have been applied or validated.

💡 When suppressing breaking changes, ensure to use the original breaking changes.

Comment generated by summarize-checks workflow run.

[Sameeksha Vaity (samvaity)]

This is a reasonable first implementation only as an optional, non-blocking preview and analysis-only first aspect. We can grow this to release automation based on how it does.. And so I think the follow ups that I would be interested in looking at are:

• metrics for runtime, request usage, failure rates, false positives, and successful mitigations

cc: Renhe Li (@lirenhe) Ray Chen (@raych1)

[Crystal YU (chunyu3)]

This is a reasonable first implementation only as an optional, non-blocking preview and analysis-only first aspect. We can grow this to release automation based on how it does.. And so I think the follow ups that I would be interested in looking at are:

• metrics for runtime, request usage, failure rates, false positives, and successful mitigations

cc: Renhe Li (Renhe Li (@lirenhe)) Ray Chen (Ray Chen (@raych1))

Agreed. I’ll work on benchmarks for the analysis (azsdk_package_detect_breaking_change ) and mitigation (azsdk_customized_code_update) tools, and monitor runtime and related metrics to evaluate their effectiveness.