Skip to content

[agentserver-responses] Keep newest history item IDs when applying limit - #48560

Merged
Shiva S (Shivakishore14) merged 8 commits into
Azure:mainfrom
Hashim1999164:fix/history-item-ids-keep-newest-48514
Aug 20, 2026
Merged

Shiva S (Shivakishore14) merged 8 commits into
Azure:mainfrom
Hashim1999164:fix/history-item-ids-keep-newest-48514

Conversation

@Hashim1999164

@Hashim1999164 Hashim Khan (Hashim1999164) commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

InMemoryResponseProvider.get_history_item_ids and FileResponseProvider.get_history_item_ids truncated with resolved[:limit], which kept the oldest IDs and dropped recent turns. Conversation history should keep the newest N items.

This changes both providers to resolved[-limit:] and tightens the unit test to assert the newest IDs are retained.

Fixes #48514

Changelog

Updated sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md under 2.1.0b2 (Unreleased).

Test plan

  1. pytest sdk/agentserver/azure-ai-agentserver-responses/tests/unit/test_in_memory_provider_crud.py::test_history__respects_limit
  2. Confirm truncated results are the last N chronological IDs

get_history_item_ids in the in-memory and file response stores used
resolved[:limit], which dropped recent turns. Slice from the end so
conversation history keeps the newest N items.
Fixes Azure#48514.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Hosted Agents sdk/agentserver/* labels Aug 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution Hashim Khan (@Hashim1999164)! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates history truncation to retain the newest items.

Changes:

  • Uses tail slicing in memory and file stores.
  • Strengthens the in-memory limit test.
  • Adds a changelog entry.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
_memory.py Keeps newest in-memory history IDs.
_file.py Keeps newest file-backed history IDs.
test_in_memory_provider_crud.py Verifies newest IDs are retained.
CHANGELOG.md Documents the bug fix.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md Outdated
@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings August 13, 2026 00:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md:7

  • The PR description says this fix belongs to 2.1.0b2 (Unreleased), but this instead changes the existing 2.1.0b1 release date and attributes the fix to that release. Add a new unreleased section and preserve the prior 2.1.0b1 (2026-08-11) heading so users can identify which package version contains the fix.
## 2.1.0b1 (2026-08-13)

### Bugs Fixed

- `get_history_item_ids` on the in-memory and file stores now keeps the newest item IDs when applying `limit`. (#48514)

Keep 2.1.0b1 as previously released and name FileResponseStore in the 2.1.0b2 notes.
Copilot AI review requested due to automatic review settings August 16, 2026 09:58
@Hashim1999164

Copy link
Copy Markdown
Contributor Author

Pushed a changelog follow up.

test_file_response_store_parity.py already expected the newest IDs (["out1", "out2", "out3"]), so that test was left unchanged.

The changelog now keeps 2.1.0b1 (2026-08-11) as released and records the history limit fix under 2.1.0b2 (Unreleased), naming FileResponseStore and InMemoryResponseProvider.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

sdk/agentserver/azure-ai-agentserver-responses/azure/ai/agentserver/responses/store/_memory.py:329

  • The function docstring for get_history_item_ids still describes returning IDs “up to limit” without specifying truncation semantics. Since this change makes limit mean “keep the most recent N”, please update the docstring (and any related protocol/interface docs, if present) to explicitly state that limit retains the newest IDs while preserving chronological order within the retained slice.
            # Keep the most recent N history items (drop oldest when over limit).
            return resolved[-limit:]

sdk/agentserver/azure-ai-agentserver-responses/azure/ai/agentserver/responses/store/_file.py:583

  • Same as the in-memory provider: please update the get_history_item_ids documentation for the file-backed store to clarify that limit keeps the most recent N IDs (drops oldest), so consumers don’t assume the previous “first N” behavior.
            # Keep the most recent N history items (drop oldest when over limit).
            return resolved[-limit:]

@github-actions

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

The python - pullrequest (Build Analyze) stage failed (build #6707528) during changelog validation for azure-ai-agentserver-responses. Two validation errors were reported:

  1. Invalid date: The changelog entry for 2.1.0b1 is dated 2026-08-11, but the validator requires it to be the latest date in the file.
  2. Empty section: The Bugs Fixed section in the changelog entry has no content — either remove the empty section header or add content to it.

Recommended next steps

  • In sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md, update the date on the 2.1.0b1 entry to today's date (or the intended release date, ensuring it is later than all other dates in the file).
  • Remove the empty ### Bugs Fixed section from the 2.1.0b1 entry, or add actual content to it.
  • See the CI troubleshooting guide: https://aka.ms/ci-fix
  • Push new commits to address the failures; this comment updates automatically on the next failing run.
Raw pipeline analysis (azsdk ci analyze)
Analyzing pipeline https://github.com/Azure/azure-sdk-for-python/pull/48560...
Getting failed workflow runs for commit 6220bba239d1621e4f4363dd2001a27d8c6fece1 in Azure/azure-sdk-for-python
Build: 6707528 Project: public PipelineUrl: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6707528

Failed Tasks

### Errors:
Verifying changelog for azure-ai-agentserver-responses
Found the following change log entry for version '2.1.0b1' in [sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md].

## 2.1.0b1 (2026-08-11)
...
[debug]Verifying as a release build because the changelog entry has a valid date.
##[error]Invalid date [ 2026-08-11 ]. The date for the changelog being released must be the latest in the file.
##[error]The changelog entry has the following sections with no content (Bugs Fixed). Please ensure to either remove the empty sections or add content to the section.
##[error]PowerShell exited with code '1'.

### Pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6707528

Failing checks on the pull request
  python - pullrequest [FAILURE] https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6707528
  python - pullrequest (Build Analyze) [FAILURE] https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6707528&view=logs&jobId=b70e5e73-bbb6-5567-0939-8415943fadb9

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 19.1 AIC · ⌖ 6.2 AIC · ⊞ 6.6K · ◷

Copilot AI review requested due to automatic review settings August 16, 2026 10:55
@Hashim1999164

Copy link
Copy Markdown
Contributor Author

Analyze failed because the validator treated 2.1.0b1 as the changelog being released. That dated entry was older than the unreleased 2.1.0b2 heading, and an earlier revision left an empty Bugs Fixed section under 2.1.0b1.

That is cleaned up. 2.1.0b1 (2026-08-11) stays as the released version with no empty sections. The history limit fix is under 2.1.0b2 (Unreleased), and the package version is now 2.1.0b2 so Analyze validates that unreleased entry.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (2)

sdk/agentserver/azure-ai-agentserver-responses/azure/ai/agentserver/responses/store/_memory.py:329

  • The new inline comment says “history items”, but resolved is a mix of history_item_ids, input_item_ids, and output_item_ids (per the method’s own logic). To avoid misleading future readers, update the comment (and ideally the docstring) to reflect that this keeps the most recent N item IDs from the resolved chain, while preserving chronological order within the returned slice.
            # Keep the most recent N history items (drop oldest when over limit).
            return resolved[-limit:]

sdk/agentserver/azure-ai-agentserver-responses/azure/ai/agentserver/responses/store/_file.py:583

  • Same as _memory.py: this comment refers to “history items”, but the function returns IDs that can include history, input, and output item IDs. Update wording (and any nearby docstring text if present in this file) to match the actual returned content and the “keep newest” semantics.
            # Keep the most recent N history items (drop oldest when over limit).
            return resolved[-limit:]

Copilot AI review requested due to automatic review settings August 16, 2026 11:15
@Hashim1999164

Copy link
Copy Markdown
Contributor Author

Updated the comments and nearby docs. They now say this keeps the most recent N item IDs from the resolved chain (history, input, and output), preserving chronological order in the returned slice.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 19, 2026 10:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 20, 2026 05:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

# Conflicts:
#	sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md
Copilot AI review requested due to automatic review settings August 20, 2026 06:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

@Shivakishore14
Shiva S (Shivakishore14) enabled auto-merge (squash) August 20, 2026 07:44
@Shivakishore14
Shiva S (Shivakishore14) merged commit e9a39aa into Azure:main Aug 20, 2026
19 checks passed
@Shivakishore14

Copy link
Copy Markdown
Member

Thanks for the PR Hashim Khan (@Hashim1999164) , much appreciated

Howie Leung (howieleung) added a commit that referenced this pull request Aug 20, 2026
* [Service Bus] Send server-timeout on management operations (#48563)

Management operations (peek, deferred receive, settlement over the
management link, lock renewal, session state, session listing,
schedule/cancel scheduled) now send `com.microsoft:server-timeout`: the
caller's remaining time less a one second buffer, or 60 seconds when none
was given, floored at zero and capped at the AMQP uint maximum. Previously
no bound was sent, so a stalled service held the call until the AMQP link
failed; it now raises a retryable `OperationTimeoutError`, surfacing after
roughly four minutes at default retries. A settle over the management link
can therefore raise `MessageLockLostError` where the call used to block.

The data path is unchanged: `receive_messages`, receiver iteration and
`send_messages` are unaffected, as is CBS token auth.

Set in the base handler after the retry loop computes the remaining time,
so it reflects the time left on the attempt. Both transports pass
application properties through unchanged, so no transport change is needed.
`REQUEST_RESPONSE_TIMEOUT` already held the correct key and was unused.

Tests cover the arithmetic and its boundaries, that the property reaches
the outgoing message, that the service's answer surfaces as a retryable
`OperationTimeoutError`, and that the value carries the transport's own
type. api.metadata.yml carries a gated parserVersion bump only;
apiMdSha256 is unchanged, so no public API moved.

Matches the .NET, Java and Go SDKs.

Refs: AB#38822503

Co-authored-by: Pranjal Patel <pranjalpatel@microsoft.com>

* Adding fix workflow to failing pipelines (#48554)

* Adding a new github actions workflow to trigger on pipeline failures.

* Updated copilot feedback.

* Update .github/workflows/pipeline-analysis-next-steps.md

Co-authored-by: Daniel Jurek <djurek@microsoft.com>

* Updated with feedback.

* Add temporary test pipeline for gh-aw trigger validation

* Revert "Add temporary test pipeline for gh-aw trigger validation"

This reverts commit 12764ed.

* . (#48155)

* Delete test-trigger-pipeline.yml

* Removed manual runs from trigger. Updated comments

* Updating the analysis workflow and adding the fix workflow.
Also added branch cleanup connected to the agentics-maintenance workflow.

* Removed testing checks.

* Updated concurrency issues and multiple suite runs.

* Updated fix workflow steps with more detail.

* Fixed edge cases and broadened branch cleanup.

* Implemented feedback.

* Removed branch cleanup. Updated analysis workflow to exclude fix PRs.

* Switched to azsdk cli to satisfy copilot feedback.

* Addressed Copilot feedback.

---------

Co-authored-by: Daniel Jurek <djurek@microsoft.com>

* Re-enable recorded azure-ai-ml e2e tests (#48538)

* Re-enable recorded azure-ai-ml e2e tests

Refresh sanitized test-proxy recordings, restore playback coverage for datastore, connection, environment, DSL, and pipeline e2e cases, and update parallel test environments for supported runtimes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014

* Fix azure-ai-ml recording placeholder mismatch

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014

* Fix azure-ai-ml playback sanitizer matching

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014

* Refresh remaining azure-ai-ml recordings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014

* [Service Bus] Make PEEK_LOCK settlement outcomes observable on pyamqp

PEEK_LOCK receiver links negotiate `rcv-settle-mode=second`, meaning settle
only once the service confirms the outcome, but the SDK sent every
disposition pre-settled and inherited a no-op `_incoming_disposition`. The
service's terminal outcome was never requested and would have been discarded
anyway, so `complete_message()` and its siblings could not fail: a settlement
the service never applied returned success in ~0.4 ms without a round trip,
and rejections carrying `com.microsoft:message-lock-lost` or
`com.microsoft:session-lock-lost` were dropped. The only symptom was the
message reappearing at lock expiry with an incremented delivery count.
Verified live: for the same dead lock in the same session, the receiver link
reported success in 0.42 ms while the management link raised
SessionLockLostError and the message stayed queued.

Settlements are now sent unsettled and wait for the service to confirm,
raising when it rejects and reporting an unconfirmed outcome so the existing
management-link fallback can re-settle authoritatively. A disposition only
confirms when the frame is settled; an outcome alone does not, matching
SenderLink. This is derived rather than configured, since .NET, Java,
JavaScript and Go all await the disposition unconditionally and none exposes
a flag to disable it. It is skipped only where there is no outcome to
observe: RECEIVE_AND_DELETE is settled by the service on delivery, and uamqp
cannot report dispositions.

Success compares the echoed outcome against the one requested, because the
service echoes what it applied rather than always replying accepted (abandon
and defer echo modified, dead_letter echoes rejected); requiring accepted
would have failed three of the four operations.

This is a behavior change: confirming costs one round trip per settlement
(~1-3 ms in region), so settle concurrently rather than serially, for example
`await asyncio.gather(*(receiver.complete_message(m) for m in messages))`.
The public surface is untouched and api.md is byte-identical to main.

Also fixes a catch-all `except AMQPException` that flattened the service's
real error condition into a generic ServiceBusConnectionError, hiding
message-lock-lost from callers, and documents the symptom in
TROUBLESHOOTING.md — a message redelivered despite a successful
complete_message() previously had no entry point.

* Allow test resource deployment to skip environment setup (#48596)

Co-authored-by: alzimmermsft <48699787+alzimmermsft@users.noreply.github.com>

* Add npm userconfig to typespec emitter template (#48639)

Fixed CFSClean

Co-authored-by: Ray Chen <raychen@microsoft.com>

* Pipeline analysis and fix patch (#48643)

* Fix pipeline analysis pull request resolution

* Fix check suite repository matching

* Fix pipeline auto-fix comment status

* Address pipeline workflow review feedback

* [CI] Remove api-consistency check (#48638)

* Remove api-consistency check.

* Restore delted api and metadata files.

* Revert eventgrid api.metadata.yml to original content

Co-authored-by: tjprescott <5723682+tjprescott@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tjprescott <5723682+tjprescott@users.noreply.github.com>

* [CI] Generate ApiHash during extended build (#48628)

* Refactoring, and add apiHash generation to build-extended-artifacts.yml

* Code review feedback.

* Test change for azure.template

* Revert azure-template changes (except pyproject.toml)

* Pass PI consistency...

* [Service Bus] Clarify application_properties returns bytes keys/values on receive (#48078)

* docs: Clarify application_properties returns bytes keys on receive (#45082)

- Correct the ServiceBusMessage constructor :paramtype for application_properties
  from Dict[str, ...] to Dict[Union[str, bytes], ...] to match the existing
  parameter annotation
- Add a note on the application_properties property that, when a message is
  received, keys and string values are returned as bytes, with the bytes-key
  access and decode pattern; note the same applies to the raw AMQP annotations
  and delivery_annotations accessed via raw_amqp_message
- Add a README sample for reading application properties from received messages
- Add a CHANGELOG Other Changes entry

Documentation-only change; no behavioral change.

* docs: Address review feedback on application_properties note (#45082)

- Guard the Optional application_properties against None before calling .get in
  both the docstring and README examples, so copying the sample does not raise
  AttributeError on a message that carries no application properties
- Correct the value-type note: an AMQP timestamp decodes to an integer
  (milliseconds), not a datetime; list only the verified native types
  (int, bool, float, uuid.UUID) and reframe as an AMQP-decode description

* eng: Refresh Service Bus API metadata

* Release agentserver responses 2.1.0b2 (#48653)

Move post-2.1.0b1 fixes into a new release entry dated 2026-08-20 and bump the package version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bdc32ee9-9e8f-4a1c-ba27-bdfa318afa1e

* [agentserver-responses] Keep newest history item IDs when applying limit (#48560)

* Keep newest history item IDs when applying limit

get_history_item_ids in the in-memory and file response stores used
resolved[:limit], which dropped recent turns. Slice from the end so
conversation history keeps the newest N items.
Fixes #48514.

* Keep newest history ids under the limit and fix changelog

* Restore an unreleased changelog section for the history id limit fix.

Keep 2.1.0b1 as previously released and name FileResponseStore in the 2.1.0b2 notes.

* Bump azure-ai-agentserver-responses to 2.1.0b2 so Analyze validates the unreleased changelog.

* Clarify that limit keeps newest item IDs from the resolved chain.

---------

Co-authored-by: Shiva S <shivakishore14@gmail.com>

* docs(servicebus): clarify and sample session listing (#48645)

* Fix Challenge Auth replay bug in Keyvault Security Domain SDK (#48636)

* Ported fix

* Updated changelog

* generated Api.md for the package as its a new CI requirement

---------

Co-authored-by: Hari K <kha@microsoft.com>

* updated changelog (#48657)

Co-authored-by: Hari K <kha@microsoft.com>

* [AutoPR azure-mgmt-compute-bulkaction]-generated-from-SDK Generation - Python-6698911 (#48575)

* Configurations:  'specification/compute/resource-manager/Microsoft.Compute/Bulkactions/tspconfig.yaml', SDK Release Type: beta, and CommitSHA: 'da7c67564bd3344610ffb0ec21a1974e79ffb00a' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6698911 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release.

* update

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3f1ab584-1bba-44a3-ac54-43dee4f2b16a

* Update CHANGELOG.md

---------

Co-authored-by: azure-sdk <azuresdk@microsoft.com>
Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
Copilot-Session: 3f1ab584-1bba-44a3-ac54-43dee4f2b16a

* [AutoPR azure-mgmt-chaos]-generated-from-SDK Generation - Python-6627851 (#48301)

* Configurations:  'specification/chaos/resource-manager/Microsoft.Chaos/Chaos/tspconfig.yaml', SDK Release Type: beta, and CommitSHA: '358882ad707dcc3b0d1de9df6ba4dfba7e85da1a' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6627851 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release.

* update

* Update CHANGELOG.md

---------

Co-authored-by: azure-sdk <azuresdk@microsoft.com>
Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com>
Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com>

* Howie/httpx2 (#48651)

* Update version

* Restore tsp-location.yaml

* Update emitter skill for Python to deal with prerequisites (#47878)

* change log (#47916)

* [azure-ai-projects] Emit SDK from TypeSpec (commit fca510e0) (#47914)

* Fix TypeSpec paths (#47959)

* Remove sample_agent_toolbox_skill.py per bakcned folks and Linda request, add new hosted agent samples for Teams message trigger and reminder preview (#48234)

* Remove sample_agent_toolbox_skill.py per bakcned folks and Linda request, add new hosted agent samples for Teams message trigger and reminder preview

* change log

* Re-emit from latest TypeSpec and do required updates (#48216)

* Re-emit, to remove WebIQ tools (#48240)

* update report

* Updates in prep for a release of 2.4.0 (#48248)

* Update rename tsp-location.yaml, so it does not break release build

* change log (#48246)

* Add sample toolboxes for synchronous and asynchronous AIProjectClient usage; update assets.json tag

* Comment out additional sample tests in TestSamples class

* Update to version 2.5.0

* Howie/sample 35 (#48310)

* Refactor agent name retrieval to use a fallback mechanism

- Updated multiple sample scripts to change the way the agent name is retrieved from environment variables.
- Replaced the default value assignment using `os.environ.get("FOUNDRY_AGENT_NAME", "MyAgent")` with a more concise approach using `os.environ.get("FOUNDRY_AGENT_NAME") or "MyAgent"`.
- This change ensures that if the environment variable is not set, the fallback value "MyAgent" is still used, while improving code readability.
- The affected files include various agent tools and hosted agent samples across the project.

* rever dataset generation job polling and update assistant prompt

* change log (#48427)

* Custom LRO pollers to enable easy access to Job ID (#48468)

* Re-emit from latest TypeSpec commit (8-10-2026 17:16) (#48526)

* update api.md files

* Update public methods report. Delete other old report

* Howie/reemit (#48608)

* Add A2A protocol support and update related models

- Introduced A2ATool and A2AToolboxTool classes for A2A protocol implementation.
- Added A2AProtocolVersion enum to define supported A2A protocol versions.
- Updated DataGenerationJobOptions to include SimulationSeedDataGenerationJobOptions and removed TaskGenerationDataGenerationJobOptions.
- Modified existing models and enums to accommodate new A2A features.
- Updated sample agent to utilize the new A2ATool class.
- Adjusted beta routines API to remove unsupported parameters and enhance pagination handling.
- Updated YAML configuration to reflect repository changes.

* Update tsp-location.yaml with latest commit and repository details

* Update sample_agent_to_agent.py skip reason in test_samples.py

* Update CHANGELOG.md to include new A2A tools and breaking changes (#48626)

* Restore tsp-location.yaml file with original content

* Update API metadata, fix README A2A preview label, and adjust package lock files

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Rename a2_a_version to a2a_version for consistency in A2A tool classes and code re-emit

* Refactor upload_session_file method and improve type hints

- Updated the `build_agents_upload_session_file_request` function to handle optional content type.
- Enhanced the `upload_session_file` method in `AgentsOperations` class to support both bytes and IO[bytes] types for content.
- Added overloads for `upload_session_file` to clarify method usage and parameters.
- Improved docstrings for better clarity on parameters and return types.
- Fixed a typo in the sample agent to agent script.
- Updated tsp-location.yaml with the latest commit hash and formatted additional directories for better readability.

* change log

* Update API metadata and improve type hints for upload_session_file method

* Add httpx to development requirements

* Update httpx version constraint in development requirements

* Add OpenAI v3 transport compatibility for httpx2 in sample executor

* version update

* Update OpenAI client to use httpx2 and raise minimum openai package version to 3.0.0

* Add cache_write_tokens to input_tokens_details in MemoryStoreOperationUsage

* Update Python version requirement to 3.10 and remove deprecated Python 3.9 classifier

* Update CHANGELOG for dependency changes and minimum Python version

---------

Co-authored-by: Darren Cohen <39422044+dargilco@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Enable bare messages= input on Relevance/Similarity/Fluency/Retrieval/ResponseCompleteness + all RAI evaluators (#48629)

* Support bare messages= input on Relevance + Similarity evaluators

Both evaluators previously required either a conversation dict or scalar
query/
esponse inputs. When the SDK batch engine invokes them from a
customer's data_mapping: {"messages": "${data.messages}"} (produced by
ACA on unified-messages evaluation groups), it sends a bare messages=[...]
kwarg alongside optional scalar context, ground_truth, and
	ool_definitions kwargs. The old code paths rejected this combination
with:

  RelevanceEvaluator: "Either 'conversation' or individual inputs must be
                       provided." ("No data to process" family)
  SimilarityEvaluator: "'query' is missing" / "Cannot provide both
                       'conversation' and individual inputs at the same time."

Fix:

- Swap ConversationValidator for MessagesOrQueryResponseInputValidator
  in each evaluator's __init__ so validation accepts either shape.
- Add a _convert_kwargs_to_eval_input override in each evaluator that
  hoists a bare messages=[...] kwarg into conversation={"messages": ...}
  and merges any top-level context / 	ool_definitions scalars into the
  same conversation dict. A top-level ground_truth scalar is stamped
  onto each assistant turn that doesn't already carry one, so the base
  _derive_conversation_converter picks it up per-response.

Regression paths preserved:
- Passing conversation={"messages": [...]} still works (unchanged).
- Passing scalar query/
esponse(/ground_truth) still works.
- SimilarityEvaluator with query=None still rejects with the pre-existing
  error message.

Verified end-to-end against real Azure OpenAI (gpt-4o-mini):
- Relevance with bare messages -> score 5.0, pass.
- Similarity with bare messages + per-turn ground_truth -> score 5.0, pass.
- Similarity with bare messages + top-level context + ground_truth +
  tool_definitions -> score 5.0, pass.
- Relevance/Similarity with q/r scalars -> score 5.0, pass (regression).
- Similarity with query=None -> correctly rejected (regression).

This is a partial migration; the same pattern applies to the other
prompty/metric/RAI/tool evaluators that today reject messages-format input
at turn/conversation level.

* Extract hoist_messages_to_conversation helper for shared reuse

The _convert_kwargs_to_eval_input normalization added to RelevanceEvaluator
and SimilarityEvaluator was identical logic in two places. Extracted it into
_evaluators/_common/_conversation_input.py as hoist_messages_to_conversation
so:

- The other ~21 bucket-3/4 evaluators (fluency, intent_resolution,
  response_completeness, retrieval, tool_call_*, RAI safety family, metric
  family) can adopt the same pattern with a ~3-line override each, instead
  of copying the same 20-line block.
- The normalization has a single documented contract about how bare
  messages= + scalar context/ground_truth/	ool_definitions kwargs
  get folded into conversation={...}.

Net line count in this commit: -39 (removed inline blocks, added helper +
imports).

Re-verified end-to-end with real gpt-4o-mini scoring — all 6 scenarios
(bare messages, bare messages + adjuncts on both evaluators, q/r baselines,
query=None regression) behave identically to the pre-extraction state.

* Support bare messages= input on Fluency, Retrieval, ResponseCompleteness

Apply the shared hoist_messages_to_conversation helper pattern to the
remaining prompty-family evaluators with a conversation overload:

- FluencyEvaluator: validator swap ConversationValidator ->
  MessagesOrQueryResponseInputValidator + convert override
- RetrievalEvaluator: convert override (no explicit validator to swap)
- ResponseCompletenessEvaluator: convert override (no explicit validator)

Not included in this commit: IntentResolutionEvaluator. Its __call__
signature only exposes one overload (query: Union[str, List[dict]],
response: Union[str, List[dict]], tool_definitions) with NO
conversation overload, so hoisting messages=[...] into
conversation={...} does not help — the base singleton matcher never
routes into a conversation converter. That evaluator needs a different
pattern (either a new conversation overload OR a custom transform that
splits the messages list into a leading query context + trailing response
turns per the intent-resolution semantic). Deferred to a follow-up.

Verified end-to-end against real gpt-4o-mini:
- Fluency messages input -> score 2.0 (baseline response scalar -> 3.0);
  no more "No data to process" error, evaluator runs to completion.
- Retrieval messages+context input -> reaches the model; both messages
  and baseline paths return NaN, so any NaN behavior is not tied to the
  migration (baseline had it too before the change).
- ResponseCompleteness messages+ground_truth input -> score 3.0 pass,
  identical to baseline.

* Support bare messages= input on all RAI service evaluators

Hoist a bare `messages=[...]` kwarg (plus optional scalar `context` /
`ground_truth` / `tool_definitions`) into `conversation={...}` inside
`RaiServiceEvaluatorBase._convert_kwargs_to_eval_input` so every RAI safety
evaluator inherits messages-input support in one change instead of eight:

- ViolenceEvaluator
- HateUnfairnessEvaluator
- SelfHarmEvaluator
- SexualEvaluator
- ProtectedMaterialEvaluator
- IndirectAttackEvaluator (XPIA)
- CodeVulnerabilityEvaluator
- ECIEvaluator

All 7 publicly-exported RAI evaluators verified to inherit the hoist via
MRO inspection. The base class already routes `conversation` into
`_evaluate_conversation` which per-turn-splits and calls the RAI service,
so no service-side change is required — the hoist just lets the customer's
`messages=` kwarg reach that existing path instead of failing at the
kwarg matcher.

Regression: pre-existing `query`/`response` scalar calls are untouched
(hoist is a no-op when `messages` is absent). Legacy-endpoint path
(`_use_legacy_endpoint`) still short-circuits with the raw conversation
dict when supplied directly.

Not exercised end-to-end here: firing a live RAI service call requires an
Azure AI project scope with RAI credentials, which is a separate infra
concern from the SDK plumbing. The kwarg-level fix (verified above) is
what the customer-facing "No data to process" error was tied to.

* Black: collapse assistant-turn conditional into one line

* Pin openai<3.0 to unblock CI (test-proxy httpx/httpx2 mismatch)

openai released v3.0 which migrated its transport layer from httpx to
httpx2. The eval SDK test proxy (tests/__openai_patcher.py) still
constructs httpx.URL objects, so with openai 3.x the tests fail with:

    TypeError: Invalid type for url. Expected str or httpx.URL,
              got <class 'httpx2.URL'>

Verified empirically: the passing PR CI run (build 6717764, queued
01:15 UTC) resolved openai==2.54.0 with httpx; the failing run (build
6717844, queued 01:50 UTC) resolved openai==3.0.0 with httpx2. The
only relevant change between the two runs was that openai 3.0 was
published to PyPI in that ~35 min window, and pip picked it because
setup.py had no upper bound.

Pinning openai<3.0 restores the CI to the passing configuration. When
the test proxy is updated to use httpx2 (or a compatibility shim), the
pin should be lifted.

This does not affect the messages= input feature under review; it is
strictly a CI unblocker.

* Address Copilot review: mutation, docs, and unit tests

Addresses three of the four Copilot review findings on the
hoist_messages_to_conversation helper:

Finding 2 (real bug): the helper mutated caller-provided message dicts
in place when stamping a top-level ground_truth. Two evaluations reusing
the same messages list with different ground_truth values would see the
first call's value leak into subsequent calls. Fix: shallow-copy the
messages list and shallow-copy each affected assistant-turn dict before
injecting ground_truth. Zero runtime cost, purely allocation.

Finding 1 (edge case): explicitly document behavior when messages entries
are not plain dicts (typed Message objects, dataclasses). The helper
skips ground_truth stamping for such items but still hoists the overall
input into conversation. The base `_derive_conversation_converter`
requires plain dicts and would fail on non-dicts regardless, so this
matches existing SDK convention.

Finding 4 (test coverage): add tests/unittests/test_conversation_input.py
with seven table-driven tests covering:
  - no messages, no conversation (noop)
  - conversation already provided (short-circuit)
  - bare messages only
  - bare messages + all adjuncts (context, ground_truth, tool_definitions)
  - non-dict messages entries skipped for stamping
  - immutability: same messages list reused across two calls with
    different ground_truth scalars (regression guard for Finding 2)
  - per-turn ground_truth is not overwritten by top-level scalar

Verified end-to-end with live gpt-4o-mini scoring on all 5 direct-file-edit
evaluators (Relevance, Similarity, Fluency, Retrieval,
ResponseCompleteness). All previously scored scenarios still score
identically; the new immutability guarantee is validated at runtime by
reusing the same messages list across two consecutive Similarity calls
with different ground_truth values and asserting the caller's assistant
dict is unchanged after both.

Finding 3 (clearer error for mixed messages+q/r inputs) is not addressed
here: the base class already rejects the mixed combination correctly,
just with a generic error message. Cosmetic; deferred.

* Move openai<3.0 pin from install_requires to dev_requirements

Addresses reviewer feedback: the previous `openai<3.0` upper bound on
install_requires punished end-user installs by making them incompatible
with the current openai 3.x line (3.0 released 2026-08-12, latest 3.3
released 2026-08-18) even though the SDK's runtime is fine with openai
3.x. The incompatibility is confined to the CI test proxy:

    tests/__openai_patcher.py:74
        request.url = httpx.URL(config.proxy_url).join(request_path)

openai 3.x switched its transport layer to httpx2 and rejects httpx.URL
objects with:

    TypeError: Invalid type for url. Expected str or httpx.URL,
              got <class 'httpx2.URL'>

Production code paths that use `AsyncAzureOpenAI` directly never
reach that patcher, so end users installing this SDK alongside openai
3.x are not affected.

Revert the setup.py upper bound and move the constraint to
dev_requirements.txt so only the test environment is pinned. Removes
this pin once tests/__openai_patcher.py is updated for httpx2
compatibility (out of scope for this PR).

* Black: collapse multi-line if in ground_truth stamping loop

* Add wiring tests for bare messages= input on the 5 direct-edit evaluators

Extends test_conversation_input.py from 7 to 25 tests. The existing 7
cover the hoist_messages_to_conversation helper in isolation. The new 18
prove each of the 5 evaluators overridden in this PR actually calls the
hoist and that the hoisted output equals the equivalent
conversation={"messages": ...} invocation.

Coverage matrix:
- A. Bare messages=[...]                                   (5 evaluators)
- B. messages= + context + ground_truth + tool_definitions (5 evaluators)
- C. Legacy (query, response, ...) regression net          (5 evaluators)
- D. Negative wiring: empty list, mixed query+messages,
     non-dict message entries                              (3 tests)

Assertion strategy: shape equivalence with the conversation={...} call
path. Robust to base-class output-shape changes because both sides
re-execute in lockstep.

25 passed, 0 failed locally (Python 3.10).

* Add wiring tests for all 8 RAI evaluators fixed via RaiServiceEvaluatorBase

Extends test_conversation_input.py from 25 to 51 tests. Every one of the
13 evaluators fixed by this PR now has direct wiring coverage.

RAI evaluators covered (parametrised over all 8):
- ViolenceEvaluator
- HateUnfairnessEvaluator
- SelfHarmEvaluator
- SexualEvaluator
- ProtectedMaterialEvaluator
- IndirectAttackEvaluator (XPIA)
- CodeVulnerabilityEvaluator
- ECIEvaluator

Coverage matrix per RAI evaluator (24 tests):
- A. Bare messages=[...]
- B. messages= + context + ground_truth + tool_definitions
- C. Legacy (query, response) regression net

Plus 2 branch tests for the RAI base override's short-circuit:
- _use_legacy_endpoint=True returns [kwargs] intact with hoisted
  conversation (single-call pre-sync-migration behavior).
- _use_legacy_endpoint=False delegates to super()'s per-turn converter
  and does not pass raw messages= kwarg through.

Same assertion strategy as the 5 direct-edit evaluators: shape
equivalence with the equivalent conversation={...} invocation.

51 passed, 0 failed locally (Python 3.10).

* Apply Black formatting to test_conversation_input.py (line-length 120)

* Update tsp-location.yaml to change commit hash and format additional directories as a list

* Update package lock and JSON files; fix variable naming in models and update commit hash in tsp-location.yaml

* update version

---------

Co-authored-by: Pranjal Patel <patelpranzp25@gmail.com>
Co-authored-by: Pranjal Patel <pranjalpatel@microsoft.com>
Co-authored-by: ReilleyMilne <91291100+ReilleyMilne@users.noreply.github.com>
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: lavakumarrepala <v-rlava@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: azure-sdk-automation[bot] <191533747+azure-sdk-automation[bot]@users.noreply.github.com>
Co-authored-by: alzimmermsft <48699787+alzimmermsft@users.noreply.github.com>
Co-authored-by: Ray Chen <raychen@microsoft.com>
Co-authored-by: Travis Prescott <tjprescott@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tjprescott <5723682+tjprescott@users.noreply.github.com>
Co-authored-by: Eldert Grootenboer <eldert@eldert.net>
Co-authored-by: Shanmukha Pasumarthy <shanmukha98@gmail.com>
Co-authored-by: Hashim Khan <64767361+Hashim1999164@users.noreply.github.com>
Co-authored-by: Shiva S <shivakishore14@gmail.com>
Co-authored-by: Hariharan K <reachk.hariharan@gmail.com>
Co-authored-by: Hari K <kha@microsoft.com>
Co-authored-by: azure-sdk <azuresdk@microsoft.com>
Co-authored-by: ChenxiJiang333 <v-chenjiang@microsoft.com>
Co-authored-by: ChenxiJiang333 <119990644+ChenxiJiang333@users.noreply.github.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
Co-authored-by: Darren Cohen <39422044+dargilco@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Rabah B <134166186+RabsB@users.noreply.github.com>
Copilot-Session: fd9e755b-1b9d-4918-aa65-1774f7f78014
Copilot-Session: bdc32ee9-9e8f-4a1c-ba27-bdfa318afa1e
Copilot-Session: 3f1ab584-1bba-44a3-ac54-43dee4f2b16a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Hosted Agents sdk/agentserver/*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[azure-ai-agentserver-responses] get_history_item_ids uses resolved[:limit] — truncates newest history instead of oldest

4 participants