Repository navigation
fix: remove run-setup.yml (causes sync drift by committing directly to public main) - #773
Closed
Brandon Miller (brandom-msft) wants to merge 1 commit into
Closed
Brandon Miller (brandom-msft) wants to merge 1 commit into
Brandon Miller (brandom-msft) wants to merge 1 commit into
Conversation
This workflow regenerates azuredeploy.json from main.bicep on push-to-main, creating commits on public that don't exist on private. This breaks the private-to-public sync marks, causing nightly sync failures. Regeneration is owned by the private repo's copy of this workflow, and the compiled ARM output flows to public via the normal sync pipeline. Refs: https://github.com/microsoft-foundry/foundry-samples-pr/actions/runs/27332078723 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
👋 Thanks for your contribution, Brandon Miller (@brandom-msft)! This repository is read-only. If you are contributing on behalf of Microsoft, please submit your PR to the private staging repository instead: See CONTRIBUTING.md for full instructions. |
foundry-samples-repo-sync Bot
pushed a commit
that referenced
this pull request
Jul 23, 2026
* Add egress control test agent sample (18-egress-control) An Agent Framework hosted agent for testing managed egress proxy policies. The agent accepts text commands (test egress, test headers, test connectivity, etc.) and makes outbound HTTP requests through the egress proxy, returning the full response to validate Allow, Deny, Transform, and Rewrite rules. Includes: - main.py with egress_test tool and aiohttp-based HTTP client - Dockerfile, agent.yaml, agent.manifest.yaml following conventions - README.md with deployment guide and 7 test scenarios covering all egress policy capabilities (Allow/Deny, Insert/Set/Remove headers, Host/Path rewrite, connectivity baseline) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix connectivity output: include title above table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update README with E2E results and advanced test scenarios - Note model deployment requirement and rate limiting considerations - Add E2E verification notes to Allow/Deny and Rewrite tests - Add advanced tests 8-12: first-match ordering, multiple transforms, combined rewrite+transform, deny-all, wildcard host matching - Update architecture diagram to show LLM in the flow - Add tip about creating agent versions with rai_config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add E2E test scripts for egress control agent Pytest-based E2E tests that create RAI egress policies, deploy agent versions, and invoke the agent to validate policy enforcement. - test_e2e_basic.py: Tests 1-7 (Allow/Deny, Transform, Rewrite, connectivity) - test_e2e_advanced.py: Tests 8-12 (first-match, multi-transform, wildcard) - conftest.py: shared helpers with env-var-based configuration - tests/README.md: setup instructions and env var reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix E2E test issues: ADC policy validation, retry, deny-all assertion - Test 7: Use wildcard allow rule instead of defaultAction=Allow (ADC rejects) - Test 6: Add retry for transient 503 from httpbin.org - Test 10: Add retry for rate limit 429 on rewrite test - Test 11: Accept generic error (not just 403) for deny-all since the agent container itself can't reach the model when all egress is blocked - Fix pytest deprecation: use cls instead of self in class-scoped fixtures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Audit mode and ManagedIdentityRef E2E tests New test files: - test_e2e_audit.py (Tests 13-15): Audit mode deny passes, audit connectivity, audit vs enforced comparison - test_e2e_managed_identity.py (Tests 16-17): MI token injection to Azure Blob Storage, MI token host scoping Updated: - conftest.py: storage helpers, MI discovery, new env vars - README.md: Audit mode section, ManagedIdentityRef setup with RBAC instructions, Troubleshooting section (addresses team feedback on basePolicyName, pre-deployed image, permissions) - tests/README.md: updated test structure and env var table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update audit tests to match current behavior (deny enforced) Audit mode doesn't passthrough yet — ADC enforces deny rules even in Audit mode. Tests now validate current behavior and are marked with TODO comments for when the Vienna audit passthrough fix (djetchev/audit-mode-passthrough) ships. Updated README audit section to document the known limitation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: hardcode model fallback, update README for audit mode and MI known issues - Default model to gpt-4o when AZURE_AI_MODEL_DEPLOYMENT_NAME is null (workaround for env var bug on version update) - Update README: audit mode passthrough is now live (July 2026) - Add known bug: managedIdentityRef token injection not functional (platform gap: IDENTITY_HEADER not passed from Vienna to ADC) - Update E2E tests to match new audit passthrough behavior - Add format field to MI test policy rules Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * fix: update manifest URL to use PR branch instead of main The sample isn't merged yet, so point azd init at the PR branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * fix: update model to gpt-4.1 for bugbash deployment - main.py fallback: gpt-4o → gpt-4.1 - agent.manifest.yaml: gpt-5.4-mini → gpt-4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * docs: add bugbash environment quick-start to README Add table with project egressbugbash, RG, endpoint, and model for bug bash participants. Update model reference to gpt-4.1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * Migrate 18-egress-control to azure.yaml layout Replace agent.yaml and agent.manifest.yaml with a single azure.yaml, matching the current azd-based sample structure. Move agent code into src/agent-framework-egress-control-responses/ and add .azdignore, .dockerignore, .env.example, plus AGENTS.md and CLAUDE.md. Update README references (extension, init URL, generated file names, manual docker build path) to the new layout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * docs: point azd init URL at canonical foundry-samples main Address review feedback: the azd ai agent init URL should reference the canonical microsoft-foundry/foundry-samples repo on main rather than the PR branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address review feedback on 18-egress-control Ankit Sinha (@ankitbko) review: - README: revert azd ext install to microsoft.foundry (azure.yaml keeps the azure.ai.agents extension namespace) - README: remove internal bug-bash environment table - README: explain the supported text commands and how they map to the tool - README: clarify Microsoft Foundry (Azure AI Services) terminology - README: document how to create/apply an egress policy via the raiPolicies API - README: clarify where the X-Adc-Proxy header and NetworkEgressDecisions telemetry are observed/configured - README: describe the ManagedIdentityRef section intent - README: replace internal codename 'Vienna' with customer-facing wording - azure.yaml: add note explaining egress policies are applied out-of-band on the account (not declared in azure.yaml) - Move the in-sample pytest E2E suite to the central daily cloud-e2e pipeline by registering test-payload.txt + test-assertions.yml under internal/tools/samples-hosted-agents/python/responses/18-egress-control Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Restore in-sample tests/ for 18-egress-control Keep the customer-facing pytest E2E suite in the sample (it has been useful for customers as a worked example) alongside the central daily cloud-e2e registration. Scrub the remaining internal 'Vienna' codename and internal work-item references from the restored test files. The central payload+assertion e2e cannot replicate the deep egress policy CRUD and agent-version deployment orchestration these tests perform, so both are kept: the central run covers deploy/invoke regression, the in-sample suite documents full policy validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address Copilot review comments on 18-egress-control - main.py: make outbound TLS verification configurable via EGRESS_TEST_VERIFY_TLS (defaults to enabled; opt out only for Full-inspection proxy testing) instead of hardcoding ssl=False - Align model references to gpt-4.1 everywhere (README, tests, conftest) to match main.py default and azure.yaml deployment - tests: xfail the managedIdentityRef suite until platform MI token injection ships (gated by EGRESS_MI_ENABLED) - tests: fix audit-mode docstrings to reflect audit passthrough (deny logged, not enforced; audit vs enforced diverge) - conftest: correct get_project_mi_principal_id docstring to match behavior (reads COGSVC_ACCOUNT_ID) - README: fix wildcard-host section to reference *.org consistently; document EGRESS_TEST_VERIFY_TLS Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix cloud e2e: remove toolbox-only test-assertions.yml from egress sample The hosted-agent assertions schema requires when.toolbox_label (a non-empty string matched against the matrix cell's toolbox). The egress-control agent has no toolbox, so the assertions file failed validation with 'document.when must be an object' and failed both the container and code[remote_build] cloud E2E jobs. Non-toolbox samples ship only test-payload.txt and rely on the default smoke validation, so drop the assertions file and keep the payload. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Rename in-sample tests/ to scenarios/ to avoid unit-test confusion Customers read these E2E runs as example policy scenarios, not unit tests. Rename the folder tests/ -> scenarios/ and files test_e2e_*.py -> scenario_*.py, update all path/filename references and docstrings, and add scenarios/pytest.ini (python_files = scenario_*.py) so `pytest scenarios/` still auto-discovers them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address Antriksh review: fix policy-schema docs, deploy guardrail via bicep, harden scenarios Review feedback on PR #645: - README schema (Tests 1-12): the policy JSON examples used a stale/flat schema (properties.defaultAction + flat actionType + transform.headerTransforms) that the raiPolicies API would reject. Re-derived every example from the known-good scenario dicts: properties.egressPolicy wrapper + basePolicyName, rules with ruleType=Fqdn and nested action{actionType, headers|rewrite}. Added a schema-convention note. All 13 JSON blocks now parse and match code. - Guardrail deployment (Ankit): the egress guardrail was only applied by the test suite, so the sample never showed deploying an agent WITH it. Added infra/egress-policy.bicep to provision the RAI egress policy, wired a policies: [rai_policy] block into azure.yaml (like the content-safety sample), and documented a "Deploy the egress guardrail" flow. - Internal detail: removed platform-internal references (IDENTITY_HEADER, Identity Proxy, ADC codename/versions, design-proposal, cross-component coordination) from README and scenario docstrings in favor of customer-facing wording. Kept the observable X-Adc-Proxy response header. - scenario_managed_identity.py: replaced module xfail with skipif + proper boolean parsing of EGRESS_MI_ENABLED, so the known-broken MI scenarios no longer create policies / deploy versions by default. - scenario_advanced.py deny-all: tightened the assertion so a model/auth/ rate-limit failure no longer passes as an egress denial (fail on ERROR:, require no success icon and a 403/Forbidden). - main.py TLS knob: EGRESS_TEST_VERIFY_TLS is now propagated to the deployed agent via azure.yaml env and the scenario-created version, so the documented setting actually reaches Full-inspection deployments. - conftest.deploy_agent_version: surface a failed traffic-pin PATCH instead of silently continuing (which could leave invocations hitting the old version and make results misleading). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix Cloud E2E: use canonical RAI policy placeholder so CI can inject a real policy The Cloud E2E runner replaces the RAI policy placeholder in azure.yaml with a real ARM ID, but only when it matches the exact canonical string ending in `/raiPolicies/<policy-name>` (see hosted-agents-cloud-e2e-runner.yml). The policies block I added used `<egress-policy-name>`, so injection was skipped and the literal placeholder reached the deploy, failing with "The specified RAI policy name ... is invalid or does not exist" (400). Align the placeholder with the canonical form used by the content-safety sample so CI swaps in the real policy and the container deploy succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Add layered azd deployment for egress control (#773) * Add layered egress infrastructure * Support provisioned and existing RAI policies --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Provision full egress-policy catalog in main.bicep Expand infra/egress/main.bicep from a single httpbin allow policy to a declarative catalog of RAI egress policies — one per guardrail pattern demonstrated by the scenarios (allow/deny, header transform insert/set/ remove, host/path rewrite, first-match priority, multi-transform, rewrite+transform, deny-all, wildcard host, audit modes, and an optional managed-identity injection policy). A loop provisions every entry; the agentPolicyName parameter selects which one is attached to the agent and exported as RAI_POLICY_ID. This makes the sample self-explanatory: the guardrail patterns are provisioned up front rather than hidden behind the test suite, which is the deployment model we want customers to use. Managed-identity injection is gated behind managedIdentityStorageHost since it requires the account MI to hold Storage Blob Data RBAC. Update the README deploy section to describe the catalog and the single-policy binding model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Cloud E2E: recognize ${RAI_POLICY_ID} layered-sample placeholder The RAI-injection step in the hosted-agents Cloud E2E runner only matched the canonical ARM-ID placeholder. Ankit's layered azd deployment (#773) changed the egress sample's azure.yaml to bind the guardrail via the ${RAI_POLICY_ID} output of a Bicep layer. CI runs with SKIP_PROVISION=true, so that layer never executes and the variable never resolves — the literal "${RAI_POLICY_ID}" reached the deploy and failed RAI validation with 400 bad_request ("The specified RAI policy name '${RAI_POLICY_ID}' is invalid or does not exist"). Broaden the injector to also recognize the ${RAI_POLICY_ID} form and swap in the real AZURE_AI_RAI_POLICY_ID, exactly as it already does for the canonical placeholder. Additive and backward-compatible: verified with mikefarah yq that both the egress sample (${RAI_POLICY_ID}) and the content-safety sample (canonical placeholder) still match and inject. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix egress catalog deployment: serialize loop and set contentFilters Validated by deploying the catalog to a live Foundry account, which surfaced two failures that would block every customer deployment: - RequestConflict: the account serializes raiPolicies writes, but the Bicep loop provisioned all policies in parallel. Add @batchsize(1) to create them one at a time. - "Content filters cannot be null": creating a policy with only basePolicyName does not inherit filters at write time. Pass an empty contentFilters array (as the scenario helpers do); the service then populates the base policy's default filters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773
foundry-samples-repo-sync Bot
pushed a commit
that referenced
this pull request
Jul 23, 2026
* Add egress control test agent sample (18-egress-control) An Agent Framework hosted agent for testing managed egress proxy policies. The agent accepts text commands (test egress, test headers, test connectivity, etc.) and makes outbound HTTP requests through the egress proxy, returning the full response to validate Allow, Deny, Transform, and Rewrite rules. Includes: - main.py with egress_test tool and aiohttp-based HTTP client - Dockerfile, agent.yaml, agent.manifest.yaml following conventions - README.md with deployment guide and 7 test scenarios covering all egress policy capabilities (Allow/Deny, Insert/Set/Remove headers, Host/Path rewrite, connectivity baseline) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix connectivity output: include title above table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update README with E2E results and advanced test scenarios - Note model deployment requirement and rate limiting considerations - Add E2E verification notes to Allow/Deny and Rewrite tests - Add advanced tests 8-12: first-match ordering, multiple transforms, combined rewrite+transform, deny-all, wildcard host matching - Update architecture diagram to show LLM in the flow - Add tip about creating agent versions with rai_config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add E2E test scripts for egress control agent Pytest-based E2E tests that create RAI egress policies, deploy agent versions, and invoke the agent to validate policy enforcement. - test_e2e_basic.py: Tests 1-7 (Allow/Deny, Transform, Rewrite, connectivity) - test_e2e_advanced.py: Tests 8-12 (first-match, multi-transform, wildcard) - conftest.py: shared helpers with env-var-based configuration - tests/README.md: setup instructions and env var reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix E2E test issues: ADC policy validation, retry, deny-all assertion - Test 7: Use wildcard allow rule instead of defaultAction=Allow (ADC rejects) - Test 6: Add retry for transient 503 from httpbin.org - Test 10: Add retry for rate limit 429 on rewrite test - Test 11: Accept generic error (not just 403) for deny-all since the agent container itself can't reach the model when all egress is blocked - Fix pytest deprecation: use cls instead of self in class-scoped fixtures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Audit mode and ManagedIdentityRef E2E tests New test files: - test_e2e_audit.py (Tests 13-15): Audit mode deny passes, audit connectivity, audit vs enforced comparison - test_e2e_managed_identity.py (Tests 16-17): MI token injection to Azure Blob Storage, MI token host scoping Updated: - conftest.py: storage helpers, MI discovery, new env vars - README.md: Audit mode section, ManagedIdentityRef setup with RBAC instructions, Troubleshooting section (addresses team feedback on basePolicyName, pre-deployed image, permissions) - tests/README.md: updated test structure and env var table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update audit tests to match current behavior (deny enforced) Audit mode doesn't passthrough yet — ADC enforces deny rules even in Audit mode. Tests now validate current behavior and are marked with TODO comments for when the Vienna audit passthrough fix (djetchev/audit-mode-passthrough) ships. Updated README audit section to document the known limitation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: hardcode model fallback, update README for audit mode and MI known issues - Default model to gpt-4o when AZURE_AI_MODEL_DEPLOYMENT_NAME is null (workaround for env var bug on version update) - Update README: audit mode passthrough is now live (July 2026) - Add known bug: managedIdentityRef token injection not functional (platform gap: IDENTITY_HEADER not passed from Vienna to ADC) - Update E2E tests to match new audit passthrough behavior - Add format field to MI test policy rules Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * fix: update manifest URL to use PR branch instead of main The sample isn't merged yet, so point azd init at the PR branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * fix: update model to gpt-4.1 for bugbash deployment - main.py fallback: gpt-4o → gpt-4.1 - agent.manifest.yaml: gpt-5.4-mini → gpt-4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * docs: add bugbash environment quick-start to README Add table with project egressbugbash, RG, endpoint, and model for bug bash participants. Update model reference to gpt-4.1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 * Migrate 18-egress-control to azure.yaml layout Replace agent.yaml and agent.manifest.yaml with a single azure.yaml, matching the current azd-based sample structure. Move agent code into src/agent-framework-egress-control-responses/ and add .azdignore, .dockerignore, .env.example, plus AGENTS.md and CLAUDE.md. Update README references (extension, init URL, generated file names, manual docker build path) to the new layout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * docs: point azd init URL at canonical foundry-samples main Address review feedback: the azd ai agent init URL should reference the canonical microsoft-foundry/foundry-samples repo on main rather than the PR branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address review feedback on 18-egress-control Ankit Sinha (@ankitbko) review: - README: revert azd ext install to microsoft.foundry (azure.yaml keeps the azure.ai.agents extension namespace) - README: remove internal bug-bash environment table - README: explain the supported text commands and how they map to the tool - README: clarify Microsoft Foundry (Azure AI Services) terminology - README: document how to create/apply an egress policy via the raiPolicies API - README: clarify where the X-Adc-Proxy header and NetworkEgressDecisions telemetry are observed/configured - README: describe the ManagedIdentityRef section intent - README: replace internal codename 'Vienna' with customer-facing wording - azure.yaml: add note explaining egress policies are applied out-of-band on the account (not declared in azure.yaml) - Move the in-sample pytest E2E suite to the central daily cloud-e2e pipeline by registering test-payload.txt + test-assertions.yml under internal/tools/samples-hosted-agents/python/responses/18-egress-control Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Restore in-sample tests/ for 18-egress-control Keep the customer-facing pytest E2E suite in the sample (it has been useful for customers as a worked example) alongside the central daily cloud-e2e registration. Scrub the remaining internal 'Vienna' codename and internal work-item references from the restored test files. The central payload+assertion e2e cannot replicate the deep egress policy CRUD and agent-version deployment orchestration these tests perform, so both are kept: the central run covers deploy/invoke regression, the in-sample suite documents full policy validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address Copilot review comments on 18-egress-control - main.py: make outbound TLS verification configurable via EGRESS_TEST_VERIFY_TLS (defaults to enabled; opt out only for Full-inspection proxy testing) instead of hardcoding ssl=False - Align model references to gpt-4.1 everywhere (README, tests, conftest) to match main.py default and azure.yaml deployment - tests: xfail the managedIdentityRef suite until platform MI token injection ships (gated by EGRESS_MI_ENABLED) - tests: fix audit-mode docstrings to reflect audit passthrough (deny logged, not enforced; audit vs enforced diverge) - conftest: correct get_project_mi_principal_id docstring to match behavior (reads COGSVC_ACCOUNT_ID) - README: fix wildcard-host section to reference *.org consistently; document EGRESS_TEST_VERIFY_TLS Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix cloud e2e: remove toolbox-only test-assertions.yml from egress sample The hosted-agent assertions schema requires when.toolbox_label (a non-empty string matched against the matrix cell's toolbox). The egress-control agent has no toolbox, so the assertions file failed validation with 'document.when must be an object' and failed both the container and code[remote_build] cloud E2E jobs. Non-toolbox samples ship only test-payload.txt and rely on the default smoke validation, so drop the assertions file and keep the payload. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Rename in-sample tests/ to scenarios/ to avoid unit-test confusion Customers read these E2E runs as example policy scenarios, not unit tests. Rename the folder tests/ -> scenarios/ and files test_e2e_*.py -> scenario_*.py, update all path/filename references and docstrings, and add scenarios/pytest.ini (python_files = scenario_*.py) so `pytest scenarios/` still auto-discovers them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Address Antriksh review: fix policy-schema docs, deploy guardrail via bicep, harden scenarios Review feedback on PR #645: - README schema (Tests 1-12): the policy JSON examples used a stale/flat schema (properties.defaultAction + flat actionType + transform.headerTransforms) that the raiPolicies API would reject. Re-derived every example from the known-good scenario dicts: properties.egressPolicy wrapper + basePolicyName, rules with ruleType=Fqdn and nested action{actionType, headers|rewrite}. Added a schema-convention note. All 13 JSON blocks now parse and match code. - Guardrail deployment (Ankit): the egress guardrail was only applied by the test suite, so the sample never showed deploying an agent WITH it. Added infra/egress-policy.bicep to provision the RAI egress policy, wired a policies: [rai_policy] block into azure.yaml (like the content-safety sample), and documented a "Deploy the egress guardrail" flow. - Internal detail: removed platform-internal references (IDENTITY_HEADER, Identity Proxy, ADC codename/versions, design-proposal, cross-component coordination) from README and scenario docstrings in favor of customer-facing wording. Kept the observable X-Adc-Proxy response header. - scenario_managed_identity.py: replaced module xfail with skipif + proper boolean parsing of EGRESS_MI_ENABLED, so the known-broken MI scenarios no longer create policies / deploy versions by default. - scenario_advanced.py deny-all: tightened the assertion so a model/auth/ rate-limit failure no longer passes as an egress denial (fail on ERROR:, require no success icon and a 403/Forbidden). - main.py TLS knob: EGRESS_TEST_VERIFY_TLS is now propagated to the deployed agent via azure.yaml env and the scenario-created version, so the documented setting actually reaches Full-inspection deployments. - conftest.deploy_agent_version: surface a failed traffic-pin PATCH instead of silently continuing (which could leave invocations hitting the old version and make results misleading). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix Cloud E2E: use canonical RAI policy placeholder so CI can inject a real policy The Cloud E2E runner replaces the RAI policy placeholder in azure.yaml with a real ARM ID, but only when it matches the exact canonical string ending in `/raiPolicies/<policy-name>` (see hosted-agents-cloud-e2e-runner.yml). The policies block I added used `<egress-policy-name>`, so injection was skipped and the literal placeholder reached the deploy, failing with "The specified RAI policy name ... is invalid or does not exist" (400). Align the placeholder with the canonical form used by the content-safety sample so CI swaps in the real policy and the container deploy succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Add layered azd deployment for egress control (#773) * Add layered egress infrastructure * Support provisioned and existing RAI policies --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Provision full egress-policy catalog in main.bicep Expand infra/egress/main.bicep from a single httpbin allow policy to a declarative catalog of RAI egress policies — one per guardrail pattern demonstrated by the scenarios (allow/deny, header transform insert/set/ remove, host/path rewrite, first-match priority, multi-transform, rewrite+transform, deny-all, wildcard host, audit modes, and an optional managed-identity injection policy). A loop provisions every entry; the agentPolicyName parameter selects which one is attached to the agent and exported as RAI_POLICY_ID. This makes the sample self-explanatory: the guardrail patterns are provisioned up front rather than hidden behind the test suite, which is the deployment model we want customers to use. Managed-identity injection is gated behind managedIdentityStorageHost since it requires the account MI to hold Storage Blob Data RBAC. Update the README deploy section to describe the catalog and the single-policy binding model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Cloud E2E: recognize ${RAI_POLICY_ID} layered-sample placeholder The RAI-injection step in the hosted-agents Cloud E2E runner only matched the canonical ARM-ID placeholder. Ankit's layered azd deployment (#773) changed the egress sample's azure.yaml to bind the guardrail via the ${RAI_POLICY_ID} output of a Bicep layer. CI runs with SKIP_PROVISION=true, so that layer never executes and the variable never resolves — the literal "${RAI_POLICY_ID}" reached the deploy and failed RAI validation with 400 bad_request ("The specified RAI policy name '${RAI_POLICY_ID}' is invalid or does not exist"). Broaden the injector to also recognize the ${RAI_POLICY_ID} form and swap in the real AZURE_AI_RAI_POLICY_ID, exactly as it already does for the canonical placeholder. Additive and backward-compatible: verified with mikefarah yq that both the egress sample (${RAI_POLICY_ID}) and the content-safety sample (canonical placeholder) still match and inject. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Fix egress catalog deployment: serialize loop and set contentFilters Validated by deploying the catalog to a live Foundry account, which surfaced two failures that would block every customer deployment: - RequestConflict: the account serializes raiPolicies writes, but the Bicep loop provisioned all policies in parallel. Add @batchsize(1) to create them one at a time. - "Content filters cannot be null": creating a policy with only basePolicyName does not inherit filters at write time. Pass an empty contentFilters array (as the scenario helpers do); the service then populates the base policy's default filters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773 * Document azd guardrail scenario invocations * Show raw output for header checks --------- Co-authored-by: Dimitar Jetchev <djetchev@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Copilot-Session: 2eb9fbb8-c320-4f0a-9818-56706d4bda09 Copilot-Session: c2887386-0625-4a4d-9346-80f996e55773
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The run-setup.yml workflow on public triggers on push to main and regenerates azuredeploy.json files from Bicep sources. When a sync PR lands a commit that touches infrastructure/infrastructure-setup-bicep/, this workflow fires and commits regenerated ARM JSON directly to public main — creating a commit that doesn't exist on private.
This breaks the nightly private-to-public sync because the marks become stale (they reference objects from a tree state that no longer matches), and seed-marks recovery correctly refuses to paper over the divergence.
Evidence
Fix
Delete the workflow from public entirely. Regeneration is owned by private; compiled ARM flows to public via the sync pipeline.
After this merges, we'll merge the pending auto/regenerate-azuredeploy-59ecf811 branch on private and re-trigger sync with seed_from_public_sha to re-anchor marks.
Also needed on private (separate PR)
Remove run-setup.yml from sync-config.json's protected_paths list — it no longer needs protection since it won't exist on public.