Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions samples/python/autopilot-agents/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ participate in Microsoft Teams conversations.
| Sample | What you will build |
| --- | --- |
| [Hello World](hello-world/README.md) | A minimal agent that responds to Teams direct messages, group chats, and channel messages that mention it. Interactively create or reuse its Foundry project and model deployment. |
| [Foundry Toolbox](foundry-toolbox/README.md) | Combine Microsoft Learn and Bing-backed Web Search in one Foundry Toolbox to answer questions about Microsoft guidance and current public web information. |
| [Meeting Assistant](meeting-assistant/README.md) | Post meeting agenda reminders and transcript-grounded closure reports. |

Start with the [Hello World walkthrough](hello-world/README.md#step-1-install-the-prerequisites).
Expand Down
19 changes: 19 additions & 0 deletions samples/python/autopilot-agents/foundry-toolbox/.agentignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.azure/
.agentignore
.git/
.gitignore
.github/
.venv/
.vscode/
.ruff_cache/
__pycache__/
*.py[cod]
AGENTS.md
README.md
azure.yaml
toolbox.yaml
pyproject.toml
tests/
scripts/
infra/
dist/
7 changes: 7 additions & 0 deletions samples/python/autopilot-agents/foundry-toolbox/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.azure/
.venv/
.vscode/
.ruff_cache/
__pycache__/
*.py[cod]
dist/
57 changes: 57 additions & 0 deletions samples/python/autopilot-agents/foundry-toolbox/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Foundry Toolbox Autopilot

- Keep the sample focused on one Agent Framework agent consuming one dedicated,
read-only Foundry Toolbox with anonymous Microsoft Learn and native Bing-backed
Foundry Web Search. Only public search terms and URLs belong in tool arguments.
Write actions, interactive consent, approval cards, workflows, and conversation
history are out of scope.
- `agent/app.py` uses `ActivityAgentServerHost(digital_worker=True)` and its
unmodified M365 stack. Preserve `/activity/messages`, the Teams selectors,
and separate `azd deploy` / `azd ai agent publish` workflows.
- Create the agent and `FoundryToolbox` inside each Activity turn using
`agent/toolbox.py:create_agent`. Never share MCP connections across turns:
their background tasks inherit caller context at connection time. Preserve
the SDK's platform headers for both model and Toolbox calls. Never synthesize
user headers or call IDs.
- Use managed identity for hosted model and Toolbox access, and Azure CLI
credentials for local execution. Neither source requires downstream user
credentials, project connections, or secrets. Native Web Search is billed
under Bing grounding terms, not a free anonymous external MCP service.
- Validate that `TOOLBOX_ENDPOINT` belongs to the configured Foundry project
before sending a bearer token. Keep the positive allowlist in
`agent/toolbox.py` consistent with `toolbox.yaml`; require both sources.
Native Web Search is `type: web_search`, named `web_search`, with a required
string `search_query`. Preserve source links in its embedded text-resource
results. Distinguish official Microsoft guidance from other web sources.
- Learn wire names may use `.` or `___` between the server label and tool.
Accept only the explicit aliases in `TOOL_NAMES`, reject duplicate aliases,
and use exact discovered names for SDK filtering and calls. Native
`web_search` is unprefixed; do not add an MCP server label or rename it.
- Preserve resource cleanup and explicit errors for empty tools, missing
sources, unsupported approval policies, failed calls, and empty model replies.
- AgentServer owns telemetry initialization. Preserve the sensitive-content
opt-out and disable only unused OpenAI Agents SDK instrumentation.
- Reuse the sibling guided-provisioning script and create/reuse Bicep pattern.
Provisioning must not alter reused resources or role assignments. Manage the
toolbox separately with `azd ai toolbox create --from-file toolbox.yaml`.
Preserve its labels and read-only allowlists; copy azd's version-pinned MCP
endpoint to `TOOLBOX_ENDPOINT`. When choosing a new version, refresh the
endpoint from `azd ai toolbox show --output json` and redeploy.
- Use a capability-first README: short numbered setup steps, shared Hello World
prerequisites and approval guidance, then
example questions and expected Teams replies. Keep the sample-specific
Toolbox requirements and provisioning/configuration/deployment commands here.
Assume familiarity only with Hello World, not with other samples. Explain
Agent Framework's value for this agent's tool handling without sample comparisons.
Keep local development and offline tests out of the required deployment path.
Focus the demonstration on questions and user-visible answers, not traces or
SDK internals. Keep maintenance guidance brief; developer details belong here
or in the source, not in a long post-demonstration appendix.
Do not refer to development history or duplicate the full common setup guide.
- Keep `requirements.txt` fully pinned with cross-platform markers, including
transitives. Do not make `pywin32` unconditional. Use Python 3.13 explicitly;
use a fresh virtual environment when changing Python minor versions.
- Run `python -m unittest discover -s tests -v` in the sample environment.
Tests use real SDKs with HTTP test transports; preserve both-source execution,
concurrent caller isolation, Activity ingress, cleanup, and failure coverage.
Offline transports do not prove live Foundry permissions or hosted deployment.
162 changes: 162 additions & 0 deletions samples/python/autopilot-agents/foundry-toolbox/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Foundry Toolbox Autopilot

Deploy a Python Autopilot that answers questions in Teams:

- Finds official Microsoft documentation and code samples with Microsoft Learn.
- Searches the public web for recent announcements and other current information.
- Combines documentation and web results into an answer with source links.

Microsoft Agent Framework accesses both sources through **one Foundry Toolbox**:
[Microsoft Learn MCP](https://learn.microsoft.com/training/support/mcp) and
[Bing-backed Foundry Web Search](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/web-search).
Both are read-only and require no downstream user credentials. The hosted agent
uses managed identity to access the Foundry project, model, and Toolbox.

Agent Framework discovers the Toolbox functions, exposes them to the model,
and invokes the tools the model selects. It returns tool results to the model
and manages additional calls when needed, letting one answer draw on both
sources without a hand-written tool-dispatch loop.

The agent responds to Teams direct messages, group chats, and channel messages
that mention it. Each question is standalone; include the context it needs
in your message.

This preview sample assumes familiarity with the
[Hello World walkthrough](../hello-world/README.md).

## Step 1: Check the prerequisites

Use the tools, tenant setup, and resource permissions described in
[Hello World prerequisites](../hello-world/README.md#step-1-install-the-prerequisites).
This sample requires **Python 3.13**, **azd 1.32.0 or later**, and the
**azure.ai.agents extension 1.0.0-beta.16 or later**. Choose a Foundry project
with Toolbox support and a model supporting the Responses API and function calling.
Check [Web Search model and region support](https://learn.microsoft.com/azure/foundry/agents/concepts/limits-quotas-regions#tool-support-by-region-and-model)
and that subscription policy allows Web Search.

Install the additional Toolbox extension **1.0.0-beta.9 or later**:

```powershell
azd ext install azure.ai.toolboxes
```

If already installed, use `azd ext upgrade azure.ai.toolboxes`.
The developer needs **Foundry User** or equivalent Toolbox data-plane permissions
at project scope. Arrange runtime model and Toolbox access for the hosted
identity as well; see [Toolbox prerequisites](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/toolbox#prerequisites)
and [hosted-agent permissions](https://learn.microsoft.com/azure/foundry/agents/concepts/hosted-agent-permissions).

General Web Search needs no separate Bing resource, API key, or project connection,
but **incurs Bing grounding charges in addition to model usage**. Search data
is processed outside Foundry's compliance and geographic boundaries.
Review the [Web Search terms, pricing, and data handling](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/web-search).
Use public inputs only; do not send secrets, private code, personal information,
or confidential questions to either source.

## Step 2: Sign in and select an environment

From `samples\python\autopilot-agents`, enter this sample's directory. Run all
subsequent commands there in PowerShell 7. Keep the parent `scripts` directory
when copying the sample; provisioning and session updates use its shared helpers.

```powershell
Set-Location .\foundry-toolbox
az login --tenant <tenant-id>
azd auth login --tenant-id <tenant-id>
azd env new my-toolbox-autopilot
```

No existing `azd` environment is required.

## Step 3: Provision, configure the Toolbox, and deploy

```powershell
azd provision
```

Provisioning lets you create or reuse a resource group, Foundry resource, project,
and model deployment. It does not grant permissions; arrange project and
model access as described in Hello World before deployment.

Create the Toolbox in that project using [toolbox.yaml](toolbox.yaml):

```powershell
$projectEndpoint = azd env get-value FOUNDRY_PROJECT_ENDPOINT
azd ai toolbox create public-docs `
--project-endpoint $projectEndpoint `
--from-file .\toolbox.yaml `
--output json
azd ai toolbox show public-docs --project-endpoint $projectEndpoint
azd env set TOOLBOX_ENDPOINT (azd env get-value TOOLBOX_PUBLIC_DOCS_MCP_ENDPOINT)
```

The definition combines an anonymous Microsoft Learn MCP source with native
Foundry Web Search. Keep its tool names and allowlists unchanged: the sample
accepts only the three Learn functions and `web_search`.
The create command saves the version-specific MCP endpoint in the active
environment; the last command copies it to the agent's `TOOLBOX_ENDPOINT`.

```powershell
azd deploy
azd ai agent show
```

Deployment packages the Python code directly, without Docker, and creates the
hosted agent **foundry-toolbox-autopilot**. It does not publish a Microsoft 365 app.
Resource creation and model/Web Search usage can incur charges.

## Step 4: Publish and create an instance

Review the publication metadata in [azure.yaml](azure.yaml), then publish:

```powershell
azd ai agent publish
```

Have an **AI Administrator** or **Global Administrator** approve the
**Foundry Toolbox Autopilot** blueprint. Create an instance through Teams
**Apps > Agents for your team**, following
[Hello World approval and instance creation](../hello-world/README.md#step-7-approve-the-blueprint).
Deployment and publication alone do not make an instance available;
tenant approval, app policies, and licensing still apply.

## Step 5: Try the agent in Teams

In a direct chat with the instance, try each source:

> Find Microsoft documentation about Foundry Toolboxes and include source links.

Expect an answer grounded in Microsoft Learn documentation with links.

> Use web search to find recent public Microsoft Foundry announcements.
> Include source links and publication dates.

Expect a summary of recent announcements with source links and publication
dates when available. Then ask a question combining both sources:

> Use Microsoft Learn to explain Foundry Toolboxes, then use web search to
> find recent public announcements about them. Distinguish the documentation
> from the announcements, include publication dates, and link to both sources.

The reply should combine official guidance with recent announcements and
include source links for both. You can also add the instance to a group chat
or mention it in a channel.

## Updating the agent

For code or runtime-setting changes:

```powershell
azd deploy
..\scripts\stop-agent-sessions.ps1 -AgentName foundry-toolbox-autopilot
```

Stopping old sandboxes preserves logical sessions and lets subsequent invocations
use the updated deployment. Republish only when publication metadata changes.
Avoid `azd down` on environments using shared resources.

## References

- [Use a toolbox with a hosted agent](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/use-toolbox-hosted-agent)
- [Microsoft Agent Framework Toolbox integration](https://learn.microsoft.com/agent-framework/integrations/by-component/tools/foundry-toolbox)
- [Bing-backed Foundry Web Search](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/web-search)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Foundry Toolbox Autopilot sample."""
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
"""Teams message selectors, matching the Hello World Autopilot."""

from collections.abc import Callable

from microsoft_agents.activity import Activity
from microsoft_agents.hosting.core import TurnContext

ActivityMatcher = Callable[[Activity], bool]


def selector(matcher: ActivityMatcher) -> Callable[[TurnContext], bool]:
return lambda context: matcher(context.activity)


def teams_direct_message(activity: Activity) -> bool:
return (
_is_teams_message(activity)
and activity.conversation is not None
and activity.conversation.conversation_type == "personal"
)


def teams_group_chat_message(activity: Activity) -> bool:
return (
_is_teams_message(activity)
and activity.conversation is not None
and activity.conversation.conversation_type == "groupChat"
)


def teams_tagged_channel_message(activity: Activity) -> bool:
return (
_is_teams_message(activity)
and activity.conversation is not None
and activity.conversation.conversation_type == "channel"
and activity.recipient is not None
and any(
mention.mentioned is not None
and mention.mentioned.id == activity.recipient.id
for mention in activity.get_mentions()
)
)


def _is_teams_message(activity: Activity) -> bool:
return (
activity.type == "message"
and activity.channel_id is not None
and activity.channel_id.channel == "msteams"
)
84 changes: 84 additions & 0 deletions samples/python/autopilot-agents/foundry-toolbox/agent/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
"""Host a toolbox-backed Autopilot on the Foundry Activity protocol."""

import logging
from functools import partial

from aiohttp import ClientError
from azure.ai.agentserver.activity import ActivityAgentServerHost
from azure.ai.agentserver.core import configure_observability
from microsoft_agents.hosting.core import (
AgentApplication,
RouteRank, # pyright: ignore[reportPrivateImportUsage]
TurnContext,
TurnState,
)

from .activity_routing import (
selector,
teams_direct_message,
teams_group_chat_message,
teams_tagged_channel_message,
)
from .toolbox import Settings, answer

logger = logging.getLogger(__name__)


def register_handlers(agent: AgentApplication[TurnState], settings: Settings) -> None:
async def send_reply(context: TurnContext, text: str) -> None:
try:
await context.send_activity(text)
except (ClientError, TimeoutError):
logger.exception("Failed to send Teams reply")

@agent.error
async def on_error(context: TurnContext, error: Exception) -> None:
logger.error("Failed to answer using the Foundry Toolbox", exc_info=error)
await send_reply(
context,
"Sorry, I couldn't answer using the Foundry Toolbox. "
"Please try again; if this continues, ask the agent owner to check "
"the toolbox configuration and permissions.",
)

async def respond(context: TurnContext, _state: TurnState) -> None:
message = (TurnContext.remove_recipient_mention(context.activity) or "").strip()
if not message:
await send_reply(
context,
"Ask me about Microsoft documentation or current public web information.",
)
return
logger.info("Answering a Teams message with the Foundry Toolbox")
await send_reply(context, await answer(settings, message))

for matcher in (
teams_direct_message,
teams_group_chat_message,
teams_tagged_channel_message,
):
agent.add_route(selector(matcher), respond)

async def on_unhandled(context: TurnContext, _state: TurnState) -> None:
logger.info("Ignoring unsupported activity type=%s", context.activity.type)

agent.add_route(lambda _context: True, on_unhandled, rank=RouteRank.LAST)


def build_app() -> ActivityAgentServerHost:
settings = Settings.from_env()
host = ActivityAgentServerHost(
digital_worker=True,
configure_observability=partial(
configure_observability,
instrumentation_options={"openai_agents": {"enabled": False}},
),
)
register_handlers(host.agent_app, settings)
return host


def main() -> None:
host = build_app()
logger.info("Starting Foundry Toolbox Autopilot")
host.run()
Loading
Loading