Task/add tracing samples

scenarios/agent-tracing/README.md

@@ -0,0 +1,31 @@
+# Agent Tracing
+
+Reasoning about agent executions is critical for troubleshooting and debugging. Complex agents can involve many nested steps, variable execution paths, and long inputs/outputs, which makes it hard to pinpoint issues. Tracing provides a clear, chronological view of the inputs and outputs for each primitive involved in a run.
+
+This scenario sets up a structure for agent tracing using OpenTelemetry. It supports:
+- Local tracing via console or any OTLP-compatible backend (e.g., Aspire Dashboard).
+- Cloud tracing via Azure Monitor when Application Insights is enabled for your Azure AI Studio project.
+
+## Structure
+- `langchain/`: Tracing patterns for LangChain flows.
+- `langgraph/`: Tracing patterns for LangGraph agents/graphs.
+- `openai-agents/`: Tracing for OpenAI Agents with Azure OpenAI.
+
+Each subfolder contains its own `requirements.txt`, optional `dev-requirements.txt`, and `.env.sample` tailored for that sample.
+
+## Prerequisites
+- Python 3.10+ recommended.
+- An Azure AI Studio project (optional, for Azure Monitor tracing).
+- If using Azure Monitor, enable the Tracing tab in your AI Studio project to provision Application Insights and retrieve the connection string.
+
+## Installation
+Navigate to a subfolder and install its `requirements.txt`. Use `dev-requirements.txt` if you want Azure Monitor integrations.
+
+## Configuration
+- Local OTLP exporter:
+  - Set `OTEL_EXPORTER_OTLP_ENDPOINT` to your backend (e.g., Aspire Dashboard default: `http://localhost:4317` for gRPC, or `http://localhost:4318` for HTTP).
+- Azure Monitor:
+  - Copy the subfolder `.env.sample` to `.env` and set required values.
+
+## Notes
+- The initial release of Azure AI Projects had a known issue where agent function tool call details might be included in traces even when content recording is disabled. Be cautious with sensitive data and review Tracing settings in AI Studio.

scenarios/agent-tracing/langchain/.env.sample

@@ -0,0 +1,9 @@
+# Azure OpenAI configuration (required)
+AZURE_OPENAI_API_KEY=
+AZURE_OPENAI_ENDPOINT=
+AZURE_OPENAI_DEPLOYMENT=
+AZURE_OPENAI_API_VERSION=2024-02-15-preview
+
+# Optional: Enable Azure Monitor tracing via Application Insights
+APPLICATION_INSIGHTS_CONNECTION_STRING=
+

scenarios/agent-tracing/langchain/README.md

@@ -0,0 +1,62 @@
+---
+page_type: sample
+languages:
+- python
+products:
+- ai-services
+- azure-openai
+description: Pure LangChain weather assistant with Azure tracing and manual tool-calling loop.
+---
+
+## Pure LangChain Weather (Tracing)
+
+### Overview
+
+This sample demonstrates a pure LangChain agent that uses a manual tool-calling loop, instrumented with Azure Application Insights via `langchain-azure-ai`. It calls Azure OpenAI chat models and a simple `get_weather` tool, and emits OpenTelemetry traces locally or to Azure Monitor.
+
+### Objective
+
+- Use `langchain` and `langchain-openai` with Azure OpenAI chat models.
+- Add tracing via `langchain-azure-ai` and OpenTelemetry.
+- Implement a manual tool-calling loop for clarity and control.
+
+### Programming Languages
+
+- Python
+
+### Estimated Runtime: 10 mins
+
+## Set up
+
+Create and activate a local virtual environment, then install dependencies:
+
+```
+python -m venv .venv
+source .venv/bin/activate  # Windows: .venv\\Scripts\\activate
+pip install -r requirements.txt
+```
+
+Copy the environment template and set required variables:
+
+```
+cp .env.sample .env
+```
+
+Required:
+
+- `AZURE_OPENAI_API_KEY`
+- `AZURE_OPENAI_ENDPOINT`
+- `AZURE_OPENAI_DEPLOYMENT`
+- `AZURE_OPENAI_API_VERSION` (default `2024-02-15-preview`)
+
+Optional for tracing:
+
+- `APPLICATION_INSIGHTS_CONNECTION_STRING`
+
+## Run
+
+```
+python weather.py
+```
+
+You can send traces to an OTLP-compatible backend by setting `OTEL_EXPORTER_OTLP_ENDPOINT`, or to Azure Monitor by setting `APPLICATION_INSIGHTS_CONNECTION_STRING`.

scenarios/agent-tracing/langchain/dev-requirements.txt

@@ -0,0 +1,9 @@
+# Full dev setup (optional)
+langchain
+langchain-openai
+langchain-azure-ai
+opentelemetry-api
+opentelemetry-sdk
+opentelemetry-exporter-otlp
+python-dotenv
+azure-monitor-opentelemetry

scenarios/agent-tracing/langchain/requirements.txt

@@ -0,0 +1,7 @@
+langchain
+langchain-openai
+langchain-azure-ai
+opentelemetry-api
+opentelemetry-sdk
+opentelemetry-exporter-otlp
+python-dotenv

scenarios/agent-tracing/langchain/weather.py

@@ -0,0 +1,215 @@
+"""
+LangChain Weather Assistant with manual tool-calling loop + Azure tracing.
+
+Env vars required:
+  AZURE_OPENAI_API_KEY=...
+  AZURE_OPENAI_ENDPOINT=https://YOUR-RESOURCE.openai.azure.com
+  AZURE_OPENAI_DEPLOYMENT=yourDeploymentName
+  AZURE_OPENAI_API_VERSION=2024-02-15-preview  (or compatible)
+
+Optional tracing:
+  APPLICATION_INSIGHTS_CONNECTION_STRING=InstrumentationKey=...;IngestionEndpoint=...
+
+Run:
+  python weather.py
+"""
+
+import os
+import json
+import logging
+from datetime import datetime
+from typing import List, Any, Optional, Dict
+
+from langchain_core.tools import tool
+from langchain_core.messages import (
+    SystemMessage,
+    HumanMessage,
+    AIMessage,
+    ToolMessage,
+    BaseMessage,
+)
+from langchain_openai import AzureChatOpenAI
+
+try:
+    from langchain_azure_ai.callbacks.tracers import AzureAIOpenTelemetryTracer
+except ImportError:
+    AzureAIOpenTelemetryTracer = None
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger("langchain_weather")
+
+
+# -----------------------------------------------------------------------------
+# Tracing Setup (cached)
+# -----------------------------------------------------------------------------
+_TRACERS: Optional[List[Any]] = None
+
+
+def setup_tracing() -> List[Any]:
+    global _TRACERS
+    if _TRACERS is not None:
+        return _TRACERS
+    tracers: List[Any] = []
+    conn = os.getenv("APPLICATION_INSIGHTS_CONNECTION_STRING")
+    if conn and AzureAIOpenTelemetryTracer:
+        try:
+            tracer = AzureAIOpenTelemetryTracer(
+                connection_string=conn,
+                enable_content_recording=True,
+                name="langchain_weather",
+                id="weather_agent",
+            )
+            tracers.append(tracer)
+            logger.info("Azure tracing enabled.")
+        except Exception as e:
+            logger.warning(f"Failed to init tracer: {e}")
+    else:
+        logger.info("Tracing not enabled (missing APPLICATION_INSIGHTS_CONNECTION_STRING or dependency).")
+    _TRACERS = tracers
+    return tracers
+
+
+def trace_config(agent_name: str, session_id: str) -> Dict[str, Any]:
+    tracers = setup_tracing()
+    return {
+        "callbacks": tracers,
+        "tags": [f"agent:{agent_name}", agent_name, "weather-langchain"],
+        "metadata": {
+            "agent_name": agent_name,
+            "agent_type": agent_name,
+            "langgraph_node": agent_name,  # kept for parity
+            "session_id": session_id,
+            "thread_id": session_id,
+            "system": "langchain-weather",
+        },
+    }
+
+
+# -----------------------------------------------------------------------------
+# Tool
+# -----------------------------------------------------------------------------
+@tool
+def get_weather(location: str, date: Optional[str] = None) -> str:
+    """
+    Return a mock weather forecast as JSON.
+    """
+    if not date:
+        date = datetime.utcnow().strftime("%Y-%m-%d")
+    seed = sum(ord(c) for c in location.lower()) % 5
+    conditions = ["Sunny", "Partly Cloudy", "Light Rain", "Overcast", "Showers"]
+    cond = conditions[seed]
+    forecast = {
+        "location": location,
+        "date": date,
+        "condition": cond,
+        "temp_high_c": 24 + seed,
+        "temp_low_c": 14 + seed,
+        "advice": "Great day outside!" if cond == "Sunny" else "Plan for changing conditions.",
+    }
+    return json.dumps(forecast, indent=2)
+
+
+TOOLS = [get_weather]
+TOOLS_BY_NAME = {t.name: t for t in TOOLS}
+
+
+# -----------------------------------------------------------------------------
+# LLM Factory
+# -----------------------------------------------------------------------------
+def build_llm(session_id: str) -> AzureChatOpenAI:
+    required = [
+        "AZURE_OPENAI_API_KEY",
+        "AZURE_OPENAI_ENDPOINT",
+        "AZURE_OPENAI_DEPLOYMENT",
+    ]
+    missing = [v for v in required if not os.getenv(v)]
+    if missing:
+        raise RuntimeError(f"Missing Azure OpenAI env vars: {', '.join(missing)}")
+    return AzureChatOpenAI(
+        api_key=os.environ["AZURE_OPENAI_API_KEY"],
+        azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
+        azure_deployment=os.environ["AZURE_OPENAI_DEPLOYMENT"],
+        api_version=os.getenv("AZURE_OPENAI_API_VERSION", "2024-02-15-preview"),
+        temperature=0.2,
+        callbacks=setup_tracing(),
+        tags=["weather_agent", "weather-langchain"],
+        metadata={
+            "agent_type": "weather_agent",
+            "agent_name": "weather_agent",
+            "system": "langchain-weather",
+            "session_id": session_id,
+            "thread_id": session_id,
+        },
+    )
+
+
+SYSTEM_PROMPT = """You are a weather assistant.
+If user asks about weather, call the get_weather tool with (location, date if given).
+If ambiguous date, assume tomorrow.
+After tool output, summarize succinctly for the user.
+"""
+
+
+# -----------------------------------------------------------------------------
+# Agent Loop (manual)
+# -----------------------------------------------------------------------------
+def run_weather_conversation(user_query: str, session_id: str) -> str:
+    llm = build_llm(session_id)
+    # Bind tools for tool-calling (function-calling) capability
+    tool_llm = llm.bind_tools(TOOLS)
+
+    messages: List[BaseMessage] = [
+        SystemMessage(content=SYSTEM_PROMPT),
+        HumanMessage(content=user_query),
+    ]
+
+    # We allow up to N reasoning/tool steps (simple guard)
+    for step in range(5):
+        logger.info(f"LLM step {step + 1}")
+        response: AIMessage = tool_llm.invoke(messages, config=trace_config("weather_agent", session_id))
+        messages.append(response)
+
+        # If the model decided not to call any tools, we stop
+        tool_calls = getattr(response, "tool_calls", None)
+        if not tool_calls:
+            logger.info("No tool calls; finishing.")
+            break
+
+        # Execute each tool call and append ToolMessage
+        for tc in tool_calls:
+            name = tc["name"]
+            args = tc.get("args", {})
+            tool_obj = TOOLS_BY_NAME.get(name)
+            if not tool_obj:
+                tool_output = f"Tool '{name}' not found."
+            else:
+                try:
+                    tool_output = tool_obj.invoke(args)
+                except Exception as e:
+                    tool_output = f"Error executing tool '{name}': {e}"
+            messages.append(
+                ToolMessage(
+                    content=tool_output,
+                    name=name,
+                    tool_call_id=tc["id"],
+                )
+            )
+
+    # Final answer: last AI message with no tool calls OR last AI message overall
+    final_ai = next((m for m in reversed(messages) if isinstance(m, AIMessage)), None)
+    return final_ai.content if final_ai else "No AI response."
+
+
+def main() -> None:
+    print("Pure LangChain Weather (Instrumented)")
+    q = input("Ask a weather question (e.g. 'Weather in Tokyo tomorrow'): ").strip()
+    if not q:
+        q = "Weather in Paris"
+    session_id = f"lc-session-{datetime.utcnow().strftime('%Y%m%d%H%M%S')}"
+    answer = run_weather_conversation(q, session_id)
+    print("\n--- Answer ---")
+    print(answer)
+
+
+if __name__ == "__main__":
+    main()

scenarios/agent-tracing/langgraph/.env.sample

@@ -0,0 +1,9 @@
+# Azure OpenAI configuration (required)
+AZURE_OPENAI_API_KEY=
+AZURE_OPENAI_ENDPOINT=
+AZURE_OPENAI_DEPLOYMENT=
+AZURE_OPENAI_API_VERSION=2024-02-15-preview
+
+# Optional: Enable Azure Monitor tracing via Application Insights
+APPLICATION_INSIGHTS_CONNECTION_STRING=
+