-
Notifications
You must be signed in to change notification settings - Fork 802
docs(http): document canonical outbound HTTP clients #2260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,7 +31,6 @@ async def my_custom_action(): | |
| | `name` | `str` | Custom name for the action | Function name | | ||
| | `is_system_action` | `bool` | Always run locally, bypassing the actions server | `False` | | ||
| | `execute_async` | `bool` | Don't block event processing while the action runs (Colang 2.x only) | `False` | | ||
| | `output_mapping` | `Callable[[Any], bool]` | Function to interpret the action result for blocking decisions | `default_output_mapping` | | ||
|
|
||
| ### Custom Action Name | ||
|
|
||
|
|
@@ -80,49 +79,25 @@ This flag is only supported in the Colang 2.x runtime. In the Colang 1.0 runtime | |
| </Note> | ||
|
|
||
| ```python | ||
| from nemoguardrails.http import HTTPClient, http_call | ||
|
|
||
| @action(execute_async=True) | ||
| async def call_external_api(endpoint: str): | ||
| async def call_external_api( | ||
| endpoint: str, | ||
| http_client: HTTPClient | None = None, | ||
| ): | ||
| """Call an external API without blocking event processing.""" | ||
| response = await http_client.get(endpoint) | ||
| response = await http_call(http_client, "GET", endpoint) | ||
| return response.json() | ||
| ``` | ||
|
|
||
| ### Output Mapping | ||
|
|
||
| The `output_mapping` parameter controls how the action's return value is interpreted to determine if output should be blocked. It accepts a callable that takes the return value and returns `True` if the output is **not safe** (should be blocked). | ||
|
|
||
| When no `output_mapping` is provided, the default behavior is: | ||
| - **Boolean results**: `True` means allowed, `False` means blocked | ||
| - **Numeric results**: Values below `0.5` are blocked | ||
| - **Other types**: Allowed by default | ||
| ### Rail Decisions | ||
|
|
||
| ```python | ||
| @action(output_mapping=lambda value: value) | ||
| async def check_hallucination(context: Optional[dict] = None): | ||
| """Return True if hallucination detected (blocked), False if safe.""" | ||
| return detect_hallucination(context.get("bot_message", "")) | ||
| ``` | ||
|
|
||
| ```python | ||
| @action(is_system_action=True, output_mapping=lambda value: not value) | ||
| async def check_output_safety(context: Optional[dict] = None): | ||
| """Return True if safe (allowed), mapped to not-blocked.""" | ||
| return is_safe(context.get("bot_message", "")) | ||
| ``` | ||
| The `@action` decorator does not interpret an action's return value as a safety decision. Ordinary custom actions can return strings, booleans, numbers, dictionaries, or other Python values for a Colang flow to consume explicitly. | ||
|
|
||
| You can also define a custom mapping function for more complex logic: | ||
| When the action itself makes a rail decision, return a [`RailOutcome`](/configure-guardrails/actions/rail-outcomes). It carries an explicit allow, block, or transform decision without relying on implicit boolean or numeric conventions. | ||
|
Comment on lines
+96
to
+98
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This guidance says ordinary boolean returns have no implicit safety meaning, but the validation examples later on the page still label bare Knowledge Base Used: Actions Framework Prompt To Fix With AIThis is a comment left during a code review.
Path: docs/configure-rails/actions/creating-actions.mdx
Line: 96-98
Comment:
**Align boolean safety examples**
This guidance says ordinary boolean returns have no implicit safety meaning, but the validation examples later on the page still label bare `True` and `False` values as allowing or blocking content without showing a consuming Colang branch. Readers can copy those examples expecting enforcement that does not occur, so either demonstrate the explicit flow branch or return `RailOutcome`.
**Knowledge Base Used:** [Actions Framework](https://app.greptile.com/nvidia-public-github/-/custom-context/knowledge-base/nvidia-nemo/guardrails/-/docs/actions.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
|
|
||
| ```python | ||
| def my_custom_mapping(result): | ||
| if isinstance(result, dict): | ||
| return result.get("score", 1.0) < 0.7 | ||
| return False | ||
|
|
||
| @action(output_mapping=my_custom_mapping) | ||
| async def score_safety(context: Optional[dict] = None): | ||
| """Return a dict with a safety score.""" | ||
| return {"score": compute_score(context.get("bot_message", ""))} | ||
| ``` | ||
| If you previously used the removed `output_mapping` decorator parameter, follow the [migration guide](/configure-guardrails/actions/rail-outcomes#migrate-from-output-mapping). | ||
|
|
||
| ## Function Parameters | ||
|
|
||
|
|
@@ -174,6 +149,8 @@ async def search_documents( | |
|
|
||
| Actions can return various types: | ||
|
|
||
| Manifest-backed rail actions are the exception. They must return a [`RailOutcome`](/configure-guardrails/actions/rail-outcomes). | ||
|
|
||
| ### Simple Return | ||
|
|
||
| ```python | ||
|
|
@@ -209,18 +186,19 @@ async def is_safe_content(context: Optional[dict] = None): | |
| Handle errors gracefully within actions: | ||
|
|
||
| ```python | ||
| from nemoguardrails.http import HTTPClient, HTTPTimeoutError, http_call | ||
|
|
||
| @action() | ||
| async def fetch_data(url: str): | ||
| async def fetch_data( | ||
| url: str, | ||
| http_client: HTTPClient | None = None, | ||
| ): | ||
| """Fetch data with error handling.""" | ||
| try: | ||
| response = await http_client.get(url) | ||
| response.raise_for_status() | ||
| response = await http_call(http_client, "GET", url) | ||
| return response.json() | ||
| except Exception as e: | ||
| # Log the error | ||
| print(f"Error fetching data: {e}") | ||
| # Return a safe default or raise | ||
| return None | ||
| except HTTPTimeoutError as error: | ||
| raise RuntimeError("External data service timed out") from error | ||
| ``` | ||
|
|
||
| ## Example Actions | ||
|
|
@@ -267,22 +245,27 @@ async def filter_sensitive_data(context: Optional[dict] = None): | |
| ### External API Action | ||
|
|
||
| ```python | ||
| import aiohttp | ||
| from nemoguardrails.http import HTTPClient, http_call | ||
|
|
||
| @action(execute_async=True) | ||
| async def query_knowledge_base(query: str, top_k: int = 5): | ||
| async def query_knowledge_base( | ||
| query: str, | ||
| top_k: int = 5, | ||
| http_client: HTTPClient | None = None, | ||
| ): | ||
| """Query an external knowledge base API.""" | ||
| async with aiohttp.ClientSession() as session: | ||
| async with session.post( | ||
| "https://api.example.com/search", | ||
| json={"query": query, "limit": top_k} | ||
| ) as response: | ||
| data = await response.json() | ||
| return data.get("results", []) | ||
| response = await http_call( | ||
| http_client, | ||
| "POST", | ||
| "https://api.example.com/search", | ||
| json={"query": query, "limit": top_k}, | ||
| ) | ||
| return response.json().get("results", []) | ||
| ``` | ||
|
|
||
| ## Related Topics | ||
|
|
||
| - [Built-in Actions](built-in-actions) - Default actions in the library | ||
| - [Action Parameters](action-parameters) - Special parameters provided automatically | ||
| - [Registering Actions](registering-actions) - Different ways to register actions | ||
| - [Outbound HTTP](outbound-http) - Send external requests through the canonical client boundary | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,223 @@ | ||
| --- | ||
| title: "Outbound HTTP in Actions" | ||
| sidebar-title: "Outbound HTTP" | ||
| description: "Use the canonical asynchronous HTTP boundary for action requests, retries, telemetry, lifecycle management, and deterministic tests." | ||
| keywords: ["HTTPClient", "http_call", "action HTTP", "HTTP retries", "HTTP telemetry"] | ||
| content: | ||
| type: "how_to" | ||
| --- | ||
|
|
||
| Use the `nemoguardrails.http` boundary for outbound HTTP requests from actions and library rails. It provides transport-neutral request, response, error, retry, and instrumentation contracts while keeping client ownership explicit. | ||
|
|
||
| Do not construct `aiohttp`, `httpx`, `requests`, or `urllib3` clients inside an action. Direct transports bypass shared ownership, retry, testing, and observability policy. | ||
|
|
||
| ## Send a request from an action | ||
|
|
||
| Accept an optional `HTTPClient` and call `http_call`: | ||
|
|
||
| ```python | ||
| from nemoguardrails.actions import action | ||
| from nemoguardrails.http import HTTPClient, http_call | ||
|
|
||
| @action() | ||
| async def query_policy_service( | ||
| text: str, | ||
| http_client: HTTPClient | None = None, | ||
| ): | ||
| response = await http_call( | ||
| http_client, | ||
| "POST", | ||
| "https://policy.example.com/v1/check", | ||
| json={"text": text}, | ||
| timeout=10.0, | ||
| ) | ||
| return response.json() | ||
| ``` | ||
|
|
||
| `http_call` raises `HTTPStatusError` for responses with status code 400 or greater by default. Pass `raise_for_status=False` only when the integration must inspect an error response and apply provider-specific behavior. | ||
|
|
||
| ## Client ownership | ||
|
|
||
| The client argument determines ownership for each call: | ||
|
|
||
| | Client value | Owner | Behavior | | ||
| | --- | --- | --- | | ||
| | Injected `HTTPClient` | Caller | `http_call` borrows the client and leaves it open. | | ||
| | `None` | `http_call` | The helper creates a client for the call and closes it after the response body is materialized. | | ||
|
|
||
| The `None` fallback is safe and deterministic. Inject a shared client for applications that make repeated requests and benefit from connection-pool reuse. | ||
|
|
||
| Create and close a shared client at the same application-lifecycle boundary: | ||
|
|
||
| ```python | ||
| from nemoguardrails import LLMRails, RailsConfig | ||
| from nemoguardrails.http import create_http_client | ||
|
|
||
| async def run(): | ||
| http_client = create_http_client(timeout=10.0) | ||
| config = RailsConfig.from_path("config") | ||
| app = LLMRails(config) | ||
| app.register_action_param("http_client", http_client) | ||
|
|
||
| try: | ||
| return await app.generate_async( | ||
| messages=[{"role": "user", "content": "Hello"}], | ||
| ) | ||
| finally: | ||
| await http_client.close() | ||
| ``` | ||
|
|
||
| The synchronous `config.py` initialization hook has no asynchronous teardown hook. Do not create a long-lived HTTP client there unless another application component owns and closes it. | ||
|
|
||
| ## Request and response contract | ||
|
|
||
| `HTTPClient.request` and `http_call` support: | ||
|
|
||
| - HTTP method and absolute URL. | ||
| - Headers and query parameters. | ||
| - A JSON body or raw string or byte content. | ||
| - A per-request total timeout. | ||
|
|
||
| They return a materialized `HTTPResponse`. Its body bytes remain available after a call-scoped client closes. | ||
|
|
||
| ```python | ||
| response.status_code | ||
| response.headers | ||
| response.content | ||
| response.text | ||
| response.json() | ||
| response.is_success | ||
| ``` | ||
|
|
||
| `HTTPResponse.json()` raises `HTTPResponseDecodeError` for invalid JSON. `HTTPResponse.raise_for_status()` and `http_call` raise `HTTPStatusError` for status codes of 400 or greater. | ||
|
|
||
| The canonical error hierarchy is: | ||
|
|
||
| - `HTTPClientError` | ||
| - `HTTPConnectionError` | ||
| - `HTTPTimeoutError` | ||
| - `HTTPStatusError` | ||
| - `HTTPResponseDecodeError` | ||
|
|
||
| Catch the narrowest error that the integration can handle without changing its intended fail-open or fail-closed behavior. Do not log response bodies, request bodies, credentials, or unsanitized exception details. | ||
|
|
||
| ## Configure the pooled transport | ||
|
|
||
| `create_http_client` creates a closable HTTPX-backed client behind the neutral protocol. The default client: | ||
|
|
||
| - Uses a 30-second total timeout. | ||
| - Verifies TLS certificates. | ||
| - Does not follow redirects. | ||
| - Pools up to 100 connections, including up to 20 keep-alive connections. | ||
|
|
||
| Override these policies explicitly when an integration requires different behavior: | ||
|
|
||
| ```python | ||
| import httpx | ||
|
|
||
| from nemoguardrails.http import HTTPTLSConfig, create_http_client | ||
|
|
||
| client = create_http_client( | ||
| timeout=10.0, | ||
| limits=httpx.Limits(max_connections=40, max_keepalive_connections=10), | ||
| follow_redirects=False, | ||
| tls=HTTPTLSConfig(ca_bundle="/path/to/ca-bundle.pem"), | ||
| ) | ||
| ``` | ||
|
|
||
| `HTTPTLSConfig` also supports a client certificate and key for mutual TLS. Configure both together. Keep certificate verification enabled in production. | ||
|
|
||
| ## Add bounded retries | ||
|
|
||
| Requests are not retried unless the client has a `RetryPolicy`. `max_attempts` includes the initial request. | ||
|
|
||
| ```python | ||
| from nemoguardrails.http import RetryPolicy, create_http_client | ||
|
|
||
| policy = RetryPolicy( | ||
| max_attempts=3, | ||
| initial_delay=0.25, | ||
| max_delay=2.0, | ||
| ) | ||
| client = create_http_client(retry_policy=policy) | ||
| ``` | ||
|
|
||
| The default policy retries eligible connection and timeout failures and the status codes `408`, `409`, `429`, `500`, `502`, `503`, and `504`. It uses bounded exponential backoff with jitter and accepts an in-policy `Retry-After` value. | ||
|
|
||
| The safe default method set excludes `POST`. Add `POST` only when the provider documents the operation as retry-safe or the request uses a supported idempotency mechanism: | ||
|
|
||
| ```python | ||
| policy = RetryPolicy( | ||
| max_attempts=3, | ||
| retryable_methods=frozenset({"POST"}), | ||
| ) | ||
| ``` | ||
|
Comment on lines
+149
to
+154
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 12 -P '^\s*class\s+RetryPolicy\b|^\s*retryable_methods\s*[:=]|retryable_methods\s*=' nemoguardrails/http
rg -n -C 6 -P 'RetryPolicy\(|retryable_methods' tests docsRepository: NVIDIA-NeMo/Guardrails Length of output: 17627 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ast
from pathlib import Path
retry_path = Path("nemoguardrails/http/retry.py")
test_path = Path("tests/http/test_retry.py")
docs_path = Path("docs/configure-rails/actions/outbound-http.mdx")
retry_tree = ast.parse(retry_path.read_text())
policy = next(
node for node in retry_tree.body
if isinstance(node, ast.ClassDef) and node.name == "RetryPolicy"
)
methods_field = next(
node for node in policy.body
if isinstance(node, ast.AnnAssign)
and isinstance(node.target, ast.Name)
and node.target.id == "retryable_methods"
)
default_call = methods_field.value
assert isinstance(default_call, ast.Call)
assert isinstance(default_call.func, ast.Name)
assert default_call.func.id == "field"
factory = next(
keyword.value for keyword in default_call.keywords
if keyword.arg == "default_factory"
)
assert isinstance(factory, ast.Lambda)
default_frozenset = ast.literal_eval(factory.body)
assert default_frozenset == frozenset(
{"DELETE", "GET", "HEAD", "OPTIONS", "PUT", "TRACE"}
)
test_text = test_path.read_text()
assert 'RetryPolicy(retryable_methods=frozenset({"post"}))' in test_text
assert 'await client.request("GET", "https://example.com")' in test_text
docs_text = docs_path.read_text()
assert "Add `POST` only" in docs_text
assert 'retryable_methods=frozenset({"POST"})' in docs_text
print("default retryable methods:", sorted(default_frozenset))
print("explicit POST set:", ["POST"])
print("explicit POST set extends defaults:", bool(default_frozenset | {"POST"} == {"POST"}))
print("source/test evidence: explicit method set is supplied directly and GET is tested as non-retryable")
print("documentation wording: says to add POST, but example contains only POST")
PYRepository: NVIDIA-NeMo/Guardrails Length of output: 832 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
import ast
from pathlib import Path
retry_path = Path("nemoguardrails/http/retry.py")
test_path = Path("tests/http/test_retry.py")
docs_path = Path("docs/configure-rails/actions/outbound-http.mdx")
tree = ast.parse(retry_path.read_text())
policy = next(
node for node in tree.body
if isinstance(node, ast.ClassDef) and node.name == "RetryPolicy"
)
field_node = next(
node for node in policy.body
if isinstance(node, ast.AnnAssign)
and isinstance(node.target, ast.Name)
and node.target.id == "retryable_methods"
)
field_call = field_node.value
factory = next(
keyword.value for keyword in field_call.keywords
if keyword.arg == "default_factory"
)
default_expr = factory.body
assert isinstance(default_expr, ast.Call)
assert isinstance(default_expr.func, ast.Name)
assert default_expr.func.id == "frozenset"
default_methods = frozenset(
ast.literal_eval(element)
for element in default_expr.args[0].elts
)
explicit_methods = frozenset({"POST"})
assert "POST" not in default_methods
assert explicit_methods != default_methods
assert default_methods | explicit_methods != explicit_methods
membership = []
for node in ast.walk(tree):
if isinstance(node, ast.Compare) and any(isinstance(op, ast.In) for op in node.ops):
membership.append(ast.unparse(node))
assert any("retryable_methods" in expression for expression in membership)
test_text = test_path.read_text()
docs_text = docs_path.read_text()
assert 'RetryPolicy(retryable_methods=frozenset({"post"}))' in test_text
assert 'await client.request("GET", "https://example.com")' in test_text
assert "Add `POST` only" in docs_text
assert 'retryable_methods=frozenset({"POST"})' in docs_text
print("default retryable methods:", sorted(default_methods))
print("explicit retryable methods:", sorted(explicit_methods))
print("explicit value extends the default:", bool(default_methods | explicit_methods == explicit_methods))
print("retry-method membership checks:", membership)
print("test and documentation evidence: an explicit POST-only set is used")
PYRepository: NVIDIA-NeMo/Guardrails Length of output: 518 Preserve the default retryable methods when adding
🤖 Prompt for AI Agents |
||
|
|
||
| Keep retry policy close to the integration that owns the provider semantics. Do not add a broad global POST retry policy. | ||
|
|
||
| ## Add privacy-safe instrumentation | ||
|
|
||
| Wrap a shared client with `instrument_http_client` to enable tracing, metrics, or both: | ||
|
|
||
| ```python | ||
| from opentelemetry import trace | ||
|
|
||
| from nemoguardrails.http import ( | ||
| RetryPolicy, | ||
| create_http_client, | ||
| instrument_http_client, | ||
| ) | ||
|
|
||
| base_client = create_http_client(retry_policy=RetryPolicy()) | ||
| http_client = instrument_http_client( | ||
| base_client, | ||
| tracer=trace.get_tracer("guardrails-app"), | ||
| metrics_enabled=True, | ||
| ) | ||
| ``` | ||
|
|
||
| Wrap the retrying client, as shown above, to record one span and one duration observation for the logical request rather than one per retry attempt. | ||
|
|
||
| Tracing emits a `CLIENT` span named `HTTP {METHOD}`. Metrics emit the `http.client.request.duration` histogram. Telemetry can include: | ||
|
|
||
| - Method, URL scheme, server address, and port. | ||
| - A sanitized URL without credentials, query parameters, or fragments. | ||
| - Raw request-body size when `content` is used. | ||
| - Response status and body size. | ||
| - Retry count and error type. | ||
|
|
||
| Instrumentation does not record header values, query values, JSON bodies, raw body content, credentials, response content, or exception messages. Telemetry failures do not change the request result. | ||
|
|
||
| Instrumentation is explicit at this boundary. Creating an HTTP client without `instrument_http_client` does not enable HTTP spans or metrics automatically. The component that creates the instrumented client must also close it. | ||
|
|
||
| See [Span Reference](/observability/tracing/span-reference#outbound-http-client-spans) and [Metric Reference](/observability/metrics/reference#http-client-metrics) for the emitted names and attributes. | ||
|
|
||
| ## Test without network access | ||
|
|
||
| Use `RecordingHTTPClient` to queue responses and inspect the exact provider request: | ||
|
|
||
| ```python | ||
| import pytest | ||
|
|
||
| from nemoguardrails.http import HTTPResponse | ||
| from nemoguardrails.testing import RecordingHTTPClient | ||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_policy_request_contract(): | ||
| client = RecordingHTTPClient( | ||
| [HTTPResponse(status_code=200, content=b'{"allowed": true}')] | ||
| ) | ||
|
|
||
| result = await query_policy_service("hello", http_client=client) | ||
|
|
||
| assert result == {"allowed": True} | ||
| assert len(client.requests) == 1 | ||
| request = client.requests[0] | ||
| assert request.method == "POST" | ||
| assert request.url == "https://policy.example.com/v1/check" | ||
| assert request.json == {"text": "hello"} | ||
| ``` | ||
|
|
||
| Queue transport errors and non-success responses to verify retry, timeout, decoding, and fail-open or fail-closed behavior. Unit tests must not call a live provider. | ||
|
|
||
| For configuration-level testing patterns, see [Testing Your Guardrails Configuration](/configure-guardrails/custom-initialization/testing-your-config). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Import
actionin every standalone example.Each block uses
@actionbut imports only HTTP symbols. Copying any block raisesNameError.docs/configure-rails/actions/creating-actions.mdx#L82-L91: importactionbefore the decorator.docs/configure-rails/actions/creating-actions.mdx#L189-L201: importactionbefore the decorator.docs/configure-rails/actions/creating-actions.mdx#L248-L263: importactionbefore the decorator.Proposed fix
+from nemoguardrails.actions import action from nemoguardrails.http import HTTPClient, http_call📍 Affects 1 file
docs/configure-rails/actions/creating-actions.mdx#L82-L91(this comment)docs/configure-rails/actions/creating-actions.mdx#L189-L201docs/configure-rails/actions/creating-actions.mdx#L248-L263🤖 Prompt for AI Agents