Skip to content

feat(guardrails): add NeMo Provider - #285

Open
christinaexyou wants to merge 2 commits into
praxis-proxy:mainfrom
christinaexyou:add-nemo-provider
Open

feat(guardrails): add NeMo Provider#285
christinaexyou wants to merge 2 commits into
praxis-proxy:mainfrom
christinaexyou:add-nemo-provider

Conversation

@christinaexyou

Copy link
Copy Markdown
Contributor

Refactor of praxis-proxy/praxis#700

Implements the NeMo provider for the ai_guardrails filter, completing request-side evaluation against NeMo Guardrails. Addresses #48.

The following changes were made:
providers/nemo.rs

  • Implements NemoProvider which sends a POST to the configured /v1/guardrail/checks endpoint and maps the NeMo response to a GuardResult

filter.rs

  • on_request_body which checks phase.request (default true), waits for end_of_stream, parses the body, calls extract_messages, invokes the provider, and acts on the verdict:

    Pass → Continue
    Block → Reject 403 with the blocked rail name(s) in the body
    Redact → Continue (body replacement placeholder for ai_guardrails: NeMo mask / redact action #49)

  • extract_messages: parses OpenAI Chat Completions API formats, returns FilterError for unsupported formats

@christinaexyou
christinaexyou requested review from a team and alexsnaps July 6, 2026 17:14
@christinaexyou
christinaexyou requested a review from shaneutt as a code owner July 6, 2026 17:14
@praxis-bot-app

praxis-bot-app Bot commented Jul 6, 2026

Copy link
Copy Markdown

Missing Signed-off-by: 25b82fc. All commits require sign-off (via git commit --signoff).

@praxis-bot-app

praxis-bot-app Bot commented Jul 6, 2026

Copy link
Copy Markdown

Unsigned commits: 3803297. Please sign your commits.

@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 8 times, most recently from 05bb6a1 to 97f4123 Compare July 6, 2026 18:58
Comment thread filters/src/guardrails/filter.rs
Comment thread filters/src/guardrails/providers/nemo.rs Outdated
Comment thread tests/integration/tests/suite/examples/guardrails.rs Outdated
Comment thread filters/Cargo.toml Outdated
Comment thread filters/src/guardrails/providers/mod.rs

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NeMo Provider Review

Solid implementation of the NeMo guardrails provider with correct request-side integration, good error handling structure, and functional integration tests covering all three verdict paths plus provider-down scenarios.

Findings

# Severity File Summary
1 Large nemo.rs Content-Length size check bypassed for chunked responses
2 Large nemo.rs No unit tests for map_nemo_response or blocked_rail_names
3 Large filter.rs No unit tests for extract_messages
4 Large tests.rs Only happy-path on_request_body test; no block/redact/error path tests
5 Medium nemo.rs Short-form separator comment
6 Medium filter.rs Short-form separator comment

Non-inline findings

[Medium] filters/src/guardrails/filter.rs and examples/configs/nemo-guardrails.yaml are both missing POSIX trailing newlines (\ No newline at end of file in the diff). Add a final newline to each.

Comment thread filters/src/guardrails/providers/nemo.rs Outdated
Comment thread filters/src/guardrails/providers/nemo.rs
Comment thread filters/src/guardrails/filter.rs
Comment thread filters/src/guardrails/tests.rs Outdated
Comment thread filters/src/guardrails/providers/nemo.rs Outdated
Comment thread filters/src/guardrails/filter.rs Outdated
@shaneutt shaneutt added this to the v0.1.0 milestone Jul 8, 2026
@shaneutt shaneutt moved this from Next to Review in AI Gateway - Model Serving Jul 8, 2026
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 3 times, most recently from 55c5f0b to 5a1ad8b Compare July 9, 2026 22:45
Comment thread filters/src/guardrails/filter.rs Outdated
Comment thread filters/src/guardrails/providers/nemo.rs
Comment thread filters/src/guardrails/providers/nemo.rs
Comment thread filters/src/guardrails/filter.rs
Comment thread filters/src/guardrails/tests.rs
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 3 times, most recently from 18925e9 to f68a92d Compare July 13, 2026 20:42
@christinaexyou

Copy link
Copy Markdown
Contributor Author

@aslakknutsen thanks so much for the detailed review !

Comment thread examples/configs/nemo-guardrails.yaml
@leseb leseb modified the milestones: v0.1.0, v0.2.0 Jul 23, 2026
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 3 times, most recently from a7d913a to bfaf7cf Compare July 24, 2026 15:09
@shaneutt shaneutt removed their assignment Jul 24, 2026
@aslakknutsen

Copy link
Copy Markdown
Contributor

@christinaexyou You need to sign the commits, besides that it looks good :)

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: NeMo Provider (+692 new lines)

Re-review focusing on new code since the last praxis-bot review. Strong progress -- all prior findings addressed or partially addressed, and the new test coverage is thorough.

Prior finding status

# Finding Status
1 Content-Length bypass for chunked responses Fixed -- incremental streaming body read with 1 MiB cap added (read_response_body)
2 No unit tests for map_nemo_response / blocked_rail_names Partially fixed -- blocked_rail_names has 5 direct tests; map_nemo_response tested indirectly via wiremock but still lacks direct unit tests for absent-content and unknown-status edge cases
3 No unit tests for extract_messages Fixed -- covered by on_request_body tests for invalid JSON, missing messages, non-array, and empty array
4 Only happy-path on_request_body test Fixed -- 12 new test functions covering block, redact, skip, error, and validation paths
5 Short-form separator in nemo.rs Fixed
6 Short-form separator in filter.rs Fixed
7 Missing trailing newlines (non-inline) Partially fixed -- filter.rs fixed; nemo-guardrails.yaml still missing POSIX trailing newline

New findings

# Severity File Summary
1 Medium filter.rs record_verdict doc comment uses // instead of /// on continuation line
2 Medium nemo.rs read_response_body doc comment makes false claim about function behavior

Comment thread filters/src/guardrails/filter.rs Outdated
Comment thread filters/src/guardrails/providers/nemo.rs Outdated
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 2 times, most recently from 5fb1387 to 4932f1e Compare July 27, 2026 22:27
@praxis-proxy praxis-proxy deleted a comment from praxis-bot-app Bot Jul 27, 2026
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 2 times, most recently from b80ca11 to 725049a Compare July 28, 2026 14:22
@leseb

leseb commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

please use this as a reference to replace callout client #599

Signed-off-by: Christina Xu <chrxu@redhat.com>
@christinaexyou
christinaexyou force-pushed the add-nemo-provider branch 2 times, most recently from e14791e to 0714921 Compare July 29, 2026 22:34
@christinaexyou

Copy link
Copy Markdown
Contributor Author

@leseb thanks for your help unblocking my PR ! fyi the NeMoProvider never used CalloutClient directly and requires praxis-core#826 to replace it with SubRequest

Signed-off-by: Christina Xu <chrxu@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Review

Development

Successfully merging this pull request may close these issues.

5 participants