A multi-agent system built with LangGraph and LangChain that processes return and exchange requests end-to-end — checking policy eligibility, screening for fraud, calculating refunds, and drafting a customer-facing resolution — with a human-in-the-loop review step before anything is sent.
This isn't a chatbot. It's a backend decision pipeline: structured input goes in, a policy-grounded decision comes out, a human reviewer signs off (or sends it back for correction), and the final outcome is persisted for future reference.
Most LLM demo projects stop at "call the API, get an answer." This project is about what happens after that — specifically:
- Can the agent's decision be traced back to a real source, not just trusted?
- Can a human correct a wrong decision without the whole pipeline rerunning from scratch?
- Can the system distinguish a deterministic fact (is this size in stock?) from a judgment call (is this return eligible?) and route each to the right kind of logic?
Those three questions shaped almost every design decision below.
Request received
(order_id, request_type,
item_condition, return_reason,
exchange_target_size?)
│
▼
Conversation Lookup
(has this order_id been seen before?)
│
▼
┌───────────────────────────────────────────────┐
│ LangGraph │
│ │
│ entry_router │
│ │ │
│ ▼ │
│ eligibility ──eligible──▶ fraud_detection │
│ │ │ │
│ ineligible clean │ suspicious │
│ │ │ │ │
│ │ ▼ │ │
│ │ refund_calculation │ │
│ │ │ │ │
│ └──────────────┬───────────┴───────┘ │
│ ▼ │
│ resolution │
│ │
│ (entry_router can resume at ANY of the above │
│ four nodes — see "Resumable entry point" below) │
└───────────────────────────────────────────────┘
│
▼
Draft shown to HUMAN REVIEWER
│
┌───────────┴───────────┐
│ │
Approve Reject + feedback
│ │
▼ ▼
Decision persisted to Revision Agent classifies
conversation_store.json which node was wrong, clears
its output + everything
downstream, graph resumes
from exactly that node
│
└──────▶ (back to draft review)
| Agent | Responsibility | Notable design choice |
|---|---|---|
| Eligibility | Is this return/exchange allowed at all? | RAG-grounded — retrieves the actual policy section via ChromaDB rather than reasoning from a hardcoded rule list, and must cite the exact SOURCE_SECTION it relied on |
| Fraud Detection | Does this request look abusive? | Checks both a global "high-return customer" flag and this specific order's prior history via conversation_store.py — repeat disputes on the same order are a stronger signal than a generic frequency flag |
| Refund Calculation | What's owed, and how? | Branches completely for exchanges vs. returns — exchanges check live inventory (a deterministic lookup, no LLM involved) before ever considering a refund; returns use Chain-of-Thought prompting to force step-by-step arithmetic before stating a final amount |
| Resolution | Draft the customer-facing message | The only agent that sees the full picture; handles five distinct message types (fulfilled exchange, store-credit fallback, approved refund, rejection, escalation) |
RAG with verifiable citations, not just retrieval
Every policy-based decision includes a SOURCE_SECTION field naming the exact section of policy.txt the agent relied on. This isn't decoration — it means a hallucinated rule can be caught with a simple text search against the real document, instead of having to trust the model's self-report. The policy document itself is chunked by section header (not fixed character windows) specifically so each citation maps to something a human can actually go read.
A resumable entry point
LangGraph fixes the entry point at compile time, which is a problem when a human reviewer corrects a specific node — restarting the whole pipeline from eligibility every time would waste LLM calls re-deriving answers that were never wrong. entry_router reads revision_history and routes directly to whichever node the reviewer's feedback actually targets, skipping redundant recomputation of already-correct upstream work.
Scoped state invalidation on correction
When a reviewer's feedback targets refund_calculation, only refund-related state fields are cleared — eligibility and fraud_detection's outputs are explicitly preserved. Re-deriving fields that weren't wrong risks the model landing on a different, possibly worse answer on something that was already correct.
Deterministic logic where the answer is a fact, not a judgment Exchange requests check size availability via a plain Python dictionary lookup, not an LLM call. Whether a size is in stock isn't something a language model should be reasoning about — it's a fact with one correct answer. The LLM is reserved for genuinely ambiguous judgment calls (is this return reason consistent with the item's condition?).
Chain-of-Thought specifically where arithmetic reliability matters The refund calculation prompt forces the model to show item price → condition tier → percentage → final arithmetic before stating the answer, rather than jumping straight to a number. Smaller models are meaningfully more reliable at percentage-based math when forced through explicit intermediate steps rather than pattern-matching directly to a plausible-looking figure.
Human-in-the-loop
The human acts before the decision is sent and inside the loop that drives the graph: revision_agent.py classifies their feedback to a specific node, clears that node's output and everything downstream, and entry_router resumes the graph from there — so the reviewer's input genuinely becomes part of state and changes what the agents compute. That's safe to allow because the reviewer's incentive is the opposite of the customer's: they're trying to catch an error, not negotiate a better number for themselves.
return-agent/
├── main.py # Entry point — collects a request, runs the reviewer loop
├── graph.py # LangGraph definition: nodes, edges, resumable entry routing
├── state.py # Shared TypedDict state passed between every node
├── conversation_store.py # Persists finalized decisions per order_id
├── agents/
│ ├── eligibility_agent.py # RAG-grounded eligibility check with cited source
│ ├── fraud_agent.py # Fraud detection, aware of this order's prior history
│ ├── refund_agent.py # CoT refund math (returns) + inventory check (exchanges)
│ ├── resolution_agent.py # Drafts the customer-facing message
│ └── revision_agent.py # Classifies reviewer feedback, scopes state invalidation
├── rag/
│ ├── ingest.py # Chunks policy.txt by section, embeds into ChromaDB
│ └── retriever.py # Retrieves relevant sections for a given query
└── data/
├── mock_db.py # Orders, inventory, high-return-customer flags
└── policy.txt # The actual return/refund/exchange policy document
pip install -r requirements.txtAdd a .env file:
GROQ_API_KEY=gsk_your_key_here
Run:
python main.pyYou'll be prompted for an order ID, request type, item condition, and (for exchanges) a target size. The pipeline runs, a draft is shown, and you act as the reviewer — approve it or describe what's wrong.
| Order ID | Scenario | What to watch for |
|---|---|---|
ORD-1001 |
Return, unused, wrong size | Should approve with a full refund — check the cited SOURCE_SECTION |
ORD-1004 |
Return, any condition | Sale item — should reject regardless of condition |
ORD-1005 |
Return, any reason | Flagged for return frequency — should escalate, not auto-approve |
| Any order | Exchange to an in-stock size | Should confirm a swap with no refund amount |
| Any order | Exchange to an out-of-stock size | Should fall back to store credit, never cash |
| Any approved case | Process the same order again | Fraud agent should reference the prior case from conversation_store.json |
| Any draft | Reject with feedback like "the refund should be 90%, not 100%" | Should resume at refund_calculation only — watch the terminal log confirm it skips re-running eligibility and fraud |
This is a portfolio project, not a production system, and a few things are deliberately out of scope:
- No real database —
mock_db.pyis an in-memory Python dict. The architecture (separating deterministic lookups from LLM judgment calls) is the point being demonstrated, not the persistence layer. - No live data correction path — if a reviewer disagrees with a deterministic fact (e.g. "that size is actually out of stock"), there's currently no mechanism for that feedback to override the inventory lookup, since
_handle_exchange()doesn't readreviewer_feedback. A real fix would mean updating the underlying inventory data, not patching around it conversationally. - No raw email/extraction layer — The layer has been deliberately removed to keep the project's focus on the multi-agent graph and reviewer-correction logic, which is the actual subject being demonstrated.
LangGraph · LangChain · Groq (LLaMA 3.1 8B) · ChromaDB · sentence-transformers · Python