[router] support manual policy (sticky + min_load) and set as default agentic routing policy#1690
[router] support manual policy (sticky + min_load) and set as default agentic routing policy#1690yueming-yuan wants to merge 2 commits into
Conversation
… under manual too
yueming-yuan
requested review from
Shi-Dong,
fzyzcjy and
guapisolo
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the routing logic in 'miles/rollout/sglang_rollout.py' to support the 'manual' router policy in addition to 'consistent_hashing' when using session IDs. The feedback suggests updating the outdated inline comments on lines 195 and 316 to accurately reflect this new behavior and avoid confusion.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # Use session_id for consistent hashing routing if router uses consistent_hashing policy | ||
| headers = None | ||
| if args.sglang_router_policy == "consistent_hashing" and sample.session_id: | ||
| if args.sglang_router_policy in ("consistent_hashing", "manual") and sample.session_id: |
There was a problem hiding this comment.
The comment on line 195 still specifically mentions the consistent_hashing policy. Since support for the manual policy has been added, please update the comment to reflect this change (e.g., by generalizing it to session-based routing).
|
|
||
| # Generate a unique session_id for each sample in the group (consistent hashing only) | ||
| if args.sglang_router_policy == "consistent_hashing": | ||
| if args.sglang_router_policy in ("consistent_hashing", "manual"): |
There was a problem hiding this comment.
The comment on line 316 still states (consistent hashing only). Since the manual policy is now also supported, please update this comment to prevent misleading future readers.
yueming-yuan
requested review from
Zhichenzzz,
maocheng23 and
yushengsu-thu
as code owners
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
8f45ecb to
bb5e2b3
Compare
yueming-yuan
changed the title
1 participant
before:
after: