[3/7][multi-lora]: trainer core - per-slot optimizers, per-adapter LR schedules, slot lifecycle, batch routing in get_batch#1744
Conversation
…ml config, shared helpers, CLI flags and validation
…d, control-plane HTTP API, named Ray actor
…schedules, slot lifecycle, batch routing in get_batch
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements support for Multi-LoRA training, introducing a per-slot decoupled Adam optimizer, per-adapter learning rate schedulers, and a control-plane HTTP API managed by a Ray controller to handle adapter lifecycles. It also updates data loading to compute per-adapter token counts and modifies loss scaling to support per-adapter normalization. The review feedback suggests improving code maintainability by moving local imports to the top of the file in model.py to avoid duplication, and using isinstance instead of direct type comparison for more robust type checking in backend.py.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
yushengsu-thu
left a comment
yushengsu-thu
left a comment
There was a problem hiding this comment.
Reviewed extensively on the umbrella PR #1638 (2xH200 E2E validated); layer verified independently (imports + fast suites green on this branch's own tree).
2 participants
#1638