Repository navigation
RFC: Nexus mesh inference in Lemonade (PR #3693) #3785
Geramy
started this conversation in
Request for Comment (RFC)
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Proposal size
Working group (a large scope increase spanning multiple release cycles)
Updates
No response
User story
I run Lemonade on a desktop at home with a large GPU, and I also use Lemonade on a laptop that travels with me and my phone. Today the laptop can only use the desktop's models if I forward a port, set up a public tunnel, or install and manage a separate VPN. All of those put the desktop's server on a network with only an API key protecting it and my other existing ports are now open and in a IP Tunnel.
With this change, I turn on Devices & mesh in Lemonade's settings on both machines and unlock each one with a FIDO2 security key. Then I invite the laptop into the desktop's device group with an 8-digit code. From then on, the chat header on the laptop has an Inference server picker. I choose the desktop, and chat, model lists, and health checks all come from the desktop over an encrypted WireGuard connection. No port forwarding, no admin rights, and no extra software. If the desktop is unreachable, the laptop stays on its current server instead of breaking.
As an owner of the group, I can see every device's name, mesh address, online state, and latency. I can invite new devices and remove ones I no longer trust from the group.
High-level design
Each Lemonade install becomes a device in a private group. The mesh decides which devices can reach each other, and each server's existing API key still decides who can use it.
For users
How it works
127.0.0.1that forwards to that peer. The app then talks to that port like any other Lemonade server.lemonade-nexus.ioby default) issues security-key challenges, invites, and group membership. It never carries inference traffic.flowchart TB subgraph local["This device"] app["Lemonade desktop app<br/>Devices and mesh panel, Inference server picker"] ctl["Local-only device controls<br/>unlock, invite, join, remove"] port["Private loopback port<br/>one per chosen peer"] mesh["Built-in Nexus client<br/>userspace WireGuard"] app -- device control --> ctl app -- chat and models --> port ctl --> mesh port --> mesh end subgraph peer["Peer device in the group"] pmesh["Built-in Nexus client"] pserver["Peer's Lemonade server<br/>checks its own API key"] pmesh --> pserver end mesh <== encrypted WireGuard socket ==> pmesh fed["Nexus federation<br/>key challenges, invites, membership"] mesh <--> fed pmesh <--> fedFor developers and packagers
Breaking changes
None for existing users. Nothing changes until someone enables Devices & mesh, existing server routes and connection settings are untouched, and mesh ports are never saved as connection settings.
For developers and packagers, the default build changes:
cargo,rustc,libsodium-dev, andlibspdlog-devon Debian, pluslibudev-devfor Linux desktop builds.127.0.0.1at startup, even when Nexus is off for now.Maintenance plan
Most of the feature is protected by CI, but three parts need people.
Covered by CI
Not yet in CI
cargo test. No workflow runs them today.Needs people
Risks
lemonade-nexus.ioAll reactions