Skip to content

Support running multiple dev instances on offset ports - #1495

Open
Fabian-Fynn wants to merge 5 commits into
mainfrom
feat/parallel-dev-ports
Open

Support running multiple dev instances on offset ports#1495
Fabian-Fynn wants to merge 5 commits into
mainfrom
feat/parallel-dev-ports

Conversation

@Fabian-Fynn

@Fabian-Fynn Fabian-Fynn commented Jun 8, 2026

Copy link
Copy Markdown

Lets several checkouts run npm run dev in parallel without host port collisions — both Docker containers and Node services pick up offset ports.

Changes

  • assign-ports.sh — reads every *_PORT from .env, finds a free +100..+900 offset, writes base+offset to .env.local (leaves .env untouched)
  • package.json — wrap dev-pm start in dotenv -c secrets so the cascaded env (incl. .env.local) is exported; docker compose prefers those real env vars over its .env file, so containers bind the offset host ports
  • dev-oidc-provider.config.mts — honor IDP_PORT instead of the hardcoded 8080
  • AGENTS.md — document the workflow

Usage
./assign-ports.sh # picks a free offset → .env.local
npm run dev

@Fabian-Fynn Fabian-Fynn changed the title Feat: parallel dev ports Support running multiple dev instances on offset ports Jun 8, 2026
@nsams

nsams commented Jun 9, 2026

Copy link
Copy Markdown
Member
  • I would prefer typescript instead of bash for non-trivial code.
  • IDP_PORT fix should be in it's own PR
  • usually we do demo first

@johnnyomair

Copy link
Copy Markdown
Contributor

IDP_PORT fix should be in it's own PR

Done for Demo here: vivid-planet/dextinity#5852. Will use this to try @manuelblum's needs-starter-pr-routine.

@johnnyomair johnnyomair left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update this PR to your updated version once that's merged. Thanks!

@Fabian-Fynn
Fabian-Fynn force-pushed the feat/parallel-dev-ports branch 2 times, most recently from a8cdd34 to e296bfd Compare June 17, 2026 11:43
@Fabian-Fynn
Fabian-Fynn requested a review from johnnyomair June 17, 2026 11:46
@Fabian-Fynn

Copy link
Copy Markdown
Author

The script is up to date.
I removed the IDP_PORT fix since it's fixed in main

VPS-Fabi and others added 2 commits June 17, 2026 14:04
Add assign-ports.sh and wire offset ports through the full dev chain so
several checkouts can run `npm run dev` in parallel without host port
collisions (Docker containers and Node services alike).

- assign-ports.sh: reads every *_PORT from .env, finds a free
  +100..+900 offset, and writes base+offset into .env.local
- package.json: wrap `dev-pm start` in `dotenv -c secrets` so the
  cascaded env (incl. .env.local) is exported into the environment that
  `docker compose up` inherits; Compose gives those real env vars
  precedence over the .env file, so containers bind offset host ports

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Fabian-Fynn
Fabian-Fynn force-pushed the feat/parallel-dev-ports branch from e296bfd to c934f92 Compare June 17, 2026 12:04
@johnnyomair
johnnyomair requested a review from nsams June 18, 2026 12:55
Comment thread package.json
"scripts": {
"create-site-configs-env": "dotenv -e .env.secrets -e .env.local -e .env -- npx @comet/cli inject-site-configs -f site-configs/site-configs.ts -i .env.site-configs.tpl -o .env.site-configs --base64",
"dev": "npm run create-site-configs-env && dev-pm start",
"dev": "npm run create-site-configs-env && dotenv -c secrets -- dev-pm start",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary for Docker to pick up the overridden ports. However, this sets the variables for all scripts in dev-pm config. I'd prefer only setting it for Docker. Maybe we need a dev:docker script? Also, -c secrets isn't necessary, simply dotenv should be enough.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't call dotenv here. Instead, change the docker script in the dev-pm config to set -a; . .env; . .env.local; set +a; docker compose up (we did this in Demo and projects).

Comment thread assign-ports.sh Outdated
@johnnyomair

Copy link
Copy Markdown
Contributor

@nsams once suggested the following alternative:

We could use a single PORT_OFFSET variable that's used all the time:

# .env
PORT_OFFSET=0

API_PORT=4$PORT_OFFSET00
ADMIN_PORT=8$PORT_OFFSET00

Your script would then only write the determined PORT_OFFSET variable to .env.local.

I like this idea. What do you think about it?

@VPS-Fabi

VPS-Fabi commented Aug 3, 2026

Copy link
Copy Markdown

@copilot resolve the merge conflicts in this pull request

@VPS-Fabi

VPS-Fabi commented Aug 3, 2026

Copy link
Copy Markdown

@nsams once suggested the following alternative:

We could use a single PORT_OFFSET variable that's used all the time:

# .env
PORT_OFFSET=0

API_PORT=4$PORT_OFFSET00
ADMIN_PORT=8$PORT_OFFSET00

Your script would then only write the determined PORT_OFFSET variable to .env.local.

I like this idea. What do you think about it?

Good idea

Co-authored-by: VPS-Fabi <296110786+VPS-Fabi@users.noreply.github.com>

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved by merging origin/main into this branch and fixing the package.json conflict in commit dfea63b8.

Copilot AI requested a review from VPS-Fabi August 3, 2026 06:52
VPS-Fabi and others added 2 commits August 3, 2026 09:56
Replaces the bash script with a Node script run via native TS support
(node assign-ports.mts, exposed as npm run assign-ports). Removes the
lsof dependency by probing ports with bind attempts on 0.0.0.0,
127.0.0.1, and ::1 — Node sets SO_REUSEADDR, so a wildcard-only probe
would miss localhost-bound listeners like Vite's dev server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All *_PORT values in .env now embed an offset digit via ${PORT_OFFSET}
(default 0), so assign-ports.mts only determines a free digit (0-9) and
writes PORT_OFFSET=<digit> to .env.local instead of rewriting every
port. The dotenv cascade expands .env.local's PORT_OFFSET inside the
.env port values.

Four ports had no zero digit to hold the offset and get new base
values: postgres 5432 -> 5032, jaeger UI 16686 -> 16086, jaeger OLTP
4318 -> 4018, valkey 6379 -> 6079.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@VPS-Fabi
VPS-Fabi requested a review from johnnyomair August 3, 2026 08:44
@VPS-Fabi

VPS-Fabi commented Aug 3, 2026

Copy link
Copy Markdown

@nsams, @johnnyomair
its now a Typescript script. I also changed it to the single PORT_OFFSET which required the port changes.

@nsams

nsams commented Aug 3, 2026

Copy link
Copy Markdown
Member

When it is a single variable in .env.local is the Script still needed?

@VPS-Fabi

VPS-Fabi commented Aug 3, 2026

Copy link
Copy Markdown

When it is a single variable in .env.local is the Script still needed?

It's not needed necessarily. You can set the ports manually. In my opinion it's still useful for finding a free port offset automatically.

@VPS-Obi
VPS-Obi removed the request for review from thomasdax98 August 4, 2026 08:12
@VPS-Obi
VPS-Obi requested review from VPS-Obi and removed request for johnnyomair August 4, 2026 08:12
Comment thread assign-ports.mts
Comment on lines +39 to +52
// Node sets SO_REUSEADDR, so a wildcard bind coexists with specific-address binds (and vice versa).
// Probe all three addresses to also catch localhost-only listeners like Vite's dev server.
async function portIsFree(port: number): Promise<boolean> {
for (const host of ["0.0.0.0", "127.0.0.1", "::1"]) {
const free = await new Promise<boolean>((resolve) => {
const server = createServer();
server.once("error", (error: NodeJS.ErrnoException) => resolve(error.code !== "EADDRINUSE"));
server.once("listening", () => server.close(() => resolve(true)));
server.listen(port, host);
});
if (!free) return false;
}
return true;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use the get-port package for this, but the custom implementation is fine as well.

Comment thread package.json
"scripts": {
"create-site-configs-env": "dotenv -e .env.secrets -e .env.local -e .env -- npx @comet/cli inject-site-configs -f site-configs/site-configs.ts -i .env.site-configs.tpl -o .env.site-configs --base64",
"dev": "npm run create-site-configs-env && dev-pm start",
"dev": "npm run create-site-configs-env && dotenv -c secrets -- dev-pm start",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't call dotenv here. Instead, change the docker script in the dev-pm config to set -a; . .env; . .env.local; set +a; docker compose up (we did this in Demo and projects).

@VPS-Obi

VPS-Obi commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

As an alternative to the script, we could instruct agents to find the offset themselves, for instance with https://github.com/sindresorhus/get-port-cli. However, I'm don't not sure how reliable this will work.

@VPS-Fabi

VPS-Fabi commented Aug 5, 2026

Copy link
Copy Markdown

As an alternative to the script, we could instruct agents to find the offset themselves, for instance with https://github.com/sindresorhus/get-port-cli. However, I'm don't not sure how reliable this will work.

I don't see a benefit in using an agent when a script can do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants