fix(entrypoint): pipe *.sql into isql to avoid 1-byte read amplification#41
Merged
fdcastel merged 2 commits intoFirebirdSQL:masterfrom May 1, 2026
Merged
Conversation
…ification Reading the init.d schema via `process_sql < "$f"` issues one read syscall per byte against a regular file FD, because isql does not set up stdio buffering on stdin. On native disk that is a ~25% cost on init-time; on layered or remote filesystems (Docker Desktop bind mounts, gRPC FUSE, virtiofs, NFS) the per-syscall overhead amplifies into 10x+ slowdowns. Restoring the v1 `cat "$f" | process_sql` form moves those byte-reads through a kernel pipe (in-memory, lock-free) and matches the existing shape of the compressed cases (`*.sql.gz`, `*.sql.xz`, `*.sql.zst`). Verified against the example.fdb repro from issue FirebirdSQL#40: v1 image and v2 image with this fix both complete the init.d phase in ~0.18s; v2 without the fix is ~0.23s on native disk. Records the rationale as D-016. Closes FirebirdSQL#40.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #40. Restores the v1-style
cat "$f" | process_sqlfor plain SQL files ininit_db(), in place of the v2 redirect formprocess_sql < "$f".Root cause
isqlreads stdin one byte perread()syscall. The cause isn't a missing stdio buffer —isqlis statically linked against the bundlededitlinefromextern/editline/, and itsread_char()calls rawread(fd, _, 1)per character, bypassing stdio entirely (so e.g.setvbuf(stdin, …)would not help). Withcat | isql, those byte-reads come from a kernel pipe (in-memory, lock-free). Withisql < file, every byte-read goes through the regular-file path (i_rwsem, atime, FS layer).strace -cagainst the example.fdb repro from #40 (~110 KB schema): isql issues ~95 600 single-byte reads on FD 0 in both modes. The number of syscalls is identical; the per-call latency is what differs.Why the magnitude is host-dependent
On native disk (no FS layering), pipe vs regular-file
read()differ by tenths of a microsecond, so the total init delta is small (~50 ms locally, ~25 % of init time). But on layered or remote filesystems — Docker Desktop bind mounts on macOS/Windows (gRPC FUSE / virtiofs), NFS, sshfs, FUSE-overlay — everyread()round-trips through a userland helper and adds tens to hundreds of microseconds. Multiplied by ~95 600 syscalls, that turns into seconds.Demonstrated by injecting per-
read()latency viaLD_PRELOAD(regular-file FDs only, mimicking what FUSE/virtiofs add over the kernel-pipe path):cat | isqlisql < filecat | isqlbarely moves becausecatreads the file in ~10 large chunks, amortising the FS overhead.isql < filescales linearly with per-syscall cost. The 25–50 μs row is the realistic regime for Docker Desktop bind mounts and matches the 8–9 s reported in #40.Verification
Ran the user's exact repro from #40 (
compose.yaml+01-schema.sql) on native ZFS:Native-disk fix is small (~50 ms) — expected. The win comes on the FS shapes that amplify per-syscall cost; the synthetic-latency table above shows the fix collapses 8–15 s back to sub-second.
Why pipe (not
isql -i)Three options were tested at 0 μs delay:
cat "$f" | process_sql— ~0.175 s (this PR)process_sql < "$f"— ~0.223 s (current master, regressed)process_sql -i "$f"— ~0.166 s (slightly faster still)isql -iis marginally faster because itfopen()s the file itself and gets default stdio buffering, but switching to it changesprocess_sql's call shape (would no longer be uniform with the compressed cases) and would require touching the function signature. The pipe form is the minimal, surgical revert and matches the existing shape of*.sql.gz/*.sql.xz/*.sql.zst(which already use a decompressor pipeline).Changes
src/entrypoint.sh: one-line revert ininit_db()for the*.sqlcase.generated/*/*/entrypoint.sh: regenerated (byte-identical tosrc/entrypoint.sh).DECISIONS.md: new D-016 recording the rationale.Test plan
example-issue-40.zip) reproduces locally on master and is fixed by this branch (native-disk magnitude).@erikv-hill-labson master, and fixes it on this branch.generated/*/*/entrypoint.share in sync withsrc/entrypoint.sh.init_dbvia/docker-entrypoint-initdb.d/).