Skip to content

build: generate flatbuffers headers at build time - #7488

Merged
DennisOSRM merged 1 commit into
Project-OSRM:masterfrom
tete17:build/flatbuffers-generate-at-build-time
Apr 27, 2026
Merged

build: generate flatbuffers headers at build time#7488
DennisOSRM merged 1 commit into
Project-OSRM:masterfrom
tete17:build/flatbuffers-generate-at-build-time

Conversation

@tete17

@tete17 tete17 commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Generate flatbuffers C++ headers at CMake build time from the .fbs schemas instead of committing pre-generated headers
  • Remove the flatbuffers version pin (was 25.9.23) from vcpkg.json — any compatible version now works
  • Delete the generated/ directory (−1,800 lines of generated code)

Why

The pre-committed *_generated.h files contain a static_assert checking for an exact flatbuffers version match. This means every flatbuffers upgrade requires manually re-running flatc and committing the results — a maintenance burden and source of version drift. With build-time generation, upgrading flatbuffers is just a vcpkg baseline bump.

How it works

Uses flatbuffers_generate_headers() from flatbuffers' own BuildFlatBuffers.cmake module (shipped by vcpkg). This creates:

  • A custom target GENERATE_osrm_fbs that runs flatc when .fbs schemas change
  • An INTERFACE library osrm_fbs whose include directory points to the generated output
  • Headers land in ${CMAKE_BINARY_DIR}/osrm_fbs/engine/api/flatbuffers/ — same include path as before, no source changes needed

Note: This PR is stacked on #7487 (vcpkg migration). Should be merged after that PR lands.

Test plan

  • CLion build with flatbuffers 25.12.19 (unpinned) — all 5 headers generated, full build succeeds
  • Generated headers contain version 25.12.19 assertions (was 25.9.23), confirming the pin is gone
  • CI green

🤖 Generated with Claude Code

@nilsnolde

Copy link
Copy Markdown
Contributor

is there a fallback in case BuildFlatBuffers.cmake in not available?

@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch 2 times, most recently from c43d945 to a11d0c7 Compare April 20, 2026 09:32
@tete17

tete17 commented Apr 20, 2026

Copy link
Copy Markdown
Contributor Author

Hey @nilsnolde not realy but I wouldn't worry.

Any distribution of flatbuffers includes this cmake file https://packages.debian.org/trixie/amd64/libflatbuffers-dev/filelist#:~:text=/usr/lib/x86_64%2Dlinux%2Dgnu/cmake/flatbuffers/BuildFlatBuffers.cmake

It is required as many projects depend on it to autogenerate the targets based on their definitions. You probablly have it in your system if you install flatbuffers through your package manager

@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch from a11d0c7 to ee7170f Compare April 20, 2026 09:45
@nilsnolde

Copy link
Copy Markdown
Contributor

ah perfect, thanks @tete17 ! yeah, I was mainly worried about debian like distros. hmm, seems bookworm doesn't have it: https://packages.debian.org/bookworm/amd64/libflatbuffers-dev/filelist. it's EOL in a few months. since trixie is just a few weeks old, I'd strongly suggest to introduce a fall back.

@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch from ee7170f to fa8d75d Compare April 20, 2026 10:49
@DennisOSRM

Copy link
Copy Markdown
Collaborator

If this is pulled from vcpkg then there shouldn't be an issue on Debian bookworm, right? Or am I missing something here?

@nilsnolde

Copy link
Copy Markdown
Contributor

IMO it's a bit nicer generally to not require vcpkg installations (always builds from source last I checked) and instead be able to use binary system packages for dependencies (except for win of course). I didn't really have the time to take a deeper look, I asked here if vcpkg is optional or required, but didn't get an answer yet.

@tete17

tete17 commented Apr 20, 2026

Copy link
Copy Markdown
Contributor Author

Hey @nilsnolde sorry for ignoring the questions.

So regarding bookworm I have a strong suspecious, but I can't test it that, the flatbuffers team just restructure the cmake files and are now in a different place. If we wanted to support a debian based system based on bookworm (Ubutnu 22.04 for that matter as well) we would need to introduce an if clause in the cmake to include the right folder.

Then again this would be to support an EOL operating system for native packages when they could just turn on the vcpkg integration and not worry about anything. Yes the initial cmake configure is annoyiong as it would take 1h in a medium system not going to lie but is a worth price to pay to ensure everybody uses the same library versions.

@DennisOSRM

DennisOSRM commented Apr 20, 2026

Copy link
Copy Markdown
Collaborator

I wouldn't worry too much about bookworm personally. It's EOL in this long term support phase in six weeks and since the beginning we had the policy of supporting the latest debian release only.

@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch from fa8d75d to f0d4b4b Compare April 20, 2026 16:39
@nilsnolde

Copy link
Copy Markdown
Contributor

yeah sorry, I read the trixie release wrong. and since vcpkg remains optional (as conan was), there's no real problem anyways.

@DennisOSRM

Copy link
Copy Markdown
Collaborator

Once #7487 is done, this needs a rebase. One idea to make this reviewable independently could be to use the other PR's branch as the base. And then after it's done to change the base branch to master.

@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch from f0d4b4b to 48792c8 Compare April 27, 2026 15:12
Replace pre-committed generated headers with CMake build-time
generation using flatbuffers_generate_headers() from the vcpkg
flatbuffers package. This eliminates the need to pin flatbuffers
to a specific version — any compatible flatc will regenerate the
headers from the .fbs schemas during the build.

- Add flatbuffers_generate_headers() call in CMakeLists.txt
- Delete generated/ directory (pre-committed headers)
- Remove flatbuffers 25.9.23 version override from vcpkg.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tete17
tete17 force-pushed the build/flatbuffers-generate-at-build-time branch from 48792c8 to bd1ad70 Compare April 27, 2026 15:16
@DennisOSRM
DennisOSRM added this pull request to the merge queue Apr 27, 2026
Merged via the queue into Project-OSRM:master with commit 4603631 Apr 27, 2026
26 checks passed
tete17 added a commit to tete17/osrm-backend that referenced this pull request May 4, 2026
The script managed git-subtree updates for vendored dependencies under
third_party/. After Project-OSRM#7487 (vcpkg migration), Project-OSRM#7488 (flatbuffers from
vcpkg), Project-OSRM#7495 (microtar -> libarchive), and Project-OSRM#7508 (vtzero unvendored
via custom vcpkg overlay), third_party/ is empty and every entry the
script knew how to update is now provided by vcpkg. Nothing left to
manage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants