Skip to content

Resolve run paths relative to the loader, not the install prefix - #232

Draft
gdevenyi wants to merge 2 commits into
BIC-MNI:develop-1.9.18from
gdevenyi:relocatable-rpath
Draft

Resolve run paths relative to the loader, not the install prefix#232
gdevenyi wants to merge 2 commits into
BIC-MNI:develop-1.9.18from
gdevenyi:relocatable-rpath

Conversation

@gdevenyi

@gdevenyi gdevenyi commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Why

Every component bakes CMAKE_INSTALL_PREFIX into its install RPATH — CMakeLists.txt:160 for the main tree, and a -DCMAKE_INSTALL_RPATH:PATH=${install_prefix}/lib${LIB_SUFFIX} in each of the 21 Build*.cmake ExternalProjects. Move the install and nothing finds its libraries any more. Together with the prefix baked into minc-toolkit-config.sh, that is what makes the toolkit non-relocatable.

This came out of the review on #224: with LD_LIBRARY_PATH gone, the RPATH is what resolves everything, so it is worth it being robust.

What

RPATH. The absolute prefix becomes $ORIGIN (@loader_path on Apple) in all 22 places, keeping whichever library-directory expression each file already used. The directory referred to is unchanged; only how it is reached differs — a mechanical, semantics-preserving substitution.

IF(APPLE)
  SET(MT_RPATH_ORIGIN "@loader_path")
ELSE()
  SET(MT_RPATH_ORIGIN "$ORIGIN")
ENDIF()
SET(CMAKE_INSTALL_RPATH ${MT_RPATH_ORIGIN}/../lib${LIB_SUFFIX})

One value covers everything: binaries install to <prefix>/bin and libraries to <prefix>/lib, and <origin>/../lib is correct from both — from a library in lib/ it normalises back to its own directory. Being a single entry, it also sidesteps the LIST_SEPARATOR a ;-joined RPATH would need to survive ExternalProject_Add.

Config script. minc-toolkit-config.sh derives MINC_TOOLKIT from its own location — it installs at the top of the prefix, so its directory is the prefix. Bash-only via BASH_SOURCE; every other shell keeps the configure-time path exactly as now.

csh is left alone. Locating a sourced script there is too fragile to be worth it, and the binaries no longer depend on the variable to find their libraries.

Not in scope

Packaging policy. CPACK_SET_DESTDIR, CPACK_PACKAGING_INSTALL_PREFIX and CPACK_PACKAGE_RELOCATABLE=OFF still pin the .deb/.rpm to a fixed prefix. That is a decision about how releases install rather than a consequence of this change — relaxing it is now possible, but should be your call in a separate PR.

Verification

  • Relocation works. Build a shared library plus a binary with this RPATH, install, move the entire prefix, run it: RUNPATH: [$ORIGIN/../lib], and the library still resolves after the move.
  • $ORIGIN survives ExternalProject with the Makefile generator, which was the main risk. CMake emits \$$ORIGIN, make collapses $$ to $, and the shell passes it through literally. Confirmed two ways: the generated ZLIB-cfgcmd.txt contains -DCMAKE_INSTALL_RPATH:PATH=$ORIGIN/../lib, and a sub-project echoing back CMAKE_INSTALL_RPATH receives $ORIGIN/../lib intact. No CMAKE_CACHE_ARGS workaround needed.
  • All 24 Build*.cmake still parse, and build_zlib() instantiates cleanly.
  • Config script: self-locates under bash and follows a relocated prefix; falls back to the baked path under zsh with no error; passes bash --posix -n and shellcheck -s sh.

Not verified: a full superbuild install-and-relocate, which is hours of build time. The mechanism is proven in isolation and the substitution is uniform, but a real install is worth doing before merging — the same ldd check suggested on #224 covers it:

make install
env -u LD_LIBRARY_PATH ldd <prefix>/bin/* 2>/dev/null | grep "not found"

then move the prefix and run it again. Both should be silent.

Relationship to #224

Independent, either order. #224 removes LD_LIBRARY_PATH because the absolute RPATH already covers it; this makes that RPATH survive a move as well. Merging both leaves the toolkit resolving its libraries with no environment variables at all.

🤖 Generated with Claude Code

https://claude.ai/code/session_019katBUnczdvUS7WQ4RpVoa

Every component baked CMAKE_INSTALL_PREFIX into its install RPATH, so a
moved install could no longer find its own libraries. That, plus the prefix
baked into minc-toolkit-config.sh, is what makes the toolkit
non-relocatable today.

Replace the absolute prefix with $ORIGIN (@loader_path on Apple) in all 22
places, keeping whichever library directory each already used, so the
directory referred to is unchanged and only how it is reached differs. One
value serves everything: binaries install to <prefix>/bin, libraries to
<prefix>/lib, and <origin>/../lib is right from both -- from a library it
normalises back to its own directory. Being a single entry it also avoids
the list separator that a ;-joined RPATH would need to survive
ExternalProject.

minc-toolkit-config.sh now derives MINC_TOOLKIT from its own location; it
installs at the top of the prefix, so that is the prefix. This is bash-only
via BASH_SOURCE -- every other shell keeps the configure-time path, as
before. csh is left alone: locating a sourced script there is too fragile
to be worth it, and the binaries no longer depend on the variable anyway.

Packaging policy is deliberately untouched. CPACK_SET_DESTDIR,
CPACK_PACKAGING_INSTALL_PREFIX and CPACK_PACKAGE_RELOCATABLE=OFF still pin
the .deb/.rpm to a fixed prefix, which is a maintainer decision about how
releases install rather than a consequence of this change. Relaxing them is
now possible, but is a separate call.

Verified:
 - build, install, move the whole prefix, run: the binary still resolves
   its library through $ORIGIN/../lib;
 - $ORIGIN survives ExternalProject with the Makefile generator -- CMake
   emits \$$ORIGIN, so make collapses it and the shell passes it through
   literally, confirmed both in the generated ZLIB-cfgcmd.txt and by a
   sub-project echoing back what it received;
 - all 24 Build*.cmake still parse, and build_zlib() instantiates;
 - the config script self-locates under bash and follows a relocated
   prefix, falls back cleanly under zsh, passes bash --posix -n and
   shellcheck -s sh.

Not verified: a full superbuild install and relocation, which is hours of
build time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019katBUnczdvUS7WQ4RpVoa
Copilot AI lite review requested due to automatic review settings August 6, 2026 19:26
@gdevenyi
gdevenyi marked this pull request as draft August 6, 2026 19:27
@gdevenyi

gdevenyi commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

The idea here is that our cloud builds will have tarballs available which can be unpacked into any directory. This will facilitate installations in module systems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates MINC toolkit installation to be more relocatable by deriving the prefix from the config script location and by switching installed RPATHs to be origin-relative.

Changes:

  • Compute MINC_TOOLKIT from the config script’s directory when running under bash.
  • Introduce MT_RPATH_ORIGIN and use origin-relative CMAKE_INSTALL_RPATH in the top-level build.
  • Propagate the origin-relative install RPATH into multiple ExternalProject builds.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
minc-toolkit-config.unix.sh.cmake Derives MINC_TOOLKIT from the script location under bash for relocatability.
minc-toolkit-config.apple.sh.cmake Same relocatable MINC_TOOLKIT logic for macOS config script.
CMakeLists.txt Defines MT_RPATH_ORIGIN and sets an origin-relative CMAKE_INSTALL_RPATH.
cmake-modules/BuildZLIB.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildPatchMorphology.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildOpenJPEG.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildOpenBLAS.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildNIFTI.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildNETCDF.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildLibarchive.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildJPEG.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildITKv4.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildHDF5.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildGIFTI.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildFREEGLUT.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildFFTW3F.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildFFTW3D.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildElastix.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildEZMINC.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildC3D.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildBICGL.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildBEAST.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildANTS.cmake Switches ExternalProject install RPATH to origin-relative.
cmake-modules/BuildABC.cmake Switches ExternalProject install RPATH to origin-relative.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +6 to +7
if [ -n "${BASH_SOURCE:-}" ] && [ -f "${BASH_SOURCE}" ]; then
MINC_TOOLKIT=$(cd -- "$(dirname -- "${BASH_SOURCE}")" && pwd -P)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this against bash 5.3.15, and the failure mode does not occur — cd+pwd -P already normalises both cases.

The concern needs BASH_SOURCE to be a bare basename while the script is not in the current directory. Bash only produces a bare name when it found the file in the cwd, in which case the cwd is the right answer:

how it is sourced BASH_SOURCE holds resolved MINC_TOOLKIT
explicit relative path ../onpath/x.sh .../onpath
bare name, found via PATH .../elsewhere/../onpath/x.sh (full) .../onpath
bare name, found in cwd x.sh (bare) cwd, which is where the script lives ✅

So the PATH case keeps the full path rather than the basename, and the bare-name case only happens when dirname . is already correct. Both come out right after cd -- "$(dirname -- ...)" && pwd -P.

Happy to add a command -v / realpath normalisation as belt-and-braces if you would rather not depend on that behaviour, but on the evidence it is not currently reachable.

Comment thread cmake-modules/BuildZLIB.cmake Outdated
-DCMAKE_SKIP_INSTALL_RPATH:BOOL=OFF
-DMACOSX_RPATH:BOOL=ON
-DCMAKE_INSTALL_RPATH:PATH=${install_prefix}/lib${LIB_SUFFIX}
-DCMAKE_INSTALL_RPATH:PATH=${MT_RPATH_ORIGIN}/../lib${LIB_SUFFIX}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, changed in the follow-up commit — all 21 sites now use -DCMAKE_INSTALL_RPATH:STRING=.

Worth noting the :PATH spelling predates this branch (I only changed the value, not the type) and it does work: the full build linked below produced correct run paths with it, and the generated ZLIB-cfgcmd.txt carried $ORIGIN/../lib through untouched. But STRING is what a list of run paths actually is, and it removes any question about a PATH-typed entry normalising $ORIGIN / @loader_path.

@gdevenyi

gdevenyi commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Verified locally with a real build, install and relocation. The "not verified" caveat in the description is now covered.

Configuration — maximum system dependencies (12 of 13 USE_SYSTEM_* on: zlib, netcdf, hdf5 2.1.1, gsl, fftw3f/3d, jpeg, openjpeg, libarchive, glut, expat-for-ITK, png), ITK 4.14 built from source, MT_BUILD_SHARED_LIBS=ON, GCC 16.1.1. Built, installed to a prefix, then moved the entire prefix and retried everything.

1. No absolute RPATH survives.

$ORIGIN-relative: 312    absolute: 0    no rpath: 88

The 88 without an RPATH are all scripts — 90 Perl, 7 shell — not ELF objects. Every actual binary and shared library got a relative run path. CMake's install step logs it explicitly:

-- Set non-toolchain portion of runtime path of ".../bin/grid_proc" to "$ORIGIN/../lib"

2. Nothing unresolved with LD_LIBRARY_PATH scrubbed, before or after the move.

3. Relocation works, including the ITK-heavy case. 118 ITK shared libraries install into the prefix. After moving the whole prefix:

$ readelf -d .../mt-reloc-prefixB/bin/c_fit_harmonics_grids | grep RUNPATH
  RUNPATH  [$ORIGIN/../lib]
$ env -u LD_LIBRARY_PATH ldd .../bin/c_fit_harmonics_grids
  resolved from moved prefix: 85    not found: 0
  libITKDICOMParser-4.14.so.1 => .../mt-reloc-prefixB/bin/../lib/libITKDICOMParser-4.14.so.1

85 libraries resolving through one $ORIGIN/../lib entry also settles the RUNPATH-is-not-transitive question raised on #224: each ITK library carries its own, so the chain resolves without help. c_fit_harmonics_grids, itk_resample and itk_morph all execute from the moved prefix with no environment variables set.

4. The config script follows the move. Sourced from the relocated prefix, MINC_TOOLKIT resolves to the new location rather than the configure-time path.

5. Test suite: 150/150 pass on the relocatable build, nu_reference_1 included.

One incidental finding, unrelated to this PR: the bundled NetCDF does not compile under GCC 16 — ncdump/utils.h:21:7: error: cannot use keyword 'false' as enumeration constant, C23 making false a keyword. USE_SYSTEM_NETCDF=ON avoids it, which is what CI does, so nothing here is affected. Worth knowing if anyone builds the bundled copy on a current toolchain.

@vfonov

vfonov commented Aug 6, 2026

Copy link
Copy Markdown
Member

only subprojects install into external during the build.

CMAKE_INSTALL_RPATH is a list of run paths, not a filesystem path, and
$ORIGIN / @loader_path are not paths a PATH-typed cache entry should be
invited to normalise. The :PATH spelling predates this branch and does work
-- the full build below produced correct run paths with it -- but STRING is
what the value actually is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019katBUnczdvUS7WQ4RpVoa
@gdevenyi

gdevenyi commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@vfonov correct, and following that up found a real gap — plus a flaw in my own verification.

On external/: yes, only the subprojects install there during the build; the main-tree components install straight into the prefix at make install. My check was run against the final prefix after make install, so it covered both routes, and both come out loader-relative:

route run path
mincinfo (minctools) main tree, direct $ORIGIN/../lib
libbicpl.so (bicpl) main tree, direct $ORIGIN/../lib
libITKCommon-4.14.so.1 subproject via external/ $ORIGIN/../lib
itk_resample (EZMINC) subproject via external/ $ORIGIN/../lib
mincbeast (BEAST) subproject via external/ $ORIGIN/../lib

Correction to my earlier numbers. I reported absolute: 0. That was wrong — my scan capped the file list at 400 entries and silently truncated. Rerun without the cap over the whole installed tree:

relative: 418   ABSOLUTE: 1   none: 138 (all scripts)
  libminc2.so.5.3.0 -> /.../mt-reloc-prefixA/lib

One object still had an absolute run path: libminc2.so. The cause is in libminc, not here — libminc/CMakeLists.txt sets a target-scoped INSTALL_RPATH to ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}, which overrides this PR's CMAKE_INSTALL_RPATH. Fixed in BIC-MNI/libminc#153.

Everything else in the earlier report stands — the relocation test, the 85 ITK libraries resolving from the moved prefix, binaries executing with no environment set, 150/150 tests. It passed despite this gap because a build with system HDF5, NetCDF and zlib leaves libminc2 with no dependency inside the prefix, so the dangling path costs nothing. With those bundled it would have shown up.

So this PR needs BIC-MNI/libminc#153 alongside it to actually deliver a relocatable install.

vfonov pushed a commit to BIC-MNI/libminc that referenced this pull request Aug 7, 2026
The target-scoped INSTALL_RPATH baked in CMAKE_INSTALL_PREFIX, so libminc2
kept pointing at the configure-time prefix. Move the install and that path
dangles: the library then has to be found some other way, by
LD_LIBRARY_PATH or by the RPATH of whatever loaded it.

libminc and the libraries it links install to the same directory, so $ORIGIN
(@loader_path on Apple) names it without naming the prefix, and survives a
move.

Found while verifying the superbuild's equivalent change
(BIC-MNI/minc-toolkit-v2#232): after that, libminc2.so.5.3.0 was the single
object in a full install of 419 binaries and libraries still carrying an
absolute run path, because this target property overrides the superbuild's
CMAKE_INSTALL_RPATH.

It went unnoticed because a build using system HDF5, NetCDF and zlib leaves
libminc2 with no dependency inside the prefix at all, so the dangling path
costs nothing. With those bundled it would matter.

Verified: standalone shared build installs with RUNPATH [$ORIGIN].

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019katBUnczdvUS7WQ4RpVoa
@gdevenyi

gdevenyi commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Closing and reopening to trigger CI — this PR never got a workflow run (zero runs recorded for the branch, while #230 and #224 from the same fork did run).

@gdevenyi gdevenyi closed this Aug 9, 2026
@gdevenyi gdevenyi reopened this Aug 9, 2026
@gdevenyi

gdevenyi commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

CI has now run for the first time on this branch — it had never triggered before (zero runs recorded, while #230 and #224 from the same fork did run), so I closed and reopened to fire it.

Result matches the base branch exactly, with no regression from the RPATH change:

job result
ubuntu (full / minimal)
fedora (full / minimal)
macos (full / minimal) nu_reference_1, 150/151

The macOS failure is the pre-existing N3 numerical one, byte-identical to what develop-1.9.18 produces on its own:

relative RMS difference: 0.000850981867212377 larger then 0.0001

Relevant to this PR specifically: no dynamic-loader errors on macOS. Nothing matching dyld, Library not loaded or image not found anywhere in the log, and 150 of 151 tests ran and passed — so the @loader_path side of the change links and runs correctly on Apple, which is the half I could not verify locally on Linux.

The only @rpath mentions are pre-existing linker warnings about the HDF5 dylib being built for a newer macOS than the deployment target, unrelated to this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants