rmpkg(main/binutils-is-llvm): make LLVM binutils default in Termux

[robertkirkman]

• My idea of a different solution to:

  • [Bug]: clang: ld: command not found #8191
  • Add back binutils to build-essential #9715
  • binutils and binutils-cross are also dependencies for Rust, but currently not installed automatically. #16588

• Reverts:

  • new package: binutils-is-llvm #10012
  • binutils: Overhaul #13045

• Make Termux behave more similarly to the LLVM-based desktop Linux distro, Chimera Linux

• Preserve as name for GNU as, since llvm-as is not compatible with software expecting GNU as, and Chimera Linux also gives that name to GNU as: binutils-is-llvm: remove symlink from llvm-as to as #11218

• Source code and runtime behavior of Chimera Linux observed to design this PR https://chimera-linux.org/

Binutil	GNU/Linux	Chimera Linux	FreeBSD	MacOS	Vanilla Android	Termux Current	Termux This PR
as	GNU	GNU	GNU	clang -cc1as	llvm- name only	GNU	GNU
elfedit	GNU	GNU	GNU	Not Available	Not Available	GNU	GNU
gprof	GNU	GNU	BSD	Not Available	Not Available	GNU	GNU
addr2line	GNU	LLVM	elftoolchain	Not Available	llvm- name only	Variable	LLVM
ar	GNU	LLVM	BSD	BSD	llvm- name only	Variable	LLVM
c++filt	GNU	LLVM	LLVM	LLVM	llvm- name only	Variable	LLVM
ld	GNU	LLVM	LLVM	Apple	LLVM	Variable	LLVM
nm	GNU	LLVM	elftoolchain	LLVM	llvm- name only	Variable	LLVM
objcopy	GNU	LLVM	elftoolchain	Not Available	llvm- name only	Variable	LLVM
objdump	GNU	LLVM	LLVM	LLVM	llvm- name only	Variable	LLVM
ranlib	GNU	LLVM	BSD	Apple	llvm- name only	Variable	LLVM
readelf	GNU	LLVM	elftoolchain	Not Available	llvm- name only	Variable	LLVM
size	GNU	LLVM	elftoolchain	Apple	llvm- name only	Variable	LLVM
strings	GNU	LLVM	elftoolchain	Apple	llvm- name only	Variable	LLVM
strip	GNU	LLVM	elftoolchain	Apple	llvm- name only	Variable	LLVM

• This is not a project to move Termux away from GNU in general. It is a project to make Termux have a consistent default implementation of a component that is among the group of components for which most distros pick a default and stick to it:

Component	GNU/Linux	Alpine Linux	Chimera Linux	FreeBSD	MacOS	Vanilla Android	Termux Current	Termux This PR
C stdlib	GNU	musl	musl	BSD	BSD	bionic	bionic	bionic
C++ stdlib	GNU	GNU	LLVM	LLVM	LLVM	LLVM	LLVM	LLVM
Compiler	GNU	GNU	LLVM	LLVM	LLVM	LLVM	LLVM	LLVM
Binutils	GNU	GNU	LLVM	LLVM	LLVM	LLVM	Variable	LLVM
Make	GNU	GNU	GNU	BSD	BSD	GNU	GNU	GNU
Coreutils	GNU	busybox	BSD	BSD	BSD	toybox	GNU	GNU

most LLVM-based distros and operating systems choose the LLVM binutils, so choosing the LLVM binutils for Termux is reasonable and will contribute to the consistency of the wider LLVM-based distro ecosystem.

Why is this necessary?

• After bump(main/binutils): 2.46.0 #28438 , it is now impossible to have binutils-is-llvm and ldd, binutils-bin or lua-language-server installed simultaneously.
  • This disturbance to the prior status quo might inconvenience users of ldd, lua-language-server, binutils-bin or similar packages who are trying to build Termux packages on-device.
  • This is considered to have occurred due to the difficulty of maintaining multiple packages that provide binutils simultaneously, so the goal is now to reevaluate the default binutils executables provided in Termux, and make changes that solve the problem without reverting bump(main/binutils): 2.46.0 #28438 while aligning Termux more closely with the current choices used by an LLVM-based desktop Linux distro that also provides a GNU binutils package, Chimera Linux.
• Making the default binutils LLVM will give users and developers of 3rd party software in Termux an environment that is more consistently closer to the build environment required by official Termux packages, without them needing to figure out on their own that binutils-is-llvm was recommended for installation during on-device builds, which will reduce confusion and make developing packages easier in the event that an incompatibility with LLVM binutils does occur that needs to be fixed.
• It will save 20 MB of space (the size of GNU binutils) for anyone who realized they needed a binutils package in addition to clang, but installed binutils rather than binutils-is-llvm without realizing that binutils-is-llvm probably would work for what they need.

Breaking change example

After this, it will unfortunately be more difficult to compile certain software in Termux.

Here is one example that I know of:

git clone https://github.com/paullouisageneau/libjuice.git
cd libjuice
make

It would work before this PR if binutils was installed, but after this PR, it will show this error:

ar: error: unknown option f

(yes, this error also happens when building libjuice in Chimera Linux)

Before this PR, it could be worked around by using pkg install binutils,

but after this PR, it will only be possible to work around by either changing the invocation of ar to gar or removing the argument f from the instance of call to ar in Makefile.

However:

• it is already more difficult to compile software in Termux that has problems with Clang but not GCC, or problems with libc++ but not libstdc++, or problems with alternative libc but not glibc, and the additional limitation imposed by defaulting to LLVM binutils is expected to be relatively minimal

• libjuice is not an official Termux package, and if it were, it would have already had to be patched to avoid this error anyway, because Termux currently has ar set to llvm-ar for building in multiple places:

termux-packages/scripts/build/toolchain/termux_setup_toolchain_29.sh

Line 11 in eaad7e0

export AR=llvm-ar

termux-packages/scripts/setup-termux.sh

Line 14 in eaad7e0

PACKAGES+=" binutils-is-llvm" # Used for checking symbols.

• This will make the default behavior of building software in Termux more consistently similar not only to the behavior of the vanilla Android NDK, and not only to behavior of the termux_setup_toolchain_29() and setup-termux.sh of termux-packages, but also to the behavior of the LLVM-based desktop Linux distro, Chimera Linux.

• Any time clang is installed, all of the LLVM binutils in the llvm package are also already installed, just with llvm- prepended to their names, so for most situations, the binutils package is implementing duplicate functionality while taking up additional space

[robertkirkman]

This change would take way too long to build in a single PR, so the idea is to merge the change first, then rebuild all affected packages afterward one by one in separate PRs.

It will not be possible to recompile ldc because it is broken in multiple ways. The ldc package needs someone who is good at both Dlang and Android to maintain it. All other affected packages will recompile successfully after these changes. ldd has been tested and will work with LLVM readelf instead of GNU readelf.

[robertkirkman]

@licy183 @truboxl @twaik TL;DR this is my idea to fix a problem introduced by

• bump(main/binutils): 2.46.0 #28438

without reverting the binutils-libs removal in that PR.

can you think of any major problems that doing this would cause?

I have shown above an example of a problem that this would cause with https://github.com/paullouisageneau/libjuice.git , but it seems minor.

[robertkirkman]

Regarding GCC in TUR, if it doesn't work with LLVM binutils, I suggest we make a separate build of GNU binutils just for it (or binutils-is-gnu symbolic link package for it) and make it depend on that one, because it also depends on ndk-sysroot-gcc-compact, which conflicts with ndk-sysroot, which is required by clang, so it is basically a separate toolchain and it makes sense for it to also have a separate binutils if necessary.

https://github.com/termux-user-repository/tur/blob/59c983b23a6b6fce86ea2d99d9d0d7eef7b81a0f/tur/gcc-15/build.sh#L103

[finagolfin]

I think this is a good move. Bump the libllvm/flang revision also.

[robertkirkman]

I think this is a good move. Bump the libllvm/flang revision also.

I'll see if I can build all the packages except ldc at once in this PR.

[robertkirkman]

It is somewhat necessary to move ldc to disabled-packages to actually commit and build all changes simultaneously,

but this seems to be something appropriate because, ldc is broken and cannot really be recompiled, bumped or used on Android until these problems are fixed:

• Android binaries from releases page are broken with stack corruption detected (-fstack-protector) ldc-developers/ldc#4383
• LDC compile error #14587

truboxl on September 27 2023: "I am proposing to move LDC to disabled"

[robertkirkman]

One reason that binutils-is-llvm was added and binutils kept providing the same commands, was because "Python wouldn't recognize llvm-ar as ar"

• Add back binutils to build-essential #9715 (comment)

It's unclear exactly how to reproduce that, but I suspect it was probably that a command named exactly ar was not present and only a command named llvm-ar was present, because Python works fine in Chimera Linux where ar is symbolically linked to llvm-ar, and the only option for GNU ar that LLVM ar will create an error when passed is f.

[robertkirkman]

@strobecat @kinke @joakim-noah do you happen to know how ldc can be recompiled? If it can't be, then unfortunately it and tilix might need to be disabled until a future time when ldc is fixed.

[licy183]

IIRC some assembly files cannot be compiled correctly under clang. I wonder if this situation has improved now.

[robertkirkman]

IIRC some assembly files cannot be compiled correctly under clang. I wonder if this situation has improved now.

If you have an idea of what to check, you can check the behavior of as in this PR to make sure it is satisfactory. As noted, #11218 is preserved and not reverted. I also observed GNU as present at /usr/bin/as in Chimera Linux.

[TomJo2000]

I think LLVM as default is definitely something we can look into.
I am however unsure if this is the best approach we can take in the short term.

My main concern is user flexibility here.
I agree that we should probably use LLVM binutils for building packages for improved consistency with our closest (loosely) related projects (e.g. AOSP, Chimera, FreeBSD).
My main concern is, what - if any, impact is switching to LLVM binutils for package builds/dependents gonna have on users?
This seems like a non-trivial question to answer and should be given plenty of testing.

As a low stakes interim option we could split out a binutils-libs subpackage from the newly unified GNU binutils package again.
That should allow us to resolve the incompatibility between lua-language-server, g-ir-scanner, ldd and binutils-is-llvm while we do a thorough evaluation of making the switch to LLVM binutils as default.

[robertkirkman]

My main concern is user flexibility here.

This does sacrifice some user flexibility for increased consistency of behavior between building software on-device with and without build-package.sh.

My main concern is, what - if any, impact is switching to LLVM binutils for package builds/dependents gonna have on users?

LLVM binutils is already used for package builds. This PR will apply LLVM binutils globally to all on-device builds regardless of whether build-package.sh/setup-termux.sh was used, except for software that detects and explicitly selects gar, gobjcopy, etc. as it would need to on Chimera Linux, FreeBSD or similar to access GNU binutils.

If you mean replacing binutils with llvm in TERMUX_PKG_DEPENDS (and binutils-bin and binutils-libs with binutils), I have tested some of those already and confirmed they will work, but the others are guessing. If you know of any in particular that will need further changes, let me know.

[kinke]

do you happen to know how ldc can be recompiled? If it can't be, then unfortunately it and tilix might need to be disabled until a future time when ldc is fixed.

Upstream, we have official cross-compiled Android binaries on GitHub, using the ~latest LTS NDK (r27c currently IIRC). (So Android builds should in principle be fine.)

There was a bigger change in LDC v1.38 - we now rely on native ELF TLS on Android. This requires API level 29 for armv7a, and level 30 for aarch64. This allowed us to drop the former custom TLS emulation scheme for Android in our custom LLVM, and enabled the usage of lld as linker (not requiring ld.bfd anymore).

What this means for Termux is that if the level 29/30 requirement is okay, the LDC build should become much simpler by upgrading to the latest version, not needing the custom LLVM anymore and probably just fine using the regular LLVM package (edit: the latest beta supports LLVM 15-21). If the Termux build environment is similar to the NDK (and IIUC gets closer to the NDK with this PR) nowadays, this might become a straight-forward CMake build.

[robertkirkman]

Upstream, we have official cross-compiled Android binaries on GitHub, using the ~latest LTS NDK (r27c currently IIRC). (So Android builds should in principle be fine.)

I can still reproduce this problem with https://github.com/ldc-developers/ldc/releases/download/v1.41.0/ldc2-1.41.0-android-aarch64.tar.xz , on 64-bit ARM Android 13 which is API level 33,

curl -O -L https://github.com/ldc-developers/ldc/releases/download/v1.41.0/ldc2-1.41.0-android-aarch64.tar.xz
tar xvf ldc2-1.41.0-android-aarch64.tar.xz
cd ldc2-1.41.0-android-aarch64/bin
./ldc2

stack corruption detected (-fstack-protector)
stack corruption detected (-fstack-protector)
Aborted                    ./ldc2

I understand that you might be aware of this problem already, but this is one of the things preventing us from upgrading to a newer version of LDC.

Is this something you can reproduce, or does that build work properly on your device?

[kinke]

Is this something you can reproduce, or does that build work properly on your device?

I've never run it on my phone. ;) - Oh, good to know this issue is still a problem (as it's been reported once only, and way before the bigger TLS change). I hope it's 'just' a build problem, e.g., caused by the D parts compiled with latest LLVM from LDC cross-compiler, and the NDK's clang being older. No stack trace available, in case it'd be a genuine bug?

[robertkirkman]

Is this something you can reproduce, or does that build work properly on your device?

I've never run it on my phone. ;) - Oh, good to know this issue is still a problem (as it's been reported once only, and way before the bigger TLS change). I hope it's 'just' a build problem, e.g., caused by the D parts compiled with latest LLVM from LDC cross-compiler, and the NDK's clang being older. No stack trace available, in case it'd be a genuine bug?

Here's a little more information I collected:

• This is what I see when I run it through gdb ./ldc2, then use bt:

stack corruption detected (-fstack-protector)

Program received signal SIGABRT, Aborted.
0x0000007fbbaeb994 in abort () from /apex/com.android.runtime/lib64/bionic/libc.so
(gdb) bt
#0  0x0000007fbbaeb994 in abort () from /apex/com.android.runtime/lib64/bionic/libc.so
#1  0x0000007fbbb005ec in __stack_chk_fail () from /apex/com.android.runtime/lib64/bionic/libc.so
#2  0x0000005559f301bc in RegisterHandlers() ()
#3  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) 

• My first test was with Termux, but the bug is confirmed to also be reproducible when using only Android and LDC, taking Termux out of the equation by connecting to Android though ADB shell and running LDC there, so this is something specific to LDC itself:

tacokoneko@CORSAIR ~/code $ adb -s 192.168.12.185:35743  push ldc2-1.41.0-android-aarch64 /data/local/tmp
ldc2-1.41.0-android-aarch64/: 763... MB/s (334740300 bytes in 6.396s)
tacokoneko@CORSAIR ~/code $ adb -s 192.168.12.185:35743  shell
a70q:/ $ cd /data/local/tmp                                                                                                                                  
a70q:/data/local/tmp $ cd ldc2-1.41.0-android-aarch64/bin/                                                                                                   
a70q:/data/local/tmp/ldc2-1.41.0-android-aarch64/bin $ ./ldc
ldc-build-plugin   ldc-build-runtime  ldc-profdata       ldc-profgen        ldc-prune-cache    ldc2
a70q:/data/local/tmp/ldc2-1.41.0-android-aarch64/bin $ ./ldc2                                                                                                
stack corruption detected (-fstack-protector)
stack corruption detected (-fstack-protector)
Aborted 
134|a70q:/data/local/tmp/ldc2-1.41.0-android-aarch64/bin $ getprop ro.build.version.sdk
33
a70q:/data/local/tmp/ldc2-1.41.0-android-aarch64/bin $ uname -a
Linux localhost 4.14.190-lineage-gef0cb31b13 #4 SMP PREEMPT Fri Jun 23 13:11:47 CEST 2023 aarch64 Toybox
a70q:/data/local/tmp/ldc2-1.41.0-android-aarch64/bin $ 

[kinke]

#0 0x0000007fbbaeb994 in abort () from /apex/com.android.runtime/lib64/bionic/libc.so
#1 0x0000007fbbb005ec in __stack_chk_fail () from /apex/com.android.runtime/lib64/bionic/libc.so
#2 0x0000005559f301bc in RegisterHandlers() ()
#3 0x0000000000000000 in ?? ()

Thx! Hmm, this RegisterHandlers() is apparently an internal LLVM function wrt. signal handling. So maybe even a build problem of our LLVM (which we also cross-compile via the NDK). I guess the -fstack-protector comes from the NDK...

Edit: Maybe worth a try: ldc-developers/ldc#5070. Unfortunately, the CI builds are currently blocked by apt.llvm.org problems.

[strobecat]

@strobecat @kinke @joakim-noah do you happen to know how ldc can be recompiled? If it can't be, then unfortunately it and tilix might need to be disabled until a future time when ldc is fixed.

Yes, the current used version of ldc isn't compatible with llvm binutils iirc🤔 I remember trying using the linker from llvm binutils to compile tilix sometimes ago. It crashed with something alike "xxx not balanced" and instructed me to use the linker from gnu binutils

[robertkirkman]

We tried for a while to get a newer version of LDC to work, but unfortunately we couldn't really get it to work in a stable way. However, at least I have eventually been able to recompile the current version of LDC available for Termux, and that version is still working, so I guess for now it will need to stay at that version.

@strobecat I tested building tilix inside Termux with the packages from this PR by applying this PR, installing the packages from the CI artifact, and using ./build-package.sh -I -f tilix, and I didn't see any errors, is that error about llvm binutils with Tilix something you can still reproduce?

[robertkirkman]

On close inspection of the BSD-family systems, I have noticed that their binutils situations are more complicated than I originally thought. they do pick defaults and stick to them, but they pick them individually on a per-program basis, and currently have mixtures of programs from several different original sources, including LLVM binutils.

This PR should, I think, continue with moving Termux to default to LLVM binutils for everything that conflicts with GNU binutils except for as, though, because that will still make Termux more consistent with the two other Linux distros that default to the LLVM C compiler, C++ compiler and C++ standard library, vanilla Android and Chimera Linux, and it will also make on-device building behavior more consistent between building with build-package.sh and building without it.

[strobecat]

We tried for a while to get a newer version of LDC to work, but unfortunately we couldn't really get it to work in a stable way. However, at least I have eventually been able to recompile the current version of LDC available for Termux, and that version is still working, so I guess for now it will need to stay at that version.

@strobecat I tested building tilix inside Termux with the packages from this PR by applying this PR, installing the packages from the CI artifact, and using ./build-package.sh -I -f tilix, and I didn't see any errors, is that error about llvm binutils with Tilix something you can still reproduce?

On-device build seems to be working. It shall be OK as long as ldc still uses ld.bfd

[robertkirkman]

I will merge it in 24 hours from now if there are not any objections.

Practically speaking, the behavior will be mostly similar to the current behavior of Termux when binutils-is-llvm is installed for a long time before now, and the only issue I personally know about that will be a breaking change affecting a lot of people is the problem with https://github.com/paullouisageneau/libjuice . I will spread a message to the people who need to know that they will need to patch libjuice (both upstream and downstream)