Running hledger-flow on Linux aarch64 / arm64 (Docker on Mac)

[desaiuditd]

Describe the bug
The new version 0.16.0 for hledger-flow fails with below error when running it inside a Debian Docker image built on MacOS machine with M1 chip.

# hledger-flow --version
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
 Trace/breakpoint trap

Version and Runtime Information

0.16.0
Debian docker image built on MacOS machine with M1 chip.
Base Docker Image python:3.13-slim

Please mention the version number of hledger-flow you are using:
Can't get the version through CLI, because that itself fails.

Is this the latest version?
Please confirm your issue using the latest version of hledger-flow, maybe it has already been fixed.
Yes. Latest version 0.16.0 from releases page.

Also include the runtime options of the command you ran, e.g:
This command also fails.

# hledger-flow --show-options import
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
 Trace/breakpoint trap

To Reproduce

Create a custom docker image with a Dockerfile.

FROM python:3.13-slim

ENV LANG="C.UTF-8"

RUN apt update -y

RUN apt install -y --no-install-recommends build-essential gcc curl

RUN cd /tmp

RUN curl -fLOC- https://github.com/simonmichael/hledger/releases/download/1.43.2/hledger-linux-x64.tar.gz && \
  tar xzf hledger-linux-x64.tar.gz && \
  mv hledger /usr/local/bin/ && \
  mv hledger-ui /usr/local/bin/ && \
  mv hledger-web /usr/local/bin/ && \
  rm hledger*

RUN curl -LOC- https://github.com/apauley/hledger-flow/releases/download/v0.16.0/hledger-flow_Linux_x86_64_v0.16.0_5901ca5.tar.gz && \
  tar xvf hledger-flow_Linux_x86_64_v0.16.0_5901ca5.tar.gz && \
  mv hledger-flow_Linux_x86_64_v0.16.0_5901ca5/hledger-flow /usr/local/bin/ && \
  rm -rf hledger-flow_Linux_x86_64_v0.16.0_5901ca5 hledger-flow_Linux_x86_64_v0.16.0_5901ca5.tar.gz

RUN hledger-flow --version

Expected behavior
hledger-flow should work inside Docker image built on MacOS with M1 chip machine.

Desktop (please complete the following information):

• OS: MacOS Apple M1 Pro chip
• Version: 15.16.1

[apauley]

Hi @desaiuditd - I've never provided binaries for all platforms, only 64 bit Mac and Linux, so I don't consider this a bug.

Maybe you could compile this project inside a docker container, and copy the resulting binary to your other container.

This page says stack doesn't work for ARM64, but cabal does, so you could try the following (using a Haskell version that corresponds to the stackage version used in hledger-flow 0.16):

docker run --rm -it --entrypoint bash haskell:9.10.2

Inside the container's shell:

# Check if the architecture is aarch64
uname -a

# See if cabal and hackage can help you out
cabal install hledger-flow

Then copy the resulting hledger-flow binary to wherever you need it

Good luck, please report back for the benefit of other people who may want to do the same.

[desaiuditd]

@apauley Possibly. Thanks for the hints.

Originally, I'm trying with python:3.13-slim base image because I've some other dependency on it for some other python based tool.

Let me try with haskell base image. Will get back here.

However, just one thing to note here that -- Version 0.15.0 was working well with my current approach mentioned above. So not sure what changed in 0.16.0.

[apauley]

I see, it looks like there was a change in how the binaries are built.

hledger-flow 0.16 is dynamically linked, while hledger-flow 0.15 (and hledger) is statically linked.

root@f5099c937afe:/# file hledger
hledger: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=31d63de5b926775d20e623523e5a50e8cc5c2e24, stripped
root@f5099c937afe:/# file hledger-flow_Linux_x86_64_v0.15.0_2b025fe/hledger-flow
hledger-flow_Linux_x86_64_v0.15.0_2b025fe/hledger-flow: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped, too many notes (256)
root@f5099c937afe:/# file hledger-flow_Linux_x86_64_v0.16.0_5901ca5/hledger-flow
hledger-flow_Linux_x86_64_v0.16.0_5901ca5/hledger-flow: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=4ec966d299d0a5760de11fe620a6f28f8152ae94, for GNU/Linux 3.2.0, stripped

[desaiuditd]

One quick fix I've found is to use --platform linux/amd64 while running the container or while building the image.
But event with that, I'm getting some warnings about ticker poll failing and interrupted system call etc.

hledger-flow import /ledger
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
hledger-flow: Ticker: poll failed: Interrupted system call: Interrupted system call
hledger-flow: Ticker: poll failed: Interrupted system call: Interrupted system call
hledger-flow: Ticker: poll failed: Interrupted system call: Interrupted system call
Found 123 input files in 0.090570833s. Proceeding with import...
Wrote include files for 123 journals in 0.187962458s
Imported 123/123 journals in 14.719900673s

hledger-flow 0.16 is dynamically linked

@apauley Curious to know if hledger-flow is going to remain that way going forward
or was this some unintentional change and you are looking to fix/revert it meanwhile.

I'm asking this because if you provide statically linked binary (similar to hledger), then we can easily use and run the tools even inside alpine based docker images, which are much smaller in size.

[desaiuditd]

Building fresh hledger-flow using cabal inside haskell container worked.

FROM haskell:slim AS hledger-flow-build

RUN cabal update && \
    cabal install hledger-flow-0.16.1

FROM debian:stable-slim

COPY --from=hledger-flow-build /root/.local/bin/hledger-flow /usr/local/bin/hledger-flow

root@a3f979930eb3:/# hledger-flow --version
hledger-flow 0.16.1 linux aarch64 ghc 9.12 UNKNOWN

root@a3f979930eb3:/# file /usr/local/bin/hledger-flow
/usr/local/bin/hledger-flow: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=7249ebf7fb82b9d0f90fd068e73d5abf61d1d29d, not stripped

The only problem I've with this approach is the docker image size for haskell to download and the time it takes to build and install hledger-flow binary with cabal.

haskell:slim docker image uncompressed size is 4.88 GB. And that too slim version.
And it's taking close to 4-5 mins just to build and install hledger-flow. It feels too much.

root@ed206d8181b2:/# time (cabal update && cabal install hledger-flow-0.16.1)
...
Starting     hledger-flow-0.16.1 (lib)
Building     hledger-flow-0.16.1 (lib)
Installing   hledger-flow-0.16.1 (lib)
Completed    hledger-flow-0.16.1 (lib)
Starting     hledger-flow-0.16.1 (exe:hledger-flow)
Building     hledger-flow-0.16.1 (exe:hledger-flow)
Installing   hledger-flow-0.16.1 (exe:hledger-flow)
Completed    hledger-flow-0.16.1 (exe:hledger-flow)
Symlinking 'hledger-flow' to '/root/.local/bin/hledger-flow'

real    2m26.676s
user    4m22.517s
sys     0m26.365s

@apauley
Do you have any plans to build and publish/release compiled binaries for linux/arm64 platform?

[desaiuditd]

@apauley Any update on this?

[desaiuditd]

@apauley Any update on this?

[apauley]

@desaiuditd this looks like an ideal improvement that you can contribute. It looks as if you really need this (and I don't), so the incentive is definitely in your favor.

I see there has been a similar contribution for hledger builds:
hledgerorg/hledger#1571

Alternatively you could create and maintain docker images for the community which would include hledger and hledger-flow.

[desaiuditd]

Thank for the encouragement @apauley

I'll try to give it a go.

No experience with Haskell project before. Will try.

[desaiuditd]

@apauley What is your process for releasing pre-built binaries to Github releases?

Are you using CircleCI jobs (config present in this repo)?
If so, Do you run that manually?
I could not see any builds for this project on CircleCI - https://app.circleci.com/pipelines/github/apauley/hledger-flow

I'm asking this because I was thinking to attempt for the improvement and raise a PR. So just wanted to make sure what build system you want to use here. CircleCI or Github Actions (which is used in hledger project PR you referenced)?

[apauley]

Hi @desaiuditd - I don't use CircleCI anymore, at some point it stopped working. I think adding a Github Actions workflow would make sense now, back when I started this repo Actions didn't exist yet.

With the recent release I manually built the binary (see bin/build-and-test), using both Linux and Mac machines.
Then I created a tar file on each platform using bin/release-tarball. Finally I pushed a new tag, created a release in Github for the tag, and uploaded the tar files to the release.

Normally I would have created a signed tag using bin/tag-release, but with the latest release I skipped the signing part and just pushed an unsigned tag.

A new Github actions workflow should build tar files and run tests for any branch/PR, on multiple platforms/architectures if possible - Linux, Mac, arm64 etc.
I think creating a git tag should still be done manually - this means we can manually decide when and what to release.
Optionally the Github action can create a new Github release whenever it sees a tag starting with v - the recent release tag was named v0.16.0

As for solving the original problem of making the binary run within docker on arm processors - there some experimentation is needed. I haven't looked in depth into how Actions builds on multiple platforms, but it may be that the resulting binary you get after building on an arm architecture is what you need. We can then add it to future releases as another tar file.

In previous releases I was able to build a static binary on Linux by using -static in package.yaml (see bin/build-and-test), but this stopped working at some point in the past.

I looked at the hledger PR I shared earlier in this thread, and it seems that it may no longer be used - have a look at what hledger currently does. It looks well documented.