Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ target
!example.config.toml
*.sqlite
Cargo.lock
!payjoin-ffi/dart/native/Cargo.lock
.vscode
mutants.out*
*.ikm
Expand Down
13 changes: 13 additions & 0 deletions payjoin-ffi/dart/.cargo/config.local.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Local Cargo overlay for development in the monorepo. The name keeps it
# out of Cargo's automatic config discovery, so only hook/build.dart reads
# it, through an explicit --config. Its presence is also how the hook tells
# a repository checkout from a published archive, which .pubignore strips
# .cargo/ from.
[patch."https://github.com/payjoin/rust-payjoin.git"]
payjoin-ffi = { path = ".." }
payjoin = { path = "../../payjoin" }

[patch.crates-io]
payjoin = { path = "../../payjoin" }
payjoin-mailroom = { path = "../../payjoin-mailroom" }
payjoin-test-utils = { path = "../../payjoin-test-utils" }
10 changes: 0 additions & 10 deletions payjoin-ffi/dart/.cargo/config.toml

This file was deleted.

3 changes: 0 additions & 3 deletions payjoin-ffi/dart/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,3 @@ doc/api/

# Auto-generated bindings
lib/payjoin.dart

# Local Cargo overlay for monorepo dev
!.cargo/config.toml
154 changes: 150 additions & 4 deletions payjoin-ffi/dart/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,34 @@ bash ./scripts/generate_bindings.sh

```shell
# Run all tests
dart test
bash ./contrib/test.sh
```

This is what CI runs. It also moves `native/Cargo.lock` aside for the run
and puts it back afterwards, which a bare `dart test` does not: the tracked
lockfile pins the published dependency graph, and the development overlay
resolves a different one, so Cargo would refuse it and then overwrite it.

### The native build hook

`hook/build.dart` builds the `native/` wrapper crate for the consumer's
target. It takes the presence of `.cargo/config.local.toml` as the signal
that it is running inside this repository rather than from a published
archive, because `.pubignore` withholds `.cargo/` from that archive. There
the overlay redirects `payjoin-ffi` to the workspace and the hook enables
`_test-utils`; from an archive the hook builds the tracked production graph
with `--locked` instead.

This cannot be an environment variable. `hooks_runner` spawns build hooks
with an allowlisted environment — `HOME`, `PATH`, a handful of toolchain
names — and a project specific variable never reaches the hook.

The hook also passes `--remap-path-prefix` for the package, output, Cargo,
and Rustup directories, so the artifact does not record where it was built.
Those prefixes go through `CARGO_ENCODED_RUSTFLAGS` rather than `RUSTFLAGS`,
since Cargo splits the latter on whitespace and a consumer's project path
may contain spaces.

## Releasing

Maintainer instructions for publishing to
Expand All @@ -51,9 +76,21 @@ validates the archive with a publish dry run
([`contrib/prepare-publish.sh`](contrib/prepare-publish.sh)).

1. Point the `payjoin-ffi` dependency in `native/Cargo.toml` at the commit
tagged for the `payjoin` release being wrapped. Consumers build from that
revision. `.cargo/config.toml` redirects it to the local workspace for
development only, and `.pubignore` withholds that file from the archive.
tagged for the `payjoin` release being wrapped, then regenerate the
tracked lockfile consumers build with `--locked` and commit it:

```shell
# from native/, so rustup picks up its pinned toolchain
(cd native && cargo generate-lockfile)
bash ./contrib/check_production_lock.sh native/Cargo.toml native/Cargo.lock
```

The checker runs in CI too. It asserts that the lockfile pins the
revision the manifest asks for, and that the wrapper is the only package
without a `source`. Nothing greps for a `path` key: a Cargo lockfile has
none, and a dependency resolved through the `.cargo/config.local.toml`
overlay is exactly a package that lost its source.

2. Set the version in `pubspec.yaml` and describe the consumer-visible
changes under a matching heading in `CHANGELOG.md`.
3. Confirm every `Build and Test Dart` job is green on the release commit
Expand All @@ -80,3 +117,112 @@ validates the archive with a publish dry run
new version and its changelog.

[automated publishing]: https://dart.dev/tools/pub/automated-publishing

## Checking native reproducibility

`contrib/check_reproducible.sh` compares two explicitly built native wrapper
artifacts byte-for-byte. It prints their SHA-256 digests, target, and compiler
toolchain versions, and works on Linux and macOS. Run its lightweight tests
without compiling Rust:

```shell
bash ./contrib/check_reproducible_test.sh
```

The following example fetches the production dependency graph once, copies
the resulting Cargo inputs, and runs two isolated builds in parallel. Set
`PAYJOIN_REPRO_PARENT` to a dedicated directory outside the repository, never
to a shared build directory. It builds the tracked `native/Cargo.lock` with
`--locked`, like a consumer does: the development `.cargo/config.local.toml`
overlay is never read, and `_test-utils` stays off.

```shell
set -euo pipefail
repro_parent="${PAYJOIN_REPRO_PARENT:?Set PAYJOIN_REPRO_PARENT to a dedicated directory}"
work_dir=$(mktemp -d "$repro_parent/payjoin-dart-repro.XXXXXX")
trap 'rm -rf "$work_dir"' EXIT
toolchain_bin=$(dirname "$(rustup which --toolchain 1.85.1 cargo)")
cargo_bin="$toolchain_bin/cargo"
rustc_bin="$toolchain_bin/rustc"
target=$("$rustc_bin" -vV | awk '/^host:/{print $2}')
artifact=libpayjoin_ffi_wrapper.so

if [[ "$(uname -s)" == Darwin ]]; then
artifact=libpayjoin_ffi_wrapper.dylib
fi

mkdir -p "$work_dir/seed" "$work_dir/cargo-seed" "$work_dir/first" \
"$work_dir/second" "$work_dir/cargo-first" "$work_dir/cargo-second"
cp -a ./native/. "$work_dir/seed/"
CARGO_HOME="$work_dir/cargo-seed" "$cargo_bin" fetch --locked \
--manifest-path "$work_dir/seed/Cargo.toml"
cp -a "$work_dir/seed/." "$work_dir/first/"
cp -a "$work_dir/seed/." "$work_dir/second/"
cp -a "$work_dir/cargo-seed/." "$work_dir/cargo-first/"
cp -a "$work_dir/cargo-seed/." "$work_dir/cargo-second/"

# Unit separated, like the hook: these prefixes are absolute paths and
# Cargo splits plain RUSTFLAGS on whitespace.
encode_flags() {
local copy=$1
printf '%s\x1f%s\x1f%s' \
"--remap-path-prefix=$work_dir/$copy=/payjoin/package" \
"--remap-path-prefix=$work_dir/cargo-$copy=/cargo" \
"--remap-path-prefix=$work_dir/$copy-target=/payjoin/output"
}

build_copy() {
local copy=$1
CARGO_ENCODED_RUSTFLAGS="$(encode_flags "$copy")" RUSTC="$rustc_bin" \
RUSTC_WRAPPER= CARGO_NET_OFFLINE=true \
CARGO_HOME="$work_dir/cargo-$copy" "$cargo_bin" build --offline \
--manifest-path "$work_dir/$copy/Cargo.toml" --release --locked \
--target "$target" --target-dir "$work_dir/$copy-target"
}

build_copy first >"$work_dir/first-build.log" 2>&1 &
first_pid=$!
build_copy second >"$work_dir/second-build.log" 2>&1 &
second_pid=$!

first_status=0
second_status=0
wait "$first_pid" || first_status=$?
wait "$second_pid" || second_status=$?
((first_status == 0 && second_status == 0))

bash ./contrib/check_reproducible.sh \
"$target" \
"$($rustc_bin --version)" \
"$(clang --version | awk 'NR == 1 {print}')" \
"$(ld.lld --version | awk 'NR == 1 {print}')" \
"$work_dir/first-target/$target/release/$artifact" \
"$work_dir/second-target/$target/release/$artifact"
```

For Android, pass an Android triple as `--target` and read the artifact from
the matching directory. The same lockfile and offline Cargo inputs must be
used. Do not replace `native/Cargo.lock` with one generated through the local
path overlay: `contrib/check_production_lock.sh` rejects that, and consumers
could not resolve it.

### Consumer smoke test

`contrib/smoke_consumer.sh` checks the published binding source in two isolated
Dart consumers. It requires a dedicated writable parent and a read-only seed
Cargo cache; the script copies that cache into per-consumer homes and removes
all generated files from its own temporary directory on exit.

```shell
PAYJOIN_SMOKE_PARENT=/path/to/dedicated/smoke-parent \
PAYJOIN_BINDING_SOURCE=/path/to/published/payjoin.dart \
PAYJOIN_SMOKE_CARGO_HOME=/path/to/dedicated/cargo-cache \
bash ./contrib/smoke_consumer.sh
```

The two runs use separate `CARGO_HOME` values, and each consumer's hook
builds under its own `.dart_tool`, so Cargo already gets a distinct target
directory per run. Setting `CARGO_TARGET_DIR` would change nothing: the hook
passes `--target-dir` explicitly, and the command line wins. Android and
cross-host consumer builds still require the equivalent target and toolchain
smoke coverage described above.
67 changes: 67 additions & 0 deletions payjoin-ffi/dart/contrib/check_production_lock.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
set -euo pipefail

# Validate the native lockfile that ships to Dart consumers.
#
# A lockfile resolved through the .cargo/config.local.toml path overlay pins
# the local workspace instead of the published Git revision, and consumers
# cannot reproduce it. Such an entry has no `path` key to grep for: a path
# dependency shows up as a package without a `source` at all, exactly like
# the root wrapper crate. So assert that the wrapper is the only sourceless
# package, and that the pinned `payjoin-ffi` revision still matches the one
# the manifest asks for, which is what `--locked` enforces at build time.

if [[ $# -ne 2 ]]; then
printf 'Usage: %s MANIFEST LOCKFILE\n' "$0" >&2
exit 2
fi

manifest=$1
lockfile=$2
root_package=payjoin-ffi-wrapper
repository=https://github.com/payjoin/rust-payjoin.git

for file in "$manifest" "$lockfile"; do
if [[ ! -f $file ]]; then
printf 'Production lockfile check failed: %s is missing\n' "$file" >&2
exit 1
fi
done

sourceless=$(awk -v root="$root_package" '
function flush() {
if (in_package && !sourced && name != root) print name
in_package = 0
}
/^\[\[package\]\]$/ { flush(); in_package = 1; name = ""; sourced = 0; next }
in_package && /^name = / { name = $3; gsub(/"/, "", name); next }
in_package && /^source = / { sourced = 1; next }
in_package && /^[[:space:]]*$/ { flush(); next }
END { flush() }
' "$lockfile")

if [[ -n $sourceless ]]; then
printf 'Production lockfile resolves packages from a local path:\n' >&2
printf '%s\n' "$sourceless" >&2
printf 'Regenerate it with .cargo/config.local.toml out of the way.\n' >&2
exit 1
fi

revision=$(sed -n \
"s|.*git = \"$repository\", rev = \"\([0-9a-f]\{40\}\)\".*|\1|p" \
"$manifest")
if [[ -z $revision ]]; then
printf 'Could not read the pinned payjoin-ffi revision from %s\n' \
"$manifest" >&2
exit 2
fi

expected="source = \"git+$repository?rev=$revision#$revision\""
if ! grep -Fq "$expected" "$lockfile"; then
printf 'Production lockfile does not pin payjoin-ffi to %s\n' \
"$revision" >&2
printf 'Regenerate it after changing the manifest revision.\n' >&2
exit 1
fi

printf 'Production lockfile check passed: %s pins %s\n' "$lockfile" "$revision"
68 changes: 68 additions & 0 deletions payjoin-ffi/dart/contrib/check_production_lock_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
CHECKER="$SCRIPT_DIR/check_production_lock.sh"
work_dir=$(mktemp -d "${TMPDIR:-/tmp}/payjoin-production-lock-test.XXXXXX")
trap 'rm -rf "$work_dir"' EXIT

pinned=1111111111111111111111111111111111111111
stale=2222222222222222222222222222222222222222

cat >"$work_dir/Cargo.toml" <<EOF
[dependencies]
payjoin-ffi = { git = "https://github.com/payjoin/rust-payjoin.git", rev = "$pinned", features = [
"dart",
] }
EOF

write_lock() {
local path=$1 revision=$2 source_line=$3
cat >"$path" <<EOF
version = 4

[[package]]
name = "payjoin-ffi-wrapper"
version = "0.1.0"
dependencies = [
"payjoin-ffi",
]

[[package]]
name = "payjoin-ffi"
version = "0.24.0"
EOF
if [[ $source_line == with-source ]]; then
printf 'source = "git+https://github.com/payjoin/rust-payjoin.git?rev=%s#%s"\n' \
"$revision" "$revision" >>"$path"
fi
}

expect_failure() {
local lockfile=$1 expected=$2
if "$CHECKER" "$work_dir/Cargo.toml" "$lockfile" \
>"$work_dir/out.log" 2>"$work_dir/error.log"; then
printf 'Expected %s to fail the check\n' "$lockfile" >&2
exit 1
fi
if ! grep -Fq "$expected" "$work_dir/error.log"; then
printf 'Failed for an unexpected reason, wanted %s\n' "$expected" >&2
cat "$work_dir/error.log" >&2
exit 1
fi
}

write_lock "$work_dir/good.lock" "$pinned" with-source
"$CHECKER" "$work_dir/Cargo.toml" "$work_dir/good.lock" >/dev/null

# A lockfile resolved through the path overlay: no `path` key anywhere, the
# patched package simply loses its source.
write_lock "$work_dir/overlay.lock" "$pinned" without-source
expect_failure "$work_dir/overlay.lock" 'resolves packages from a local path'

write_lock "$work_dir/stale.lock" "$stale" with-source
expect_failure "$work_dir/stale.lock" "does not pin payjoin-ffi to $pinned"

expect_failure "$work_dir/missing.lock" 'missing.lock is missing'

printf 'Production lockfile checker tests passed\n'
44 changes: 44 additions & 0 deletions payjoin-ffi/dart/contrib/check_reproducible.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -euo pipefail

if [[ $# -ne 6 ]]; then
printf 'Usage: %s TARGET RUSTC CLANG LINKER FIRST_ARTIFACT SECOND_ARTIFACT\n' "$0" >&2
exit 2
fi

target=$1
rustc_version=$2
clang_version=$3
linker_version=$4
first_artifact=$5
second_artifact=$6

if [[ ! -f $first_artifact || ! -f $second_artifact ]]; then
printf 'Reproducibility check failed: both artifacts must be regular files\n' >&2
exit 2
fi

sha256() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | cut -d ' ' -f 1
elif command -v shasum >/dev/null 2>&1; then
shasum -a 256 "$1" | cut -d ' ' -f 1
else
printf 'Cannot compute SHA-256: sha256sum or shasum is required\n' >&2
exit 2
fi
}

printf 'Target: %s\n' "$target"
printf 'rustc: %s\n' "$rustc_version"
printf 'clang: %s\n' "$clang_version"
printf 'linker: %s\n' "$linker_version"
printf '%s: %s\n' "$first_artifact" "$(sha256 "$first_artifact")"
printf '%s: %s\n' "$second_artifact" "$(sha256 "$second_artifact")"

if ! cmp -s "$first_artifact" "$second_artifact"; then
printf 'Reproducibility check failed: artifacts differ byte-for-byte\n' >&2
exit 1
fi

printf 'Reproducibility check passed: artifacts are byte-for-byte identical\n'
Loading
Loading