Add basic bazel build (S-Core Integration) - #348
Conversation
| ) | ||
|
|
||
| # --- crate_universe: auto-generate Bazel targets for external Cargo dependencies --- | ||
| crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate") |
There was a problem hiding this comment.
| crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate") | |
| crate = use_extension("@rules_rust//crate_universe:extensions.bzl", "crate", isolate = True) |
Eclipse S-CORE also has a "crates universe", defined here. I think this clashes with the universe defined here:
ERROR: /var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/extensions.bzl:941:21: Traceback (most recent call last):
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/extensions.bzl", line 941, column 21, in _crate_impl
fail("Defined two crate universes with the same name in different MODULE.bazel files (`{}`). Either give one a different name, or use `use_extension(isolate=True)`".format(
Error in fail: Defined two crate universes with the same name in different MODULE.bazel files (`crate_index`). Either give one a different name, or use `use_extension(isolate=True)`
ERROR: Analysis of target '@@classic-diagnostic-adapter+//:opensovd-cda' failed; build aborted: error evaluating module extension @@rules_rust+//crate_universe:extensions.bzl%crate
INFO: Elapsed time: 80.397s, Critical Path: 0.03s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED:
Fetching module extension @@rules_rust+//crate_universe:extensions.bzl%crate; Generating hub and spokes 73sWhile having a multi-verse would be cool 😄 - maybe the solution is as simple as implementing the isolate = True proposal? Another (maybe more maintainable / long-term) solution could be to use the score-crates also here, but this introduces a dependency to S-CORE which may not really be wanted. Here, a Bazel expert should weigh in.
There was a problem hiding this comment.
Thanks for the hint! I added the flag and enabled the experimental support for it via .baselrc :)
🤞that it works! :)
As for using the score-crates. Thats something we could evaluate. But it might be a higher maintenance burden than we would want for now (on both sides). As we would need to align with score whenever we want to update our lock file (due to new rustsec advisories, etc.). I think its a great topic for the discussion on aligning releases and things though!
edit:
sorry, need to update the modules, as i had to rebase on main for gh to start the actions again. It dragged in a new dependency I had not noticed
There was a problem hiding this comment.
With the fix I am getting a bit further now:
Compiling cargo-bazel v0.17.0 (/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe)
Finished `release` profile [optimized] target(s) in 50.58s
Error: Failed to splice workspace
Caused by:
Some manifests are not being tracked.
Please add the following labels to the `manifests` key:
//cda-build:Cargo.toml
//cda-comm-doip:Cargo.toml
//cda-comm-uds:Cargo.toml
//cda-core:Cargo.toml
//cda-database:Cargo.toml
//cda-extra:Cargo.toml
//cda-health:Cargo.toml
//cda-interfaces:Cargo.toml
//cda-main:Cargo.toml
//cda-plugin-security:Cargo.toml
//cda-sovd:Cargo.toml
//cda-sovd-interfaces:Cargo.toml
//:cda-storage/Cargo.toml
//cda-tracing:Cargo.toml
//comm-mbedtls/mbedtls-rs:Cargo.toml
//comm-mbedtls/mbedtls-sys:Cargo.toml
//:integration-tests/Cargo.toml
//opensovd-axum-extra:Cargo.toml.
ERROR: /var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/private/common_utils.bzl:55:13: Traceback (most recent call last):
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/extensions.bzl", line 1009, column 37, in _crate_impl
_generate_hub_and_spokes(
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/extensions.bzl", line 633, column 51, in _generate_hub_and_spokes
splice_outputs = splice_workspace_manifest(
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/private/splicing_utils.bzl", line 174, column 19, in splice_workspace_manifest
cargo_bazel_fn(
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/private/common_utils.bzl", line 88, column 23, in _execute
return execute(
File "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust+/crate_universe/private/common_utils.bzl", line 55, column 13, in execute
fail(_EXECUTE_ERROR_MESSAGE.format(
Error in fail: Command ["/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust++cu_nr+cargo_bazel_bootstrap/cargo-bazel", "splice", "--output-dir", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/modextwd/rules_rust++_crate+classic-diagnostic-adapter++crate/crate_index/splicing-output", "--splicing-manifest", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/modextwd/rules_rust++_crate+classic-diagnostic-adapter++crate/crate_index/splicing_manifest.json", "--config", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/modextwd/rules_rust++_crate+classic-diagnostic-adapter++crate/crate_index/config.json", "--cargo-lockfile", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/classic-diagnostic-adapter+/Cargo.lock", "--cargo", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust++rust_host_tools+rust_host_tools/bin/cargo", "--rustc", "/var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/rules_rust++rust_host_tools+rust_host_tools/bin/rustc"] failed with exit code 1.
STDOUT ------------------------------------------------------------------------
STDERR ------------------------------------------------------------------------
Error: Failed to splice workspace
Caused by:
Some manifests are not being tracked.
Please add the following labels to the `manifests` key:
//cda-build:Cargo.toml
//cda-comm-doip:Cargo.toml
//cda-comm-uds:Cargo.toml
//cda-core:Cargo.toml
//cda-database:Cargo.toml
//cda-extra:Cargo.toml
//cda-health:Cargo.toml
//cda-interfaces:Cargo.toml
//cda-main:Cargo.toml
//cda-plugin-security:Cargo.toml
//cda-sovd:Cargo.toml
//cda-sovd-interfaces:Cargo.toml
//:cda-storage/Cargo.toml
//cda-tracing:Cargo.toml
//comm-mbedtls/mbedtls-rs:Cargo.toml
//comm-mbedtls/mbedtls-sys:Cargo.toml
//:integration-tests/Cargo.toml
//opensovd-axum-extra:Cargo.toml.
ERROR: Analysis of target '@@classic-diagnostic-adapter+//:opensovd-cda' failed; build aborted: error evaluating module extension @@rules_rust+//crate_universe:extensions.bzl%crate%classic-diagnostic-adapter@_+crate
INFO: Elapsed time: 72.528s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED:
Fetching module extension @@rules_rust+//crate_universe:extensions.bzl%crate%classic-diagnostic-adapter@_+crate; Splicing Cargo workspace for `crate_index` 63s
There was a problem hiding this comment.
It is not really hard to run this, but just the workspace is a bit "hacked"... maybe you can have a look anyway. I will try to create a minimal reproducer as well.
There was a problem hiding this comment.
Did a 'best guess' fix and added the manifests into the root MODULE.bazel.
A minimal reproducer would be awesome though <3
There was a problem hiding this comment.
... aaand another step further! It seems to pick up the edition now. However, there is now yet another problem; I assume with the sysroot / SDK. I need to check if that header is really missing or what is going on here. Will update here!
bazel build --config eb-aarch64-rpi4b @classic-diagnostic-adapter//:opensovd-cda --experimental_isolated_extension_usages --@classic-diagnostic-adapter//:tls_backend=mbedtls
INFO: Invocation ID: cb03646c-512b-47cd-8f09-a93ef003f898
WARNING: For repository 'rules_shell', the root module requires module version rules_shell@0.6.0, but got rules_shell@0.6.1 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'rules_cc', the root module requires module version rules_cc@0.2.16, but got rules_cc@0.2.17 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'rules_python', the root module requires module version rules_python@1.8.3, but got rules_python@1.8.5 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'aspect_rules_lint', the root module requires module version aspect_rules_lint@2.0.0, but got aspect_rules_lint@2.3.0 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'buildifier_prebuilt', the root module requires module version buildifier_prebuilt@8.2.0.2, but got buildifier_prebuilt@8.5.1 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'score_rust_policies', the root module requires module version score_rust_policies@0.0.3, but got score_rust_policies@0.0.5 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
INFO: Analyzed target @@classic-diagnostic-adapter+//:opensovd-cda (587 packages loaded, 63925 targets configured).
ERROR: /var/cache/bazel/976537d6d94a3965f1386b2d77b73fa0/external/classic-diagnostic-adapter+/comm-mbedtls/mbedtls-sys/BUILD.bazel:30:19: Running Cargo build script build_script failed: (Exit 1): runner failed: error executing CargoBuildScriptRun command (from target @@classic-diagnostic-adapter+//comm-mbedtls/mbedtls-sys:build_script) bazel-out/k8-opt-exec-ST-db392155ee03/bin/external/rules_rust+/cargo/cargo_build_script_runner/runner ... (remaining 10 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Build script process failed with exit code 101
--stdout:
cargo:rerun-if-changed=wrapper.h
cargo:rerun-if-changed=patches
cargo:rerun-if-changed=csrc
cargo:rerun-if-changed=build.rs
cargo:rerun-if-env-changed=MBEDTLS_WRAPPER_H
--stderr:
Skipping source patches as requested by MBEDTLS_SKIP_PATCH=1
Skipping cmake/cc build as requested by MBEDTLS_SKIP_BUILD=1
/usr/include/limits.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
thread 'main' (133734) panicked at external/classic-diagnostic-adapter+/comm-mbedtls/mbedtls-sys/build.rs:192:10:
bindgen failed to generate bindings: ClangDiagnostic("/usr/include/limits.h:26:10: fatal error: 'bits/libc-header-start.h' file not found\n")
stack backtrace:
0: 0x581cff6a7145 - <<std[ea9f4e93d12430d]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[d1bb96a9607206a1]::fmt::Display>::fmt
1: 0x581cff6c1dd7 - core[d1bb96a9607206a1]::fmt::write
2: 0x581cff674e16 - std[ea9f4e93d12430d]::io::default_write_fmt::<std[ea9f4e93d12430d]::sys::stdio::unix::Stderr>
3: 0x581cff6a6e12 - <std[ea9f4e93d12430d]::sys::backtrace::BacktraceLock>::print
4: 0x581cff6883de - std[ea9f4e93d12430d]::panicking::default_hook::{closure#0}
5: 0x581cff689f68 - std[ea9f4e93d12430d]::panicking::default_hook
6: 0x581cff68a12e - std[ea9f4e93d12430d]::panicking::panic_with_hook
7: 0x581cff6a6b68 - std[ea9f4e93d12430d]::panicking::panic_handler::{closure#0}
8: 0x581cff6a3dd9 - std[ea9f4e93d12430d]::sys::backtrace::__rust_end_short_backtrace::<std[ea9f4e93d12430d]::panicking::panic_handler::{closure#0}, !>
9: 0x581cff68880d - __rustc[43d5200f6d4d6223]::rust_begin_unwind
10: 0x581cff0c867c - core[d1bb96a9607206a1]::panicking::panic_fmt
11: 0x581cff0c82a2 - core[d1bb96a9607206a1]::result::unwrap_failed
12: 0x581cff0ce252 - build_script_[5bd22ae657791950]::generate_bindings
13: 0x581cff0cfec6 - build_script_[5bd22ae657791950]::main
14: 0x581cff0d5bd3 - std[ea9f4e93d12430d]::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()>
15: 0x581cff0d4d79 - std[ea9f4e93d12430d]::rt::lang_start::<()>::{closure#0}
16: 0x581cff689b24 - std[ea9f4e93d12430d]::rt::lang_start_internal
17: 0x581cff0d1ca5 - main
18: 0x7e50f75331ca - __libc_start_call_main
at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
19: 0x7e50f753328b - __libc_start_main_impl
at ./csu/../csu/libc-start.c:360:3
20: 0x581cff0c874a - _start
21: 0x0 - <unknown>
Target @@classic-diagnostic-adapter+//cda-main:opensovd-cda failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 15.292s, Critical Path: 9.30s
INFO: 38 processes: 1284 action cache hit, 10 internal, 28 processwrapper-sandbox.
ERROR: Build did NOT complete successfully
There was a problem hiding this comment.
Slowly getting there 😅
that looks like the typical bindgen error I had when crosscompiling as well. can you try to set the env variable for the build script BINDGEN_SYSROOT=PATH/TO/SDK/SYSROOT. This causes bindgen to pass along the --sysroot={} flag to clang.
If that doesn't work / isn't feasible, the other option would be to check in the generated bindings here as a workaoround and skip the bindgen step.
There was a problem hiding this comment.
I got a green build now - but it involves again quite some "hammering" on the cross-build. Some of it in the toolchain setup at S-CORE, but - unfortunately - also some of it in the mbedtls-build-scripts. I need to clean this up and create a minimal variant so we can take a real look at it and decide what to do next.
However, the good news is: I think that the CDA itself is building just fine. "Just" the cross-build (not really unexpected), and especially cross-building mbedtls, seem problematic. But I guess with a bit of Bazel expertise, this can be cleaned up.
There was a problem hiding this comment.
thats amazing news! :)
if you have a branch / repo in score where the hammering is setup, i'm happy to also take a look at it and see if / how we can improve the cross compile support in the cda here :) While bazel is not my expertise I have quite a bit of history with cross compiling rust (and to parts c/c++) projects :D
There was a problem hiding this comment.
Done!
eclipse-score/reference_integration#240 contains the working example - without cross-compilation, though; that requires a special variant of the EB corbos Toolkit (it will contain mandatory fixes), which I have to release first.
However, even with this stripped-down example we're making good progress, I think!
|
|
||
| package(default_visibility = ["//visibility:public"]) | ||
|
|
||
| cmake( |
There was a problem hiding this comment.
I tried it out in eclipse-score/inc_diagnostics#6 but got Linux CI failure caused by an include path mismatch in the mbedtls foreign_cc build. CMAKE_C_FLAGS currently uses -I$$EXT_BUILD_ROOT$$/comm-mbedtls/mbedtls-sys/csrc, but in the sandbox execroot the header is actually under
There was a problem hiding this comment.
This heavily rings a bell of my own attempt: https://github.com/eclipse-score/reference_integration/pull/240/changes#diff-fa7479953709e4ac35300da1947c5184d0b3a540643e95a90adcb57d495b6907
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
add isolate=True for CDA crates universe Thanks @opajonk! Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Signed-off-by: Elena Gantner <elena.gantner@mercedes-benz.com>
Add missing cda-plugin-security dependency in cda-main, adjust ExtractHost import ordering for checkstyle, and configure reuse-annotate to skip unrecognized Bazel files.
Move com-params resolver usage to cda-main mdd module, add missing error docs and allow reasons, update FG operation tests to use ExtractHost, remove unsupported reuse-annotate arg, restore SPDX headers for Bazel files, and normalize trailing newline in bazel/rust_crate.bzl.
Exclude extensionless Bazel files from reuse-annotate, apply checkstyle formatting in config tests, and import resolve_com_params in mdd tests to fix unresolved symbol errors in Rust CI.
FScholPer
changed the title
Restore DB_HEALTH_COMPONENT_KEY used by cda-main and apply end-of-file fixer update for cda-plugin-runtime-update/BUILD.bazel.
github-actions
Bot
added
needs review
📊 Coverage Report (unit + integration)Total line coverage: 73.5% |
Exclude extensionless legal docs from reuse-annotate and align SPDX header format on Bazel files/workflow to match hook expectations.
1069757 to
04fad49
Compare
theswiftfox
added
the
don't merge yet
|
@FScholPer I'm not sure why there are code changes required beyond anything bazel related? Therefore setting this PR to draft again. I'd ask for any discussion / input to be done in the tracking issue. |
Sure but we should reuse https://github.com/eclipse-opensovd/classic-diagnostic-adapter/pull/348/changes#diff-22a6ee78ead32a4a9cca3c945e6bd9aea9db3d814bfb77d414b2245ff70f568d |
3 participants
Summary
This PR adds a basic bazel build for the cda. Some adaptions were made to the mbedtls build.rs to support building via bazel instead of during the rust build step.
Checklist
Related
Notes for Reviewers
Elena Gantner elena.gantner@mercedes-benz.com, Mercedes-Benz Tech Innovation GmbH
Provider Information