OCI multi-arch image transitions breaking springboot rule

[code-weirdo]

Hi,

I've been trying to work out how to generate multi-arch oci images for a Spring Boot application in Bazel, and I've run into what looks like an incompatibility between rules_spring and the way in which rules_oci recommends setting up multi-arch images (using transitions). As soon as I enable the transitions with the springboot rule, I get the following error:

bazel-springboot-multiarch-oci % bazel build //...
DEBUG: /private/var/tmp/_bazel_nicholas/21415562e9c8d9edcc79611fff0f1338/external/rules_jvm_external~/private/extensions/maven.bzl:155:14: The maven repository 'maven' is used in two different bazel modules, originally in '' and now in 'protobuf'
INFO: Analyzed 18 targets (169 packages loaded, 3692 targets configured).
INFO: Found 18 targets...
INFO: Elapsed time: 6.063s, Critical Path: 4.47s
INFO: 15 processes: 8 internal, 6 darwin-sandbox, 1 local.
INFO: Build completed successfully, 15 total actions
nicholas@C5XPV6FG90 bzl-test % bazel build //...
DEBUG: /private/var/tmp/_bazel_nicholas/21415562e9c8d9edcc79611fff0f1338/external/rules_jvm_external~/private/extensions/maven.bzl:155:14: The maven repository 'maven' is used in two different bazel modules, originally in '' and now in 'protobuf'
INFO: Analyzed 19 targets (76 packages loaded, 2289 targets configured).
ERROR: /Users/nicholas/projects/bazel-springboot-multiarch-oci/BUILD.bazel:22:11: Executing genrule //:_genmanifest failed: (Exit 1): bash failed: error executing Genrule command (from target //:_genmanifest) /bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux_aarch64/bin/jar: cannot execute binary file
ERROR: //springboot/write_manifest.sh could not find the spring-boot jar
ERROR: /Users/nicholas/projects/bazel-springboot-multiarch-oci/BUILD.bazel:22:11: Executing genrule //:_genmanifest failed: (Exit 1): bash failed: error executing Genrule command (from target //:_genmanifest) /bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
external/rules_spring/springboot/write_manifest.sh: line 24: external/rules_java~~toolchains~remotejdk21_linux/bin/jar: cannot execute binary file
ERROR: //springboot/write_manifest.sh could not find the spring-boot jar
INFO: Elapsed time: 0.556s, Critical Path: 0.15s
INFO: 7 processes: 7 internal.
ERROR: Build did NOT complete successfully

I'm pretty new to Bazel, so I don't fully understand how transitions affect things, but it seems that multi-arch images should be something common enough that they should be supported. I've also tested out the multi-arch builds with Java only, which seems to work fine and leads me to believe that the issue I'm seeing is isolated to the transition breaking something in rules_spring.

I've set up a workspace with the simplest Java application to prove out that I could build multi-arch Java images here, with two branches.

The java-multiarch branch has a working basic Hello World Java application that gets built with java_binary, and then gets wrapped up in a multi-arch oci image and that seems to work correctly.

In the springboot branch, I've converted the application to a simple Spring Boot application, which I can successfully containerise, however, the wheels fall off as soon as I enable the transition to turn it into a multi-arch image (by uncommenting the mult-arch block in BUILD.bazel).

As I say, I'm fairly new to Bazel so this could just be something I'm doing wrong but I'm not quite sure how to debug this further.

Any help would really be appreciated.

Regards
Nic

[plaird]

Hi Nic,

Thanks for reporting. The bad news, you probably lost a few hours to this. The good news, I have the fix for you and yes it works (I tested in your fork).

Please add this attribute to your springboot() rule to override the default:

jartools_toolchains = ["@bazel_tools//tools/jdk:current_host_java_runtime"],

We do multi-arch internally, and I wondered if anyone else was going to have this problem. I was stubborn and didn't make this the default because it isn't hermetic and that annoys me. But I think multi-arch use case is becoming more common so I should set the default to be ready for that.

For posterity, the default at time of reporting was:

jartools_toolchains = ["@bazel_tools//tools/jdk:current_java_runtime"],

[code-weirdo]

Hi @plaird,

Wow, that was quick. Thanks for taking a look.

Just for my own edification, what is it about using the remote toolchain that causes the issue? Is it something that can be remediated in the longer term, or is it a limitation we're going to have to live with? Keeping things hermetic would obviously be a win.

[plaird]

Great, hopefully you are back in business.

what is it about using the remote toolchain that causes the issue

The runtime JDK, which was the default, is the one that matches the target architecture. But the jar utility needs to run on your host, so jar needs to be compiled to the host architecture.

The reason I left it like that is it is just my understanding that the host java toolchain can in some cases use the JDK installed on your machine, outside of the ones managed by Bazel. This may not actually be true, or not true in most cases, but I am not enough of an expert to know. I have found the evolution of java toolchain changes in Bazel over time to be confusing.

In reality, the implementation of jar (which is what we use this for) is pretty stable across time, so not a big concern regardless. I was just being pedantic.

[code-weirdo]

Great, hopefully you are back in business.

Yep, thanks. Spent the morning testing it all out and it's working well now. I really appreciate the help on that!

To be fair, most things Bazel are confusing, especially to someone new to it :) Your explanation makes sense to me though. Thanks again.