GH-15265: [Java] Publish SBOM artifacts #15267
Conversation
|
|
|
Should the sbom be published with the nightlies? |
|
Could you also update diff --git a/ci/scripts/java_full_build.sh b/ci/scripts/java_full_build.sh
index 1c07971bcc..2734f3e9db 100755
--- a/ci/scripts/java_full_build.sh
+++ b/ci/scripts/java_full_build.sh
@@ -65,7 +65,13 @@ find . \
-exec echo {} ";" \
-exec cp {} $dist_dir ";"
find ~/.m2/repository/org/apache/arrow \
- "(" -name "*.jar" -o -name "*.zip" -o -name "*.pom" ")" \
+ "(" \
+ -name "*.jar" -o \
+ -name "*.json" -o \
+ -name "*.pom" -o \
+ -name "*.xml" -o \
+ -name "*.zip" \
+ ")" \
-exec echo {} ";" \
-exec cp {} $dist_dir ";"
diff --git a/dev/tasks/java-jars/github.yml b/dev/tasks/java-jars/github.yml
index cfa1dbed49..c42c9e6f4d 100644
--- a/dev/tasks/java-jars/github.yml
+++ b/dev/tasks/java-jars/github.yml
@@ -204,5 +204,7 @@ jobs:
$GITHUB_WORKSPACE/arrow \
$GITHUB_WORKSPACE/arrow/java-dist
{{ macros.github_upload_releases(["arrow/java-dist/*.jar",
+ "arrow/java-dist/*.pson",
"arrow/java-dist/*.pom",
+ "arrow/java-dist/*.xml",
"arrow/java-dist/*.zip"])|indent }}
diff --git a/dev/tasks/tasks.yml b/dev/tasks/tasks.yml
index 66df61e215..8437ad0778 100644
--- a/dev/tasks/tasks.yml
+++ b/dev/tasks/tasks.yml
@@ -801,6 +801,9 @@ tasks:
ci: github
template: java-jars/github.yml
artifacts:
+ - arrow-algorithm-{no_rc_snapshot_version}-cyclonedx.json
+ - arrow-algorithm-{no_rc_snapshot_version}-cyclonedx.xml
+ - arrow-algorithm-{no_rc_snapshot_version}-javadoc.jar
- arrow-algorithm-{no_rc_snapshot_version}-javadoc.jar
- arrow-algorithm-{no_rc_snapshot_version}-sources.jar
- arrow-algorithm-{no_rc_snapshot_version}-tests.jar
# NOTE!!! We need to add more entries for *-cyclonedx.{json,xml} |
|
Thank you so much, @assignUser and @kou . I addressed your comments. |
ci/scripts/java_full_build.sh
Outdated
| -name "*.pom" -o \ | ||
| -name "*.xml" -o \ | ||
| -name "*.zip" \ | ||
| ")" \ | ||
| -exec echo {} ";" \ | ||
| -exec cp {} $dist_dir ";" | ||
| find ~/.m2/repository/org/apache/arrow \ |
There was a problem hiding this comment.
Hmm, I think the previous comment diff points to L73 instead of L63?
diff --git a/ci/scripts/java_full_build.sh b/ci/scripts/java_full_build.sh
index 1c07971bcc..2734f3e9db 100755
--- a/ci/scripts/java_full_build.sh
+++ b/ci/scripts/java_full_build.sh
@@ -65,7 +65,13 @@ find . \
-exec echo {} ";" \
-exec cp {} $dist_dir ";"
find ~/.m2/repository/org/apache/arrow \
- "(" -name "*.jar" -o -name "*.zip" -o -name "*.pom" ")" \
+ "(" \
+ -name "*.jar" -o \
+ -name "*.json" -o \
+ -name "*.pom" -o \
+ -name "*.xml" -o \
+ -name "*.zip" \
+ ")" \
-exec echo {} ";" \
-exec cp {} $dist_dir ";"I.e., the one finding the files in local repo dir.
There was a problem hiding this comment.
Oh, you are right. I'll fix it soon.
| @@ -801,6 +801,8 @@ tasks: | |||
| ci: github | |||
| template: java-jars/github.yml | |||
| artifacts: | |||
| - arrow-algorithm-{no_rc_snapshot_version}-cyclonedx.json | |||
| - arrow-algorithm-{no_rc_snapshot_version}-cyclonedx.xml | |||
There was a problem hiding this comment.
Oh, do we need to repeat this for all jars like arrow-avro/arrow-c-data/arrow-compression/...?
There was a problem hiding this comment.
I think so. That is why @kou said the diff is incomplete for dev/tasks/tasks.yml and there is a # NOTE!!! We need to add more entries for *-cyclonedx.{json,xml}.
There was a problem hiding this comment.
Got it. Thank you for the confirmation.
There was a problem hiding this comment.
I added them all at de44a62
|
@github-actions crossbow submit java-jars |
|
To have them uploaded to nightlies.apache.org the nightly java job needs to be updated to: https://github.com/apache/arrow/blob/master/.github/workflows/java_nightly.yml#L110 |
|
Revision: de44a62 Submitted crossbow builds: ursacomputing/crossbow @ actions-588eb7d929
|
|
Thank you, @assignUser . |
|
The python failure looks irrelevant to this Java plugin PR. Please let me know if there is some thing I need to do more, @assignUser , @kou , @viirya . Thank you in advance. |
|
Yeah the failure of the crossbow job is due to C++ stuff but it also means that the changed logic won't be tested. As this won't succeed and the changes from this PR wont be used (unless built locally ofc) until that is fixed I would prefer to wait for #25633 to be closed so we can properly test this. For the SBOM files to be uploaded with the official releases we will also need to update https://github.com/apache/arrow/blob/-/dev/release/06-java-upload.sh which could be done as a follow up if the creation of the sbom artifacts for local builds is important for you. |
It seems that we don't need it. https://github.com/apache/arrow/blob/master/dev/release/06-java-upload.sh#L123 will collect new files. |
|
According to the above comment about |
|
Please let me know if there is something I can help, @assignUser . |
|
@dongjoon-hyun please rebase, the fix for the c++ issue was merged. Afterwards please trigger the crossbow with |
|
Once the job complets succesfully you can take a look at the artifacts if they correctly contian the sbom. I will merge then and it will be part of 11.0.0 |
|
Got it, @assignUser ! |
dongjoon-hyun
requested review from
assignUser,
kou,
raulcd and
lidavidm
as code owners
|
@github-actions crossbow submit java-jars |
|
Revision: 97af0c9 Submitted crossbow builds: ursacomputing/crossbow @ actions-18dd62a120
|
There was a problem hiding this comment.
SBOM artifacts are uploaded to releases and will thus be picked up by the release scripts https://github.com/ursacomputing/crossbow/releases/tag/actions-18dd62a120-github-java-jars
|
Benchmark runs are scheduled for baseline = 641d1da and contender = 5580f27. 5580f27 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
This closes #15265