Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-523: [Release] Add support for building .jar including JNI binaries #517

Merged
merged 7 commits into from
Jan 16, 2025

Conversation

kou
Copy link
Member

@kou kou commented Jan 15, 2025

Fixes GH-523.

Merge test_jni.yml to rc.yml so that .jar including JNI binaries can be built and uploaded to GitHub Releases for RC.

ccache support is also enabled for fast CI.

@kou kou force-pushed the ci-rc-jni branch 2 times, most recently from 2b8fe63 to 76d58fc Compare January 15, 2025 01:11
tar -xf apache-arrow-java-*.tar.gz --strip-components=1
- name: Download the latest Apache Arrow C++
run: |
ci/scripts/download_cpp.sh
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to use the main branch instead of related Apache Arrow C++ on daily run. But it's out-of-scope in this PR.

- name: Extract Download the latest Apache Arrow C++
run: |
tar -xf apache-arrow-java-*.tar.gz --strip-components=1
# We need 19.0.0 for latest Boost support
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload:
name: Upload
if: github.ref_type == 'tag'
# if: github.ref_type == 'tag'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revert this after I confirmed that this job works.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- name: Upload
run: |
# GH-499: How to create release notes?
version=${GITHUB_REF_NAME%-rc*}
version=${version#v}
rc=${GITHUB_REF_NAME#*-rc}
gh release create ${GITHUB_REF_NAME} \
echo gh release create ${GITHUB_REF_NAME} \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revert this after I confirmed that this job works.

Comment on lines +109 to +118
- name: Checkout apache/arrow-testing
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: apache/arrow-testing
path: arrow/testing
- name: Checkout apache/parquet-testing
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: apache/parquet-testing
path: arrow/cpp/submodules/parquet-testing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need testing data? (I don't think we need to run C++ tests here?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I'll disable C++ tests.
(I thought that we need to run C++ tests because it seems that we don't have enough test for JNI related codes.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The C++ tests we run here are just the same tests that apache/arrow already runs right? (I agree we need more testing for the JNI code itself, though...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right but we use different build options that aren't covered by apache/arrow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm if they're that different then should we be testing the config upstream? Or otherwise in that case I don't mind running the C++ tests here if our config is very different (it mostly just adds a lot of build time)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Let's do it in upstream.

kou added 6 commits January 15, 2025 12:51
…binaries

Fixes apacheGH-500.

Merge `test_jni.yml` to `rc.yml` so that .jar including JNI binaries
can be released.

ccache support is also enabled for fast CI.

Verified

This commit was signed with the committer’s verified signature.
Preston-Landers Preston Landers

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@kou
Copy link
Member Author

kou commented Jan 15, 2025

CI passed. This is ready.

@kou kou changed the title GH-500: [Release] Add support for releasing .jar including JNI binaries GH-500: [Release] Add support for building .jar including JNI binaries Jan 15, 2025
@kou kou changed the title GH-500: [Release] Add support for building .jar including JNI binaries GH-523: [Release] Add support for building .jar including JNI binaries Jan 15, 2025
@kou
Copy link
Member Author

kou commented Jan 15, 2025

I'll merge this today.

I'll work on the followings as separated tasks:

  • Add support for building .jar for Windows
  • Sign uploaded .jar and related files
  • Upload signed .jar and related files to staging
  • Publish files on staging after release vote passed

@kou kou merged commit 6e3c8ba into apache:main Jan 16, 2025
22 checks passed
@kou kou deleted the ci-rc-jni branch January 16, 2025 00:14
@lidavidm lidavidm added this to the 18.2.0 milestone Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Release] Add support for building .jar including JNI binaries
2 participants