SNOW-917458 Snowflake JDBC drivers fails on JDK 21 with Arrow

[sfc-gh-wfateem]

Cherry picking fix for GH-35053 into Arrow v10.0.

Overview

SNOW-917458

Fixes #1512

[sfc-gh-wfateem]

Reviewing failures. I also need to amend the pom.xml for the FIPS version of the JDBC driver.

[sfc-gh-wfateem]

Ready for review.

[sfc-gh-igarish]

@sfc-gh-wfateem did you build the ARROW libs or downloaded from Apache site? We couldn't use directly from Apache site as it spill some logs when Arrow libs initialized and we had a regression in the past. So we have arrow repo in monorepo and build over there. Please check.

[sfc-gh-igarish]

@sfc-gh-skumbham please review it as you worked on revert arrow libs last month.

[sfc-gh-wfateem]

@sfc-gh-igarish I built the Arrow dependencies, but I just used the OSS GitHub repository. I'm not actually familiar with the issue you're referring to so I would need to sync up with someone about that. In any case, this looks like it will break support for JDK 8. In order to make this work I had to upgrade the Netty library version which seemed to introduce a crashing issue on a JDK 8 runtime environment:
netty/netty/#12119

We'll either need to hold off until there's a fix, or this would have to be a breaking change release. There has been a conversation about JDK 8 support since that's reached the end of public updates.

[sfc-gh-wfateem]

It looks like it's a problem on just some JDK 8 versions, but also in my case removing the following JVM argument addressed the issue:

 -XX:StartFlightRecording=name=background,maxsize=200m,filename=jfr_data.jfr

So it's still a breaking change. I think it's important that we support JDK 21, but we would have to make it abundantly clear that a new release may break JDK 8 runtime environments. Unless we want to wait for a fix.

I'm looking to see if there's a way around this somehow.

[sonarqubemergegate]

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[wendigo]

Any news?

[Komdosh]

Any updates?

[sfc-gh-wfateem]

There was a discussion around some security issues with other libraries and it was decided to upgrade Jackson separately in PR #1545 (SNOW-924672). This PR will just include the Arrow library upgrade.
@wendigo @Komdosh sorry for the delay, I was out for a few days. I'll get around to this by next week at the latest.

[wendigo]

@sfc-gh-wfateem any progress? :) 21.0.1 is out

[tpoll]

Any updates? Is there anything to community can do to help this progress?

[sajjoseph]

We are also impacted by this issue. Any quick updates?

[Komdosh]

@sfc-gh-wfateem
Any new updates?

[sfc-gh-wfateem]

@Komdosh @tpoll sorry for the delay. I've been out for a while and I had an unfortunate mishap with my laptop, so I only got a replacement recently.
The challenge is that we maintain our own Arrow library dependency because of some historical reason around logging, which I'm not fully aware of. That's probably something we want to explore to find out if that problem is still applicable or not with the latest Arrow library version, because if not then I don't see why we have to continue maintaining Arrow separately. That would also block the other initiative to release a non-shaded version of the JDBC driver.

The original PR I created included the open source Apache Arrow dependency while I should be building that using our version instead. I don't have access to that particular repository which is why I haven't been able to complete this PR as of yet. I'm working on either getting access to that repository or have someone else with access assist with building the required dependency.

I appreciate your patience here.

[sajjoseph]

Thanks @sfc-gh-wfateem.
I was able to manually build a version of snowflake-jdbc driver with your changes in and it is working in JDK 21.

[wendigo]

@sfc-gh-wfateem do you plan to merge and release this change anytime soon?

[sonarqubemergegate]

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[sfc-gh-ext-simba-jl]

LGTM

[sfc-gh-igarish]

LGTM

[sfc-gh-wfateem]

@wendigo thanks for your patience.
The fix is now merged and that should be included in the next release. I think that should roughly be the first half of December.