Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setup-gradle with gradle 7.x fails to replace environment variables in toolchains.xml #511

Closed
asmundh opened this issue Jan 14, 2025 · 8 comments · Fixed by #518
Closed
Milestone

Comments

@asmundh
Copy link

asmundh commented Jan 14, 2025

Figured I'd move this to its own issue. We are experiencing issues when bumping setup-gradle from V2 (gradle-buld-action@v2) to v4 (also tested with v3). The version bump makes our gradle workflow several minutes slower.

The problem seemingly occurs here. See below for outputs from our workflow.

Specifically, when running run ./gradlew build after setup-gradle@v4 gradle logs that it cannot find certain directories because they are named in the toolchains.xml-file after running setup-gradle. However, after setup-java has ran, only JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64 and JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64 are set.

Starting a Gradle Daemon (subsequent builds will be faster)
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_17_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_21_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_8_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_11_X64}' (Maven Toolchains) used for java installations does not exist

Hi, @bigdaz. We are experiencing the same issues @TWiStErRob was. I will answer your questions to him with my own configuration and findings. Judging by my toolchains.xml-file after the setup-gradle step it seems the action overrides its content with a bad configuration (possibly when it logs Merged default JDK locations into /home/runner/.m2/toolchains.xml).

Let me know if you want me to address this in a new issue.

Does the setup-java step precede the setup-gradle step in your workflow?
Yes. See the output of setup-java below.

Output from setup-java
Run actions/setup-java@v4
  with:
    java-version: 17
    distribution: temurin
    java-package: jdk
    check-latest: false
    server-id: github
    server-username: GITHUB_ACTOR
    server-password: GITHUB_TOKEN
    overwrite-settings: true
    job-status: success
    token: ***
Installed distributions
  Resolved Java 17.0.13+11 from tool-cache
  Setting Java 17.0.13+11 as the default
  Creating toolchains.xml for JDK version 17 from temurin
  Writing to /home/runner/.m2/toolchains.xml
  
  Java configuration:
    Distribution: temurin
    Version: 17.0.13+11
    Path: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
  
Creating settings.xml with server-id: github
Writing to /home/runner/.m2/settings.xml

The warnings you see: are they emitted by the setup-gradle step, or by a later step that executes a Gradle build?

They are logged by my subsequent ./gradlew build. See output of setup-gradle below.

setup-gradle output
Run gradle/actions/setup-gradle@v4.2.2
  with:
    cache-disabled: false
    cache-read-only: true
    cache-write-only: false
    cache-overwrite-existing: false
    cache-cleanup: on-success
    gradle-home-cache-includes: caches
  notifications
  
    add-job-summary: always
    add-job-summary-as-pr-comment: never
    dependency-graph: disabled
    dependency-graph-report-dir: dependency-graph-reports
    dependency-graph-continue-on-failure: true
    build-scan-publish: false
    validate-wrappers: true
    allow-snapshot-wrappers: false
    gradle-home-cache-strict-match: false
    workflow-job-context: null
    github-token: ***
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
    JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
Merged default JDK locations into /home/runner/.m2/toolchains.xml
Restore Gradle state from cache
  Gradle User Home cache not found. Will initialize empty.
All Gradle Wrapper jars are valid
  ✓ Found known Gradle Wrapper JAR files:
    14dfa961b6704bb3decdea06502781edaa796a82e6da41cd2e1962b14fbe21a3 gradle/wrapper/gradle-wrapper.jar

Are the referenced environment variables (eg JAVA_HOME_21_X64) defined at the point the warnings are emitted, and do they provide an absolute path to an installed JDK?

Yes. See the below environment variables for my run: ./gradlew build

./gradlew initial output
Run ./gradlew $gradle_operations
  ./gradlew $gradle_operations
  shell: /usr/bin/bash -e {0}
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
    JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
    GRADLE_ACTION_ID: gradle/actions/setup-gradle
    GRADLE_USER_HOME: /home/runner/.gradle
    GRADLE_BUILD_ACTION_SETUP_COMPLETED: true
    GRADLE_BUILD_ACTION_CACHE_RESTORED: true
    DEVELOCITY_INJECTION_INIT_SCRIPT_NAME: gradle-actions.inject-develocity.init.gradle
    DEVELOCITY_AUTO_INJECTION_CUSTOM_VALUE: gradle-actions
    GITHUB_DEPENDENCY_GRAPH_ENABLED: false
    REPO: redacted
    gradle_operations: build
Downloading https://services.gradle.org/distributions/gradle-7.6.4-bin.zip
...........10%............20%............30%...........40%............50%............60%...........70%............80%............90%............100%
Welcome to Gradle 7.6.4!
Here are the highlights of this release:
 - Added support for Java 19.
 - Introduced `--rerun` flag for individual task rerun.
 - Improved dependency block for test suites to be strongly typed.
 - Added a pluggable system for Java toolchains provisioning.
For more details see https://docs.gradle.org/7.6.4/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_11_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_17_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_8_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_21_X64}' (Maven Toolchains) used for java installations does not exist
Task :checkKotlinGradlePluginConfigurationErrors

What is the content of /home/runner/.m2/toolchains.xml both before and after the setup-gradle step?

toolchains.xml before
<toolchains xmlns="http://maven.apache.org/TOOLCHAINS/1.1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/TOOLCHAINS/1.1.0 https://maven.apache.org/xsd/toolchains-1.1.0.xsd">
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>17</version>
      <vendor>temurin</vendor>
      <id>temurin_17</id>
    </provides>
    <configuration>
      <jdkHome>/opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64</jdkHome>
    </configuration>
  </toolchain>
</toolchains>
toolchains.xml after
<!-- JDK Toolchains installed by default on GitHub-hosted runners -->
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>21</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_21_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>8</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_8_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>11</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_11_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>17</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_17_X64}</jdkHome>
    </configuration>
  </toolchain>
</toolchains>

Originally posted by @asmundh in #89

@bigdaz
Copy link
Member

bigdaz commented Jan 14, 2025

Does the "after" version of toolchains.xml really look like this, without the opening <toolchains> element?

<!-- JDK Toolchains installed by default on GitHub-hosted runners -->
  <toolchain>
  </toolchain>
</toolchains>

If so, this is clearly invalid XML and I would expect a different error message to result.

It's also hard to understand how the different entries like env.JAVA_HOME_11_X64 are generated if these environment variables are not set on the machine. This is the code. It would appear that the process.env call is indeed listing these values, since they are not hard-coded anywhere.

@asmundh
Copy link
Author

asmundh commented Jan 15, 2025

I'm sorry, I re-checked my logs and the blank line made me misread the file! The actual file looks like below. setup-gradle has indeed just added more <toolchain>-elements without disrupting what was already there.

In other words, it's working as intended. Should we simply ignore the lines about missing directories?

<toolchains xmlns="http://maven.apache.org/TOOLCHAINS/1.1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/TOOLCHAINS/1.1.0 https://maven.apache.org/xsd/toolchains-1.1.0.xsd">
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>17</version>
      <vendor>temurin</vendor>
      <id>temurin_17</id>
    </provides>
    <configuration>
      <jdkHome>/opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64</jdkHome>
    </configuration>
  </toolchain>

<!-- JDK Toolchains installed by default on GitHub-hosted runners -->
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>11</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_11_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>21</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_21_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>8</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_8_X64}</jdkHome>
    </configuration>
  </toolchain>
  <toolchain>
    <type>jdk</type>
    <provides>
      <version>17</version>
    </provides>
    <configuration>
      <jdkHome>${env.JAVA_HOME_17_X64}</jdkHome>
    </configuration>
  </toolchain>
</toolchains>

@bigdaz
Copy link
Member

bigdaz commented Jan 15, 2025

The only way I can see this happening is:

  • When setup-gradle runs, it detects these 4 environment variables: JAVA_HOME_11_X64, JAVA_HOME_21_X64, JAVA_HOME_8_X64, JAVA_HOME_17_X64
  • When Gradle executes, either these environment variables do not exist, or they point to non-existent files.

This is odd, because I'd expect the same environment variables to exist in both cases.

Can you please check a few things?

  • Add a separate step that executes env | grep '^JAVA' to inspect the environment variables available on the runner
  • Try using the more-standard shell: bash in the command that runs ./gradlew

Thanks!

@asmundh
Copy link
Author

asmundh commented Jan 16, 2025

When the workflow starts it has the following JAVA-environment variables. It stays unchanged through both setup-java and setup-gradle. The workflow runs on Ubuntu 24 (details below).

Run env | grep '^JAVA'
  env | grep '^JAVA'
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
JAVA_HOME_11_X64=/usr/lib/jvm/temurin-11-jdk-amd64
JAVA_HOME=/usr/lib/jvm/temurin-17-jdk-amd64
JAVA_HOME_8_X64=/usr/lib/jvm/temurin-8-jdk-amd64
JAVA_HOME_21_X64=/usr/lib/jvm/temurin-21-jdk-amd64
JAVA_HOME_17_X64=/usr/lib/jvm/temurin-17-jdk-amd64
Current runner version: '2.321.0'
Operating System
  Ubuntu
  24.04.1
  LTS
Runner Image
  Image: ubuntu-24.04
  Version: 20250105.1.0
  Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250105.1/images/ubuntu/Ubuntu2404-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250105.1
Runner Image Provisioner
  2.0.414.1

Using shell: bash does not appear to change the execution output of ./gradlew.

Run ./gradlew $gradle_operations
  ./gradlew $gradle_operations
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
    JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.13-11/x64
    GRADLE_ACTION_ID: gradle/actions/setup-gradle
    GRADLE_USER_HOME: /home/runner/.gradle
    GRADLE_BUILD_ACTION_SETUP_COMPLETED: true
    GRADLE_BUILD_ACTION_CACHE_RESTORED: true
    DEVELOCITY_INJECTION_INIT_SCRIPT_NAME: gradle-actions.inject-develocity.init.gradle
    DEVELOCITY_AUTO_INJECTION_CUSTOM_VALUE: gradle-actions
    GITHUB_DEPENDENCY_GRAPH_ENABLED: false
    REPO: ****
    NEXUS_USERNAME: ***
    NEXUS_PASSWORD: ***
    gradle_operations: test build
Downloading https://services.gradle.org/distributions/gradle-7.6.4-bin.zip
...........10%............20%............30%...........40%............50%............60%...........70%............80%............90%............100%
Welcome to Gradle 7.6.4!
Here are the highlights of this release:
 - Added support for Java 19.
 - Introduced `--rerun` flag for individual task rerun.
 - Improved dependency block for test suites to be strongly typed.
 - Added a pluggable system for Java toolchains provisioning.
For more details see https://docs.gradle.org/7.6.4/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_11_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_8_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_17_X64}' (Maven Toolchains) used for java installations does not exist
Directory '/home/runner/.gradle/daemon/7.6.4/${env.JAVA_HOME_21_X64}' (Maven Toolchains) used for java installations does not exist
> Task :checkKotlinGradlePluginConfigurationErrors
> Task :processResources
> Task :kaptGenerateStubsKotlin
...

Output of ls -la /usr/lib/jvm below. It has no folders in it and

total 40
drwxrwxrwx   6 root root 4096 Jan  5 21:34 .
drwxr-xr-x 109 root root 4096 Jan  5 21:41 ..
-rwxrwxrwx   1 root root 2071 Dec 20 11:53 .temurin-11-jdk-amd64.jinfo
-rwxrwxrwx   1 root root 1805 Dec 20 15:19 .temurin-17-jdk-amd64.jinfo
-rwxrwxrwx   1 root root 1871 Dec 19 00:00 .temurin-21-jdk-amd64.jinfo
-rwxrwxrwx   1 root root 2573 Dec 20 11:22 .temurin-8-jdk-amd64.jinfo
drwxrwxrwx   9 root root 4096 Jan  5 21:33 temurin-11-jdk-amd64
drwxrwxrwx   9 root root 4096 Jan  5 21:34 temurin-17-jdk-amd64
drwxrwxrwx   9 root root 4096 Jan  5 21:34 temurin-21-jdk-amd64
drwxrwxrwx   8 root root 4096 Jan  5 21:33 temurin-8-jdk-amd64

output of ls -la /home/runner/.gradle/ before running ./gradlew below.

total 16
drwxr-xr-x  4 runner docker 4096 Jan 16 11:15 .
drwxr-x--- 16 runner docker 4096 Jan 16 11:15 ..
drwxr-xr-x  2 runner docker 4096 Jan 16 11:15 .setup-gradle
drwxr-xr-x  2 runner docker 4096 Jan 16 11:15 init.d

After running ./gradlew

ls -la /home/runner/.gradle/daemon
total 12
drwxr-xr-x  3 runner docker 4096 Jan 16 11:25 .
drwxr-xr-x 12 runner docker 4096 Jan 16 11:26 ..
drwx------  2 runner docker 4096 Jan 16 11:25 7.6.4

ls -la /home/runner/.gradle/daemon/7.6.4
total 624
drwx------ 2 runner docker   4096 Jan 16 11:25 .
drwxr-xr-x 3 runner docker   4096 Jan 16 11:25 ..
-rw------- 1 runner docker 620194 Jan 16 11:29 daemon-1841.out.log
-rw------- 1 runner docker    640 Jan 16 11:29 registry.bin
-rw-r--r-- 1 runner docker     17 Jan 16 11:29 registry.bin.lock

@bigdaz
Copy link
Member

bigdaz commented Jan 18, 2025

Thanks for your detailed reporting. For some reason, it seems that Gradle is unable to resolve the environment variables that are encoded in toolchains.xml. I can't really explain why.

I've put together a setup-gradle version that should avoid this problem, by encoding the JDK paths directly in toolchains.xml (rather than encoding these as ${env.JAVA_HOME...}). Can you please try it out in your workflow? Just use the following action declaration:

gradle/actions/setup-gradle@release/toolchains-xml

I've tested using this workflow and it appears to be detecting toolchains as expected. If this fix resolves your problem I'll definitely merge these changes and release. If not, the messages you see should give more clarity into what's going on.

@asmundh
Copy link
Author

asmundh commented Jan 20, 2025

I tried using gradle/actions/setup-gradle@release/toolchains-xml and indeed it did not print the lines!

I also just now tried upgrading to use gradle 8.12 instead of 7.6.4, which also seems to resolve the issue, even when using v4 of the setup-gradle action.

For posterity and in case others face the same issue, should I update the title of this issue?

@bigdaz
Copy link
Member

bigdaz commented Jan 20, 2025

@asmundh Yes, please update the title to accurately reflect the issue. I guess Gradle 7.x may not be correctly replacing environment variables in toolchains.xml.

Thanks for reporting that the fix works for you. To avoid other Gradle 7.x users hitting this issue, I'll merge the fix into main and it will be included in the next release.

@asmundh asmundh changed the title setup-gradle overrides toolchains.xml with non-working JDK locations setup-gradle with gradle 7.x fails to replace environment variables in toolchains.xml Jan 21, 2025
@asmundh
Copy link
Author

asmundh commented Jan 21, 2025

Great, thank you for the help!

bigdaz added a commit that referenced this issue Jan 21, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Fixes #511
@bigdaz bigdaz added this to the v4.3.0 milestone Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants