google collections issue still present in 10.12.7

[hazendaz]

See https://github.com/spotbugs/spotbugs/

Execution failed for task ':eclipsePlugin-junit:checkstyleTest'.

Could not resolve all files for configuration ':eclipsePlugin-junit:checkstyle'.
Could not resolve com.google.guava:guava:33.0.0-jre.
Required by:
project :eclipsePlugin-junit > com.puppycrawl.tools:checkstyle:10.12.7
> Module 'com.google.guava:guava' has been rejected:
Cannot select module with conflict on capability 'com.google.collections:google-collections:33.0.0-jre' also provided by [com.google.collections:google-collections:1.0(runtime)]
Could not resolve com.google.collections:google-collections:1.0.
Required by:
project :eclipsePlugin-junit > com.puppycrawl.tools:checkstyle:10.12.7 > org.apache.maven.doxia:doxia-core:1.12.0 > org.codehaus.plexus:plexus-container-default:2.1.0
> Module 'com.google.collections:google-collections' has been rejected:
Cannot select module with conflict on capability 'com.google.collections:google-collections:1.0' also provided by [com.google.guava:guava:33.0.0-jre(jreRuntimeElements)]

[hazendaz]

Fix #14140 did not solve.

[rnveach]

Did we ever identify why CI isn't finding this issue?

[romani]

CI job to catch this is disabled until open issue, there was some CI/infra problem that forced us to disable and we never comeback.

If we fix #12558 it will never leak again

[romani]

@XN137, did this resolve issue for you? others report that there is still issue

[XN137]

did this resolve issue for you? others report that there is still issue

no, it did not.
i was relying on the output of mvn dependency:tree to confirm the dependency was gone, but looks like this is not to be trusted?!

looking at https://repo1.maven.org/maven2/com/puppycrawl/tools/checkstyle/10.12.7/checkstyle-10.12.7.pom

we can see the exclusion being present:

    <dependency>
      <groupId>org.apache.maven.doxia</groupId>
      <artifactId>doxia-core</artifactId>
      <version>${doxia.version}</version>
      <exclusions>
        <!-- Excluded due to security vulnerability in version 1.11
             https://github.com/checkstyle/checkstyle/pull/13357#issuecomment-1639480844 -->
        <exclusion>
          <groupId>commons-codec</groupId>
          <artifactId>commons-codec</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.google.collections</groupId>
          <artifactId>google-collections</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

so lets try to use checkstyle from a simple maven project:

➜  cat pom.xml
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
     
      <groupId>com.mycompany.app</groupId>
      <artifactId>my-app</artifactId>
      <version>1.0-SNAPSHOT</version>
     
      <name>my-app</name>
     
      <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
      </properties>
     
      <dependencies>
        <dependency>
          <groupId>com.puppycrawl.tools</groupId>
          <artifactId>checkstyle</artifactId>
          <version>10.12.7</version>
        </dependency>
      </dependencies>
     
    </project>

➜  ll ~/.m2/repository/com/google
total 0

➜  mvn clean install >/dev/null 2>&1
                      
➜  ll ~/.m2/repository/com/google   
total 24K
drwxrwxr-x 3 user user 4,0K Jan  6 10:38 code
drwxrwxr-x 3 user user 4,0K Jan  6 10:38 collections
drwxrwxr-x 4 user user 4,0K Jan  6 10:38 errorprone
drwxrwxr-x 3 user user 4,0K Jan  6 10:38 google
drwxrwxr-x 6 user user 4,0K Jan  6 10:38 guava
drwxrwxr-x 3 user user 4,0K Jan  6 10:38 j2objc

➜  mvn --no-snapshot-updates dependency:tree
[INFO] Scanning for projects...
[INFO] 
[INFO] ----------------------< com.mycompany.app:my-app >----------------------
[INFO] Building my-app 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- dependency:2.8:tree (default-cli) @ my-app ---
[INFO] com.mycompany.app:my-app:jar:1.0-SNAPSHOT
[INFO] \- com.puppycrawl.tools:checkstyle:jar:10.12.7:compile
[INFO]    +- info.picocli:picocli:jar:4.7.5:compile
[INFO]    +- org.antlr:antlr4-runtime:jar:4.13.1:compile
[INFO]    +- commons-beanutils:commons-beanutils:jar:1.9.4:compile
[INFO]    |  +- commons-logging:commons-logging:jar:1.2:compile
[INFO]    |  \- commons-collections:commons-collections:jar:3.2.2:compile
[INFO]    +- com.google.guava:guava:jar:33.0.0-jre:compile
[INFO]    |  +- com.google.guava:failureaccess:jar:1.0.2:compile
[INFO]    |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO]    |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO]    |  +- com.google.errorprone:error_prone_annotations:jar:2.23.0:compile
[INFO]    |  \- com.google.j2objc:j2objc-annotations:jar:2.8:compile
[INFO]    +- org.reflections:reflections:jar:0.10.2:compile
[INFO]    |  +- org.javassist:javassist:jar:3.28.0-GA:compile
[INFO]    |  \- org.slf4j:slf4j-api:jar:1.7.32:compile
[INFO]    +- net.sf.saxon:Saxon-HE:jar:12.4:compile
[INFO]    |  +- org.xmlresolver:xmlresolver:jar:5.2.2:compile
[INFO]    |  |  +- org.apache.httpcomponents.client5:httpclient5:jar:5.1.3:runtime
[INFO]    |  |  |  +- org.apache.httpcomponents.core5:httpcore5-h2:jar:5.1.3:runtime
[INFO]    |  |  |  \- commons-codec:commons-codec:jar:1.15:runtime
[INFO]    |  |  \- org.apache.httpcomponents.core5:httpcore5:jar:5.1.3:runtime
[INFO]    |  \- org.xmlresolver:xmlresolver:jar:data:5.2.2:compile
[INFO]    +- org.checkerframework:checker-qual:jar:3.42.0:compile
[INFO]    +- org.apache.maven.doxia:doxia-core:jar:1.12.0:compile
[INFO]    |  +- org.apache.maven.doxia:doxia-sink-api:jar:1.12.0:compile
[INFO]    |  +- org.apache.maven.doxia:doxia-logging-api:jar:1.12.0:compile
[INFO]    |  +- org.codehaus.plexus:plexus-utils:jar:3.3.0:compile
[INFO]    |  +- org.codehaus.plexus:plexus-container-default:jar:2.1.0:compile
[INFO]    |  |  +- org.codehaus.plexus:plexus-classworlds:jar:2.6.0:compile
[INFO]    |  |  \- org.apache.xbean:xbean-reflect:jar:3.7:compile
[INFO]    |  +- org.codehaus.plexus:plexus-component-annotations:jar:2.1.0:compile
[INFO]    |  +- org.apache.commons:commons-lang3:jar:3.8.1:compile
[INFO]    |  +- org.apache.commons:commons-text:jar:1.3:compile
[INFO]    |  +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[INFO]    |  \- org.apache.httpcomponents:httpcore:jar:4.4.14:compile
[INFO]    \- org.apache.maven.doxia:doxia-module-xdoc:jar:1.12.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.817 s
[INFO] Finished at: 2024-01-06T10:38:15+01:00
[INFO] ------------------------------------------------------------------------

as we can see, it keeps downloading google collections, even though dependency:tree
does not show it.

i am guessing that the dependency plugin applies an exclusion to all transitive dependencies but the normal maven dependency mechanism (and gradle) does not?
i would guess it's a bug in the dependency plugin then...

so we could try to make the exclusion explicitly below plexus-container-default instead?

or we could go back to what I had proposed originally and upgrade plexus-container-default to 2.1.1 instead (see diff of the versions)

or we could investigate why checkstyle depends on doxia with compile scope at all
(or upgrade doxia to a non-ancient version but according to previous comments this seems more involved?)

[XN137]

actually seems like maven doesnt really download the jar, it just retrieves the pom (even though it will not put the jar on the classpath of the module later on):

➜  ls ~/.m2/repository/com/google/collections/google-collections/1.0 
google-collections-1.0.pom  google-collections-1.0.pom.sha1  _remote.repositories

➜  ls ~/.m2/repository/com/google/guava/guava/33.0.0-jre             
guava-33.0.0-jre.jar  guava-33.0.0-jre.jar.sha1  guava-33.0.0-jre.pom  guava-33.0.0-jre.pom.sha1  _remote.repositories

so maybe gradle also does this and runs its conflict check before applying exclusions?
or does not respect excludes transitively as maven does?!

[hazendaz]

Seems spring is aware of issue with gradle here spring-gradle-plugins/dependency-management-plugin#211 and has not deprecated their plugin due to the issue. So it seems that is at least a possible work around to the issue but haven't looked into it.

In other words, this is a gradle bug which is why we are not seeing the problem with maven.

[hazendaz]

PR above from spotbugs should be enough to help others. It works locally for me on windows, just waiting for GHA to confirm for linux, mac, windows.

[hazendaz]

PR above from spotbugs should be enough to help others. It works locally for me on windows, just waiting for GHA to confirm for linux, mac, windows.

ok spring work around fixes the issue.

[hazendaz]

Closing issue as not checkstyles problem, its a gradle issue. Work around can be seen here spotbugs/spotbugs#2798

[hazendaz]

Thanks everyone for looking into this.

[JoseLion]

This is not a bug with Gradle. As mentioned in gradle/gradle#27035 (comment), the difference with Maven is that:

Gradle is now able to detect that Guava and Google Collections are in conflict and should not be used together

Checkstyle has Guava as a direct dependency, but it also has Google Collections as a transitive dependency through org.apache.maven.doxia:doxia-core:1.12.0. I think the real problem is that Checkstyle ships with Doxia on its classpath when it's only used for the generation of website content (as mentioned in #14140 (comment)). Maybe this is a good moment to pick up the work of removing Doxia as a dependency 🙂

Adding the Spring Dependency Management Plugin as a workaround sounds like an overkill, especially if you have nothing related to Spring in the project. The real workaround is to tell Gradle to select Guava classes to solve the conflict, as described in Guava's v32.1.0 release notes. I.e., adding the code below to your build.gradle file:

configurations.checkstyle {
  resolutionStrategy.capabilitiesResolution.withCapability("com.google.collections:google-collections") {
    select("com.google.guava:guava:0")
  }
}

[hazendaz]

Thank you. I will try this and we do have spring but I still stand by that gradle has a bug. Checkstyle excluded that transient. The link you showed was before the transient was excluded. Gradle should respect that just like maven does. It does not so it's a bug as far as I'm concerned. A plugin should not cause this issue. Maven does not do that and dependency tree shows that. Spring notes this issue well in gradle resolutions for years with exclusions. Not saying I liked our fix as i suspect it would cause other unwanted side effects but gradle has issues here when it fails to respect pom exclusions. That is regardless of what a tool uses or not. Anyway your suggestion looks better in that it will remove my suspicion that dependency management will cause more issues for our user base and thus if it works I have less to subsequently confirm. I'll respond back with results of this likely later tonight. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Jose Luis Leon ***@***.***> Sent: Tuesday, January 9, 2024 10:22:51 PM To: checkstyle/checkstyle ***@***.***> Cc: Jeremy Landis ***@***.***>; State change ***@***.***> Subject: Re: [checkstyle/checkstyle] google collections issue still present in 10.12.7 (Issue #14211) This is not a bug with Gradle. As mentioned in gradle/gradle#27035 (comment)<gradle/gradle#27035 (comment)>, the difference with Maven is that: Gradle is now able to detect that Guava and Google Collections are in conflict and should not be used together Checkstyle has Guava as a direct dependency, but it also has Google Collections as a transient dependency through org.apache.maven.doxia:doxia-core:1.12.0. I think the real problem is that Checkstyle ships with Doxia on its classpath when it's only used for the generation of website content (as mentioned in #14140 (comment)<#14140 (comment)>). Maybe this is a good moment to pick up the work of removing Doxia as a dependency 🙂 Adding the Spring Dependency Management Plugin as a workaround sounds like an overkill to me, especially if you have nothing related to Spring in the project. The real workaround is to tell Gradle to select Guava classes to solve the conflict, as described in Guava's v32.1.0 release notes<https://github.com/google/guava/releases/tag/v32.1.0>. I.e., adding the code below to your build.gradle file: configurations.checkstyle { resolutionStrategy.capabilitiesResolution.withCapability("com.google.collections:google-collections") { select("com.google.guava:guava:0") } } — Reply to this email directly, view it on GitHub<#14211 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAHODIZJB7NVJTUMP2A7AKDYNYCQXAVCNFSM6AAAAABBIOEIG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBUGEZDSOJUHA>. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[matthewlowry]

The work-around suggested by @JoseLion works for me. Thanks!

Hadn't seen this when upgrading my other projects because they are all Spring Boot projects so the Spring magicians were taking care of this for me in those projects 😃.

[hazendaz]

@JoseLion Worked for me too, moving forwards with the patch. Thank you! I trust this one better to not mess up our users :)

[romani]

Workarounds:
https://github.com/spotbugs/spotbugs/pull/2798/files#diff-ce2f19941bd52f3190df72640978dc7baa8040ff85bb73e3a409285ba2c01dc8R4

https://github.com/JabRef/jabref/pull/10812/files#diff-49a96e7eea8a94af862798a45174e6ac43eb4f8b4bd40759b5da63ba31ec3ef7R267

But we definitely come back to dependency issue to upgrade without hacks in gradle config.

[dtrunk90]

I agree, it actually seems to be a gradle issue not excluding the transient dependency. But still, a dependency (doxia) which is only required to generate website content shouldn't be shipped at all. I don't want to put a hack inside my gradle build script. So, I'll stick with the old version until this gets fixed. Is there any follow-up issue to get doxia unshipped? Isn't there something similar to compileOnly in maven?

[sdavids]

@romani

But we definitely come back to dependency issue to upgrade without hacks in gradle config.

Which open issue should we subscribe to?

#14123 and this one are closed ...

[romani]

@hazendaz , do you still have this problem ?
I hope to reuse your project spotbugs in CI build to never have this problems in future. Here is proposal to fix issue #14669

[dtrunk90]

I can also check tomorrow (UTC+1) if it solves the issue with gradle.

[romani]

released https://github.com/checkstyle/checkstyle/releases/tag/checkstyle-10.14.2

[romani]

reproduced: and 10.14.2 fixing a problem.

✔ ~/java/github/spotbugs/spotbugs [master|✚ 1] 
17:17 $ git diff
diff --git a/gradle/checkstyle.gradle b/gradle/checkstyle.gradle
index cd1a99d..b92ee41 100644
--- a/gradle/checkstyle.gradle
+++ b/gradle/checkstyle.gradle
@@ -1,12 +1,6 @@
 // Setup checkstyle
 apply plugin: 'checkstyle'
 
-configurations.checkstyle {
-  resolutionStrategy.capabilitiesResolution.withCapability("com.google.collections:google-collections") {
-    select("com.google.guava:guava:33.1.0-jre")
-  }
-}
-
 checkstyle {
   toolVersion '10.14.1'
   ignoreFailures false


✔ ~/java/github/spotbugs/spotbugs [master|✚ 1] 
17:15 $ ./gradlew :eclipsePlugin-junit:checkstyleTest

> Configure project :eclipsePlugin
.....
> Configure project :spotbugs
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :spotbugs-annotations
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :spotbugs-ant
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness-core
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness-jupiter
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :
The 'sonarqube' task depends on compile tasks. This behavior is now deprecated and will be removed in version 5.x. To avoid implicit compilation, set property 'sonar.gradle.skipCompile' to 'true' and make sure your project is compiled, before analysis has started.
The 'sonar' task depends on compile tasks. This behavior is now deprecated and will be removed in version 5.x. To avoid implicit compilation, set property 'sonar.gradle.skipCompile' to 'true' and make sure your project is compiled, before analysis has started.

> Task :eclipsePlugin-junit:checkstyleTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':eclipsePlugin-junit:checkstyleTest'.
> Could not resolve all files for configuration ':eclipsePlugin-junit:checkstyle'.
   > Could not resolve com.google.guava:guava:33.0.0-jre.
     Required by:
         project :eclipsePlugin-junit > com.puppycrawl.tools:checkstyle:10.14.1
      > Module 'com.google.guava:guava' has been rejected:
           Cannot select module with conflict on capability 'com.google.collections:google-collections:33.0.0-jre' also provided by [com.google.collections:google-collections:1.0(runtime)]
   > Could not resolve com.google.collections:google-collections:1.0.
     Required by:
         project :eclipsePlugin-junit > com.puppycrawl.tools:checkstyle:10.14.1 > org.apache.maven.doxia:doxia-core:1.12.0 > org.codehaus.plexus:plexus-container-default:2.1.0
      > Module 'com.google.collections:google-collections' has been rejected:
           Cannot select module with conflict on capability 'com.google.collections:google-collections:1.0' also provided by [com.google.guava:guava:33.0.0-jre(jreRuntimeElements)]

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.6/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 2s
24 actionable tasks: 2 executed, 22 up-to-date

17:10 $ git diff
diff --git a/gradle/checkstyle.gradle b/gradle/checkstyle.gradle
index cd1a99d..dda04a3 100644
--- a/gradle/checkstyle.gradle
+++ b/gradle/checkstyle.gradle
@@ -1,14 +1,8 @@
 // Setup checkstyle
 apply plugin: 'checkstyle'
 
-configurations.checkstyle {
-  resolutionStrategy.capabilitiesResolution.withCapability("com.google.collections:google-collections") {
-    select("com.google.guava:guava:33.1.0-jre")
-  }
-}
-
 checkstyle {
-  toolVersion '10.14.1'
+  toolVersion '10.14.2'
   ignoreFailures false
   configFile file("$rootDir/spotbugs/etc/checkstyle.xml") // TODO : This config file is lame and should be moved out...
 }


✘-1 ~/java/github/spotbugs/spotbugs [master|✚ 1] 
17:11 $ ./gradlew :eclipsePlugin-junit:checkstyleTest

......
> Configure project :spotbugs
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :spotbugs-annotations
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :spotbugs-ant
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness-core
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :test-harness-jupiter
The signing key and password are null. This can be ignored if this is a pull request.

> Configure project :
The 'sonarqube' task depends on compile tasks. This behavior is now deprecated and will be removed in version 5.x. To avoid implicit compilation, set property 'sonar.gradle.skipCompile' to 'true' and make sure your project is compiled, before analysis has started.
The 'sonar' task depends on compile tasks. This behavior is now deprecated and will be removed in version 5.x. To avoid implicit compilation, set property 'sonar.gradle.skipCompile' to 'true' and make sure your project is compiled, before analysis has started.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.6/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 5s
24 actionable tasks: 2 executed, 22 up-to-date

[hazendaz]

Thanks @romani, outside of gradle usage never had the issue. Glad you were able to pull off our patch and end up fixing within checkstyle.

If I recall I don't think this affected our latest spotbugs release from back in December. However, if anyone updated checkstyle with gradle for other usages it would have surfaced the same.