Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NullPointerException in FieldItemSummary #2759

Closed
romani opened this issue Dec 9, 2023 · 13 comments
Closed

NullPointerException in FieldItemSummary #2759

romani opened this issue Dec 9, 2023 · 13 comments

Comments

@romani
Copy link
Contributor

romani commented Dec 9, 2023

detected during version bump spotbugs-maven-plugin from 4.7.3.5 to 4.8.2.0
checkstyle/checkstyle#14117

https://app.circleci.com/pipelines/github/checkstyle/checkstyle/22221/workflows/e397e60b-04ca-4618-b94e-5ca0be2a20d8/jobs/446175

OS:

 Kernel Version: 5.15.0-1039-aws
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64

Special docker container:

Starting container checkstyle/jdk-11-groovy-git-mvn:11.0.20.1__2.4.21__2.42.0__3.9.5
this means
jdk11.0.20.1
mvn 3.9.5

Execution log:

.....
[�[1;34mINFO�[m] �[1m--- �[0;32mspotbugs:4.8.2.0:spotbugs�[m �[1m(spotbugs)�[m @ �[36mcheckstyle�[0;1m ---�[m
[�[1;34mINFO�[m] Fork Value is true
     [java] The following errors occurred during analysis:
     [java]   Exception analyzing com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageParser$ClassBlockContext using detector edu.umd.cs.findbugs.detect.FieldItemSummary
     [java]     java.lang.NullPointerException
     [java]       At edu.umd.cs.findbugs.visitclass.DismantleBytecode.visit(DismantleBytecode.java:719)
     [java]       At edu.umd.cs.findbugs.detect.FieldItemSummary.visit(FieldItemSummary.java:131)
     [java]       At edu.umd.cs.findbugs.visitclass.BetterVisitor.visitCode(BetterVisitor.java:218)
     [java]       At edu.umd.cs.findbugs.visitclass.PreorderVisitor.visitCode(PreorderVisitor.java:250)
     [java]       At edu.umd.cs.findbugs.bcel.OpcodeStackDetector.visitCode(OpcodeStackDetector.java:65)
     [java]       At org.apache.bcel.classfile.Code.accept(Code.java:121)
     [java]       At edu.umd.cs.findbugs.visitclass.PreorderVisitor.doVisitMethod(PreorderVisitor.java:322)
     [java]       At edu.umd.cs.findbugs.visitclass.PreorderVisitor.visitJavaClass(PreorderVisitor.java:404)
     [java]       At org.apache.bcel.classfile.JavaClass.accept(JavaClass.java:244)
     [java]       At edu.umd.cs.findbugs.BytecodeScanningDetector.visitClassContext(BytecodeScanningDetector.java:38)
     [java]       At edu.umd.cs.findbugs.DetectorToDetector2Adapter.visitClass(DetectorToDetector2Adapter.java:76)
     [java]       At edu.umd.cs.findbugs.FindBugs2.lambda$analyzeApplication$1(FindBugs2.java:1108)
     [java]       At java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
     [java]       At edu.umd.cs.findbugs.CurrentThreadExecutorService.execute(CurrentThreadExecutorService.java:86)
     [java]       At java.base/java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:242)
     [java]       At edu.umd.cs.findbugs.FindBugs2.analyzeApplication(FindBugs2.java:1118)
     [java]       At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:309)
     [java]       At edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:395)
     [java]       At edu.umd.cs.findbugs.FindBugs2.main(FindBugs2.java:1231)

Note:
on versions below it works wihtou exception (my local):

$ mvn --version
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Maven home: /opt/maven/apache-maven-3.8.4
Java version: 11.0.14.1, vendor: Amazon.com Inc., runtime: /opt/jvm/amazon-corretto-11.0.14.10.1-linux-x64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-210-generic", arch: "amd64", family: "unix"
@romani
Copy link
Contributor Author

romani commented Dec 9, 2023

the same NPE even on old container that we used for several years "checkstyle/jdk-11-groovy-git-mvn:11.0.13__3.0.9__2.25.1__3.6.3"
so regression is in FieldItemSummary
This is very strange we have execution of spotbugs from in many other CIs (different OS, and no problems on such executions)

Command that we use in CI for our repo:
mvn -e --no-transfer-progress clean test-compile pmd:check spotbugs:check

@gtoison
Copy link
Contributor

gtoison commented Dec 14, 2023

As far as I can see the code (edu.umd.cs.findbugs.visitclass.DismantleBytecode or edu.umd.cs.findbugs.detect.FieldItemSummary) hasn't changed much recently so this is very strange indeed.
Is there a way to reproduce the problem without docker (simply with maven?), that would facilitate the troubleshooting

@romani
Copy link
Contributor Author

romani commented Dec 14, 2023

As I stated above, we don't know a way to reproduce. Might be very special nuance of jdk version jdk11.0.20.1.

@gtoison
Copy link
Contributor

gtoison commented Dec 14, 2023

Would it be possible to share a zip with the compiled .class files of the project?

@romani
Copy link
Contributor Author

romani commented Dec 31, 2023

Here are all binaries https://app.circleci.com/pipelines/github/checkstyle/checkstyle/22599/workflows/002e52f5-5903-49da-be98-1100895997dc/jobs/462597/artifacts, zip is a bit more complicated to make, let's try this way first.

Binaries are generated by pull request checkstyle/checkstyle#14205

In case such PR be closed, we can always reopen it to rerun CI.

@gtoison
Copy link
Contributor

gtoison commented Jan 2, 2024

Thank you for the samples! I tried to reproduce the problem but strangely got an other error which ultimately boils down to BCEL-370 so I submitted an fix attempt here: apache/commons-bcel#254

Unless I'm mistaken your build runs SpotBugs after Jacoco has modified the class files. The updated bytecode contains instructions that are valid but not normally emitted by compilers. This happens on JDK 11 because the Constant_Dynamic instruction was added there, so Jacoco only uses it on >= 11.

So until the issue is fixed in BCEL (and maybe SpotBugs) I think that you should try a find a way to run SpotBugs before Jacoco

I'm still looking into the error you got, but here's the one I see:

The following errors occurred during analysis:
  Exception analyzing com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageParser$ClassBlockContext using detector edu.umd.cs.findbugs.detect.NoteNonnullReturnValues
    java.lang.IllegalArgumentException: Unknown or invalid constant type at 72
      At org.apache.bcel.generic.LDC.getValue(LDC.java:113)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberFrameModelingVisitor.visitLDC(ValueNumberFrameModelingVisitor.java:529)
      At org.apache.bcel.generic.LDC.accept(LDC.java:58)
      At edu.umd.cs.findbugs.ba.AbstractFrameModelingVisitor.analyzeInstruction(AbstractFrameModelingVisitor.java:84)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transferInstruction(ValueNumberAnalysis.java:228)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transferInstruction(ValueNumberAnalysis.java:57)
      At edu.umd.cs.findbugs.ba.AbstractDataflowAnalysis.transfer(AbstractDataflowAnalysis.java:136)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transfer(ValueNumberAnalysis.java:214)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transfer(ValueNumberAnalysis.java:57)
      At edu.umd.cs.findbugs.ba.Dataflow.execute(Dataflow.java:378)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:76)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:46)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.AnalysisFactory.getValueNumberDataflow(AnalysisFactory.java:118)
      At edu.umd.cs.findbugs.classfile.engine.bcel.TypeDataflowFactory.analyze(TypeDataflowFactory.java:73)
      At edu.umd.cs.findbugs.classfile.engine.bcel.TypeDataflowFactory.analyze(TypeDataflowFactory.java:43)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.CFGFactory.analyze(CFGFactory.java:160)
      At edu.umd.cs.findbugs.classfile.engine.bcel.CFGFactory.analyze(CFGFactory.java:65)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.ba.ClassContext.getMethodAnalysis(ClassContext.java:1010)
      At edu.umd.cs.findbugs.ba.ClassContext.getMethodAnalysisNoDataflowAnalysisException(ClassContext.java:995)
      At edu.umd.cs.findbugs.ba.ClassContext.getCFG(ClassContext.java:301)
      At edu.umd.cs.findbugs.detect.BuildNonnullReturnDatabase.analyzeMethod(BuildNonnullReturnDatabase.java:87)
      At edu.umd.cs.findbugs.detect.BuildNonnullReturnDatabase.considerMethod(BuildNonnullReturnDatabase.java:76)
      At edu.umd.cs.findbugs.detect.BuildNonnullReturnDatabase.visitClassContext(BuildNonnullReturnDatabase.java:67)
      At edu.umd.cs.findbugs.DetectorToDetector2Adapter.visitClass(DetectorToDetector2Adapter.java:76)
      At edu.umd.cs.findbugs.FindBugs2.lambda$analyzeApplication$1(FindBugs2.java:1108)
      At java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      At edu.umd.cs.findbugs.CurrentThreadExecutorService.execute(CurrentThreadExecutorService.java:86)
      At java.base/java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:247)
      At edu.umd.cs.findbugs.FindBugs2.analyzeApplication(FindBugs2.java:1118)
      At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:309)
      At edu.umd.cs.findbugs.test.AnalysisRunner.run(AnalysisRunner.java:115)
      At edu.umd.cs.findbugs.test.SpotBugsRunner.performAnalysis(SpotBugsRunner.java:61)
      At edu.umd.cs.findbugs.test.SpotBugsRunner.performAnalysis(SpotBugsRunner.java:75)
      At edu.umd.cs.findbugs.visitclass.DismantleBytecodeTest.testIssue(DismantleBytecodeTest.java:67)
      At java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      At java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
      At java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      At java.base/java.lang.reflect.Method.invoke(Method.java:568)
      At org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
      At org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)

gtoison added a commit to gtoison/spotbugs that referenced this issue Jan 8, 2024
Class files modified by jacoco cause the spotbugs analysis to crash due
to spotbugs#2759
hazendaz pushed a commit that referenced this issue Jan 13, 2024
* test: added .class samples modified by jacoco

Class files modified by jacoco cause the spotbugs analysis to crash due
to #2759

* fix: upgrade BCEL to 6.8.1RC1

* fix: added partial support for CONSTANT_Dynamic

Class files using condy (Dynamic class-file constant), like JaCoCo >=
0.8.4
creates them (see [1] and [2] for more details), causes errors we using
Spotbugs to parse these class files.

The `FooWithMember.clazz` embedded in this commit has been generated
from this class source file:

```java
package com.foo;

class Foo {
    int a;

    public Foo(int a) {
        this.a = a;
    }

    public int getA() {
        return this.a;
    }
}
```

These modifications don't completely fix the support of
`CONSTANT_Dynamic`.
We continue to have error from BCEL (maybe [3] is insufficient):

```
  Exception analyzing com.foo.Foo using detector
edu.umd.cs.findbugs.detect.FindNullDeref
    java.lang.IllegalArgumentException: Unknown or invalid constant type
at 32
      At org.apache.bcel.generic.LDC.getValue(LDC.java:113)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberFrameModelingVisitor.visitLDC(ValueNumberFrameModelingVisitor.java:529)
      At org.apache.bcel.generic.LDC.accept(LDC.java:58)
      At edu.umd.cs.findbugs.ba.AbstractFrameModelingVisitor.analyzeInstruction(AbstractFrameModelingVisitor.java:84)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transferInstruction(ValueNumberAnalysis.java:228)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transferInstruction(ValueNumberAnalysis.java:57)
      At edu.umd.cs.findbugs.ba.AbstractDataflowAnalysis.transfer(AbstractDataflowAnalysis.java:136)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transfer(ValueNumberAnalysis.java:214)
      At edu.umd.cs.findbugs.ba.vna.ValueNumberAnalysis.transfer(ValueNumberAnalysis.java:57)
      At edu.umd.cs.findbugs.ba.Dataflow.execute(Dataflow.java:378)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:76)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:46)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.AnalysisFactory.getValueNumberDataflow(AnalysisFactory.java:118)
      At edu.umd.cs.findbugs.classfile.engine.bcel.TypeDataflowFactory.analyze(TypeDataflowFactory.java:73)
      At edu.umd.cs.findbugs.classfile.engine.bcel.TypeDataflowFactory.analyze(TypeDataflowFactory.java:43)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.CFGFactory.analyze(CFGFactory.java:160)
      At edu.umd.cs.findbugs.classfile.engine.bcel.CFGFactory.analyze(CFGFactory.java:65)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.AnalysisFactory.getCFG(AnalysisFactory.java:93)
      At edu.umd.cs.findbugs.classfile.engine.bcel.DepthFirstSearchFactory.analyze(DepthFirstSearchFactory.java:49)
      At edu.umd.cs.findbugs.classfile.engine.bcel.DepthFirstSearchFactory.analyze(DepthFirstSearchFactory.java:32)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.classfile.engine.bcel.AnalysisFactory.getDepthFirstSearch(AnalysisFactory.java:98)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:68)
      At edu.umd.cs.findbugs.classfile.engine.bcel.ValueNumberDataflowFactory.analyze(ValueNumberDataflowFactory.java:46)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.analyzeMethod(AnalysisCache.java:368)
      At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getMethodAnalysis(AnalysisCache.java:321)
      At edu.umd.cs.findbugs.ba.ClassContext.getMethodAnalysis(ClassContext.java:1010)
      At edu.umd.cs.findbugs.ba.ClassContext.getValueNumberDataflow(ClassContext.java:333)
      At edu.umd.cs.findbugs.detect.FindNullDeref.findPreviouslyDeadBlocks(FindNullDeref.java:301)
      At edu.umd.cs.findbugs.detect.FindNullDeref.analyzeMethod(FindNullDeref.java:271)
      At edu.umd.cs.findbugs.detect.FindNullDeref.visitClassContext(FindNullDeref.java:216)
      At edu.umd.cs.findbugs.DetectorToDetector2Adapter.visitClass(DetectorToDetector2Adapter.java:76)
      At edu.umd.cs.findbugs.FindBugs2.lambda$analyzeApplication$1(FindBugs2.java:1108)
      At java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      At edu.umd.cs.findbugs.CurrentThreadExecutorService.execute(CurrentThreadExecutorService.java:86)
      At java.base/java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:247)
      At edu.umd.cs.findbugs.FindBugs2.analyzeApplication(FindBugs2.java:1118)
      At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:309)
      At edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:395)
```

[1]: jacoco/jacoco@36b4e9c#diff-1d450b7247672d9362bac0ad7dfd7bd810e4293ab57f0100821a70a5e64e6911R56-R57
[2]: jacoco/jacoco#845
[3]: https://issues.apache.org/jira/browse/BCEL-362

* fix: Ignore ConstantDynamic in ResolveAllReferences

* deps: reverted to using the default repos now that BCEL 6.8.1 is out

* doc: added a changelog entry

* test: converted acceptConstantDynamic to a parameterized test

---------

Co-authored-by: Jean-Baptiste Bronisz <jean-baptiste.bronisz@adevinta.com>
@gtoison
Copy link
Contributor

gtoison commented Jan 13, 2024

So this should be fixed in the upcoming release now that the problem in BCEL is solved and #2807 is merged

@romani
Copy link
Contributor Author

romani commented Jan 16, 2024

@gtoison, kind of new problem after update to 4.8.3.0
#2819

@romani
Copy link
Contributor Author

romani commented Feb 13, 2024

@gtoison, is this issue fixed in 4.8.3.1 ?

@hazendaz
Copy link
Member

hazendaz commented Feb 13, 2024 via email

@hazendaz
Copy link
Member

hazendaz commented Feb 13, 2024 via email

@gtoison
Copy link
Contributor

gtoison commented May 6, 2024

Hello @romani, this was fixed by #2807 (as well as #2819), right?

@romani
Copy link
Contributor Author

romani commented May 7, 2024

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants