Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deserialization of a nested inherited type with an unknown json object fails with a NullPointerException #589

Closed
DXTR66 opened this issue Feb 10, 2023 · 8 comments
Closed

Deserialization of a nested inherited type with an unknown json object fails with a NullPointerException #589

DXTR66 opened this issue Feb 10, 2023 · 8 comments
Labels
bug Something isn't working right

Comments

@DXTR66
Copy link

DXTR66 commented Feb 10, 2023 •
edited

Describe the bug
Deserialization of an inherited type structure fails with a NullPointerException if the structure itself is nested and the structure contains an unknown object which should be skipped.

To Reproduce
Very basic model:

// base class
@JsonbTypeInfo(key = "_type", value = @JsonbSubtype(type = Derivation.class, alias = "derivationA"))
public class Base {
    private Long id;
}

// derivation of the base class
public class Derivation extends Base {}

// some arbitrary 'outer' root element
public class Outer {
    private Base inner;
}

JSON representation containing an unknown object in the nested object.

{
  "inner" : {
      "id" : 123,
      "_type" : "derivationA",
      "unmapped" : {}
    }
}

If you try to deserialize the given JSON in the given model with JsonbBuilder.create().fromJson(json, Outer.class); this exception is thrown

jakarta.json.bind.JsonbException: Internal error: null
	at org.eclipse.yasson.internal.DeserializationContextImpl.deserializeItem(DeserializationContextImpl.java:142)
	at org.eclipse.yasson.internal.DeserializationContextImpl.deserialize(DeserializationContextImpl.java:127)
	at org.eclipse.yasson.internal.JsonBinding.deserialize(JsonBinding.java:55)
	at org.eclipse.yasson.internal.JsonBinding.fromJson(JsonBinding.java:62)
	at x.a.b.bla.Bla.bla(Bla.java:52)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:95)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:91)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:60)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: java.lang.UnsupportedOperationException
	at jakarta.json.stream.JsonParser.skipObject(JsonParser.java:502)
	at org.eclipse.yasson.internal.deserializer.ObjectDeserializer.deserialize(ObjectDeserializer.java:86)
	at org.eclipse.yasson.internal.deserializer.ObjectDeserializer.deserialize(ObjectDeserializer.java:31)
	at org.eclipse.yasson.internal.deserializer.DefaultObjectInstanceCreator.deserialize(DefaultObjectInstanceCreator.java:57)
	at org.eclipse.yasson.internal.deserializer.DefaultObjectInstanceCreator.deserialize(DefaultObjectInstanceCreator.java:29)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:85)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:34)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:46)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:26)
	at org.eclipse.yasson.internal.deserializer.InheritanceInstanceCreator.deserialize(InheritanceInstanceCreator.java:72)
	at org.eclipse.yasson.internal.deserializer.InheritanceInstanceCreator.deserialize(InheritanceInstanceCreator.java:31)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:85)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:34)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:46)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:26)
	at org.eclipse.yasson.internal.deserializer.ContextSwitcher.deserialize(ContextSwitcher.java:36)
	at org.eclipse.yasson.internal.deserializer.ContextSwitcher.deserialize(ContextSwitcher.java:22)
	at org.eclipse.yasson.internal.deserializer.ObjectDeserializer.deserialize(ObjectDeserializer.java:78)
	at org.eclipse.yasson.internal.deserializer.ObjectDeserializer.deserialize(ObjectDeserializer.java:31)
	at org.eclipse.yasson.internal.deserializer.DefaultObjectInstanceCreator.deserialize(DefaultObjectInstanceCreator.java:57)
	at org.eclipse.yasson.internal.deserializer.DefaultObjectInstanceCreator.deserialize(DefaultObjectInstanceCreator.java:29)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:85)
	at org.eclipse.yasson.internal.deserializer.PositionChecker.deserialize(PositionChecker.java:34)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:46)
	at org.eclipse.yasson.internal.deserializer.NullCheckDeserializer.deserialize(NullCheckDeserializer.java:26)
	at org.eclipse.yasson.internal.DeserializationContextImpl.deserializeItem(DeserializationContextImpl.java:138)
	... 74 more

Expected behavior
The unmapped object should be skipped, no error should occur.

System information:

  • OS: Windows
  • Java Version: 17
  • Yasson Version: 3.0.1
@DXTR66 DXTR66 added the bug Something isn't working right label Feb 10, 2023
@greek1979
Copy link
Contributor

Let me take a look...

@greek1979
Copy link
Contributor

The first observation that struck me is that Yasson version 1.1.1 is nowhere to be found in public Maven repositories... If we look at https://mvnrepository.com/artifact/org.eclipse/yasson , we can versions 1.0.1 (Nov 2017), 1.0.10 (Nov 2021) and 1.0.11 (Jan 2022) there. Perhaps, you are using one of those?

@DXTR66
Copy link
Author

DXTR66 commented Feb 17, 2023

Big sry, my bad - it is of course version 3.0.1 (which is part of Wildfly 27). I updated my opening post.

@greek1979
Copy link
Contributor

Oh, ok, I see. After some experiments, I managed to reproduce the error on 3.x code branch. It seems as the root cause of this is use of annotations (to specify the exact mapping of JSON-to-Java objects); the use of annotations cause Yasson to rely on the JsonStructureToParserAdapter internal JSON parser that simply do not have the skipObject() (skip this nested object and advance to the next property of the parent / enclosing object) and skipArray() (the same, but for arrays) methods implemented! As simple as that. Why it does not implement them, I am not exactly sure. Probably an omission.

Let me try to implement them to see whether this will resolve the issue. If it does, then obviously you would have to wait for the next Yasson version - or even wait until Wildfly developers or maintainers choose to adopt it. OR grab it manually and add to Wildfly server libraries. The only workaround would be to set the failOnUnknownProperty property to true i.e. causing JSON parser to trip and abort on an unmapped property such as one in your example. But if you do not have full control over the inbound JSON contents, then this may not be a good idea....

greek1979 added a commit to greek1979/yasson that referenced this issue Feb 17, 2023
@greek1979
Fix issue eclipse-ee4j#589 (implement skipObject() and skipArray() me…
ec00199 
…thods)
@greek1979 greek1979 mentioned this issue Feb 17, 2023
Merged
@redmitry
Copy link

redmitry commented Feb 17, 2023 •
edited

Pretty sure this is the same issue.

The problem is that JsonStructureToParserAdapter peek object without skipping to the next property.
Could you confirm that it is fixed with the patch?

Thank you,

D.

yasson-test-589.zip

P.S. the version is 3.0.2

@greek1979
Copy link
Contributor

@redmitry , will look into this on Monday. Seems similar indeed (I have noticed that JsonStructureToParserAdapter's next() logic is a bit shaky...)

@greek1979
Copy link
Contributor

Well, after studying and debuggin the Yasson code over the weekend, I must say the last issue is not an easy one to resolve... Apparently, JsonStructureToParserAdapter works well with non-annotated (non-polymorphic) POJOs, and is able to deserialize input JSON messages / strings without issues, relying on the concept of nested object iterators - for each inner object (property) to map and deserialize, an iterator is added to the context chain. However, then type customisations (annotations) join the game, handling each subtype annotation creates its own, independent "deserialization context", with its own set of iterators processing key by key, one at a time. This works fairly well, except that "parent" context and parent iterators know nothing about this! So once we get back to the parent POJO - the "Pets" in @redmitry test case - parent and nested deserialization contexts (and iterators) are out of sync and things get broken.

(thinking about a workaround for this that would not break other things - few things already tried did break other tests...)

@greek1979
Copy link
Contributor

@redmitry , thank you for bringing up a rather peculiar deserialization issue, and putting together a very useful test case! That issue is resolved now by amending the original code patch - it turns out, the root cause of the ORIGINAL bug described here was two-fold, and now both places are hopefully patched, and JSON deserialization is no longer "leaky".

greek1979 added a commit to greek1979/yasson that referenced this issue Feb 20, 2023
@greek1979
Avoid duplicate deserialization of nested polymorphic objects (eclips…
bbcf8f2 
…e-ee4j#589)
greek1979 added a commit to greek1979/yasson that referenced this issue Feb 24, 2023
@greek1979
Fix issue eclipse-ee4j#589 (implement skipObject() and skipArray() me…
a4d7749 
…thods)
greek1979 added a commit to greek1979/yasson that referenced this issue Feb 24, 2023
@greek1979
Avoid duplicate deserialization of nested polymorphic objects (eclips…
f44a65b 
…e-ee4j#589)
greek1979 added a commit to greek1979/yasson that referenced this issue Feb 24, 2023
@greek1979
Copyright fixes (eclipse-ee4j#589)
3c78ff7
greek1979 added a commit to greek1979/yasson that referenced this issue Feb 25, 2023
@greek1979
Fixed checkstyle ImportOrder error (eclipse-ee4j#589)
52c034c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working right
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DXTR66 @greek1979 @redmitry