Verify AOT testing support for all tests in the `spring-test` module #29122

sbrannen · 2022-09-09T15:37:18Z

Overview

Now that we have AOT testing support in place, we should introduce a way to run end-to-end tests for all Spring TestContext Framework integration tests in the spring-test module.

This will help us to discover issues with our ongoing AOT efforts.

Related Issues

The text was updated successfully, but these errors were encountered:

This commit introduces endToEndTestsForEntireSpringTestModule() in AotIntegrationTests to allow us to periodically check on our AOT support. Status quo: - several test classes cannot be processed for AOT due to exceptions thrown during processing - some generated classes fail to compile - some tests fail See spring-projectsgh-29122

See gh-29122

…ngTestModule() Current results for the spring-test module: Test run finished after 6785 ms [ 403 containers found ] [ 6 containers skipped ] [ 397 containers started ] [ 0 containers aborted ] [ 381 containers successful ] [ 16 containers failed ] [ 757 tests found ] [ 41 tests skipped ] [ 703 tests started ] [ 9 tests aborted ] [ 599 tests successful ] [ 95 tests failed ] Failing Test Classes: org.springframework.test.context.testng.transaction.ejb.RollbackForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.RollbackForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.env.ExplicitPropertiesFileTestPropertySourceTests$ClasspathTests$PlaceholderAndClasspathPrefixTests org.springframework.test.context.env.ExplicitPropertiesFileTestPropertySourceTests$ClasspathTests$PlaceholderTests org.springframework.test.context.env.ExplicitPropertiesFileTestPropertySourceTests$FileSystemTests$CustomPlaceholderTests org.springframework.test.context.env.ExplicitPropertiesFileTestPropertySourceTests$FileSystemTests$PlaceholdersFollowedByRelativePathsTests org.springframework.test.context.env.ExplicitPropertiesFileTestPropertySourceTests$FileSystemTests$UserDirAndCustomPlaceholdersTests org.springframework.test.web.servlet.samples.client.context.WebAppResourceTests org.springframework.test.web.servlet.samples.client.context.XmlConfigTests org.springframework.test.web.servlet.samples.context.WebAppResourceTests org.springframework.test.web.servlet.samples.context.XmlConfigTests org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests org.springframework.test.context.junit4.nested.NestedTestsWithSpringRulesTests$NestedTestCase org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests See gh-29122

This commit applies @⁠DisabledInAotMode to test classes in the spring-test module that will never be able to be processed for AOT optimizations. Test classes that fail for reasons that can potentially be addressed in a future version of the framework have not been annotated with @⁠DisabledInAotMode. See gh-29122

Prior to this commit, the NestedTestCase resulted in errors during our AOT end-to-end integration tests since it did not comply to our "*Tests" naming convention. See spring-projectsgh-29122

Prior to this commit, AotIntegrationTests only printed "failing test classes" for tests that failed at the class level. This commit updates the reporting logic to report test classes for failing test methods as well. Current results for the spring-test module: Test run finished after 6531 ms [ 403 containers found ] [ 27 containers skipped ] [ 376 containers started ] [ 0 containers aborted ] [ 366 containers successful ] [ 10 containers failed ] [ 757 tests found ] [ 61 tests skipped ] [ 689 tests started ] [ 9 tests aborted ] [ 585 tests successful ] [ 95 tests failed ] Failing Test Classes: org.springframework.test.context.configuration.interfaces.ContextHierarchyInterfaceTests org.springframework.test.context.configuration.interfaces.SqlConfigInterfaceTests org.springframework.test.context.expression.ExpressionUsageTests org.springframework.test.context.groovy.AbsolutePathGroovySpringContextTests org.springframework.test.context.groovy.GroovySpringContextTests org.springframework.test.context.groovy.RelativePathGroovySpringContextTests org.springframework.test.context.hierarchies.meta.MetaHierarchyLevelTwoTests org.springframework.test.context.hierarchies.standard.ClassHierarchyWithMergedConfigLevelOneTests org.springframework.test.context.hierarchies.standard.ClassHierarchyWithMergedConfigLevelTwoTests org.springframework.test.context.hierarchies.standard.ClassHierarchyWithOverriddenConfigLevelTwoTests org.springframework.test.context.hierarchies.standard.DirtiesContextWithContextHierarchyTests org.springframework.test.context.hierarchies.standard.SingleTestClassWithTwoLevelContextHierarchyAndMixedConfigTypesTests org.springframework.test.context.hierarchies.standard.SingleTestClassWithTwoLevelContextHierarchyTests org.springframework.test.context.hierarchies.standard.TestHierarchyLevelTwoWithBareContextConfigurationInSubclassTests org.springframework.test.context.hierarchies.standard.TestHierarchyLevelTwoWithBareContextConfigurationInSuperclassTests org.springframework.test.context.hierarchies.standard.TestHierarchyLevelTwoWithSingleLevelContextHierarchyAndMixedConfigTypesTests org.springframework.test.context.hierarchies.standard.TestHierarchyLevelTwoWithSingleLevelContextHierarchyTests org.springframework.test.context.hierarchies.web.ControllerIntegrationTests org.springframework.test.context.hierarchies.web.DispatcherWacRootWacEarTests org.springframework.test.context.hierarchies.web.RootWacEarTests org.springframework.test.context.jdbc.CustomScriptSyntaxSqlScriptsTests org.springframework.test.context.jdbc.GlobalCustomScriptSyntaxSqlScriptsTests org.springframework.test.context.jdbc.InferredDataSourceTransactionalSqlScriptsTests org.springframework.test.context.jdbc.InfrastructureProxyTransactionalSqlScriptsTests org.springframework.test.context.jdbc.MultipleDataSourcesAndTransactionManagersSqlScriptsTests org.springframework.test.context.jdbc.MultipleDataSourcesAndTransactionManagersTransactionalSqlScriptsTests org.springframework.test.context.jdbc.NonTransactionalSqlScriptsTests org.springframework.test.context.jdbc.RepeatableSqlAnnotationSqlScriptsChildTests org.springframework.test.context.jdbc.RepeatableSqlAnnotationSqlScriptsParentTests org.springframework.test.context.jdbc.TransactionalAfterTestMethodSqlScriptsTests org.springframework.test.context.jdbc.TransactionalInlinedStatementsSqlScriptsTests org.springframework.test.context.junit.jupiter.nested.ContextHierarchyNestedTests$NestedTestCaseWithInheritedConfigTests org.springframework.test.context.junit.jupiter.nested.ContextHierarchyNestedTests$NestedTestCaseWithInheritedConfigTests$DoubleNestedTestCaseWithOverriddenConfigTests org.springframework.test.context.junit.jupiter.nested.ContextHierarchyNestedTests$NestedTestCaseWithInheritedConfigTests$DoubleNestedTestCaseWithOverriddenConfigTests$TripleNestedWithInheritedConfigAndTestInterfaceTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$ConfigOverriddenByDefaultTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests$TripleNestedWithInheritedConfigAndTestInterfaceTests org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests org.springframework.test.context.junit4.AbsolutePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ClassPathResourceSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ConcreteTransactionalJUnit4SpringContextTests org.springframework.test.context.junit4.InheritedConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.MultipleResourcesSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests org.springframework.test.context.junit4.RelativePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.SpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.annotation.AnnotationConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.rules.TransactionalSqlScriptsSpringRuleTests org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests org.springframework.test.context.testng.ConcreteTransactionalTestNGSpringContextTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.RollbackForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.RollbackForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.transaction.ejb.CommitForRequiredEjbTxDaoTests org.springframework.test.context.transaction.ejb.CommitForRequiresNewEjbTxDaoTests org.springframework.test.context.transaction.ejb.RollbackForRequiredEjbTxDaoTests org.springframework.test.context.transaction.ejb.RollbackForRequiresNewEjbTxDaoTests org.springframework.test.web.servlet.samples.client.context.JavaConfigTests org.springframework.test.web.servlet.samples.client.context.WebAppResourceTests org.springframework.test.web.servlet.samples.client.context.XmlConfigTests org.springframework.test.web.servlet.samples.context.JavaConfigTests org.springframework.test.web.servlet.samples.context.WebAppResourceTests org.springframework.test.web.servlet.samples.context.XmlConfigTests See gh-29122

This commit applies @⁠DisabledInAotMode to all integration test classes in the spring-test module that use @⁠ContextHierarchy since that feature is not supported for AOT optimizations. Current AOT test results for the spring-test module: Test run finished after 6888 ms [ 403 containers found ] [ 57 containers skipped ] [ 346 containers started ] [ 0 containers aborted ] [ 338 containers successful ] [ 8 containers failed ] [ 757 tests found ] [ 98 tests skipped ] [ 654 tests started ] [ 9 tests aborted ] [ 576 tests successful ] [ 69 tests failed ] Failing Test Classes: org.springframework.test.context.configuration.interfaces.SqlConfigInterfaceTests org.springframework.test.context.expression.ExpressionUsageTests org.springframework.test.context.groovy.AbsolutePathGroovySpringContextTests org.springframework.test.context.groovy.GroovySpringContextTests org.springframework.test.context.groovy.RelativePathGroovySpringContextTests org.springframework.test.context.hierarchies.meta.MetaHierarchyLevelTwoTests org.springframework.test.context.hierarchies.standard.TestHierarchyLevelTwoWithBareContextConfigurationInSubclassTests org.springframework.test.context.jdbc.CustomScriptSyntaxSqlScriptsTests org.springframework.test.context.jdbc.GlobalCustomScriptSyntaxSqlScriptsTests org.springframework.test.context.jdbc.InferredDataSourceTransactionalSqlScriptsTests org.springframework.test.context.jdbc.InfrastructureProxyTransactionalSqlScriptsTests org.springframework.test.context.jdbc.MultipleDataSourcesAndTransactionManagersSqlScriptsTests org.springframework.test.context.jdbc.MultipleDataSourcesAndTransactionManagersTransactionalSqlScriptsTests org.springframework.test.context.jdbc.NonTransactionalSqlScriptsTests org.springframework.test.context.jdbc.RepeatableSqlAnnotationSqlScriptsChildTests org.springframework.test.context.jdbc.RepeatableSqlAnnotationSqlScriptsParentTests org.springframework.test.context.jdbc.TransactionalAfterTestMethodSqlScriptsTests org.springframework.test.context.jdbc.TransactionalInlinedStatementsSqlScriptsTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$ConfigOverriddenByDefaultTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests$TripleNestedWithInheritedConfigAndTestInterfaceTests org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests org.springframework.test.context.junit4.AbsolutePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ClassPathResourceSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ConcreteTransactionalJUnit4SpringContextTests org.springframework.test.context.junit4.InheritedConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.MultipleResourcesSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests org.springframework.test.context.junit4.RelativePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.SpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.annotation.AnnotationConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.rules.TransactionalSqlScriptsSpringRuleTests org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests org.springframework.test.context.testng.ConcreteTransactionalTestNGSpringContextTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.CommitForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.RollbackForRequiredEjbTxDaoTestNGTests org.springframework.test.context.testng.transaction.ejb.RollbackForRequiresNewEjbTxDaoTestNGTests org.springframework.test.context.transaction.ejb.CommitForRequiredEjbTxDaoTests org.springframework.test.context.transaction.ejb.CommitForRequiresNewEjbTxDaoTests org.springframework.test.context.transaction.ejb.RollbackForRequiredEjbTxDaoTests org.springframework.test.context.transaction.ejb.RollbackForRequiresNewEjbTxDaoTests See gh-29122

… tests See gh-29122

... to avoid database name conflicts when test classes are executed in a different order. See gh-29122

See gh-29122

Most of the remaining failures are due to @⁠Resource usage. Current AOT test results for the spring-test module: Test run finished after 6140 ms [ 399 containers found ] [ 64 containers skipped ] [ 335 containers started ] [ 0 containers aborted ] [ 333 containers successful ] [ 2 containers failed ] [ 745 tests found ] [ 116 tests skipped ] [ 628 tests started ] [ 1 tests aborted ] [ 589 tests successful ] [ 38 tests failed ] Failing Test Classes: org.springframework.test.context.groovy.AbsolutePathGroovySpringContextTests org.springframework.test.context.groovy.GroovySpringContextTests org.springframework.test.context.groovy.RelativePathGroovySpringContextTests org.springframework.test.context.jdbc.BeforeTestClassSqlScriptsTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$ConfigOverriddenByDefaultTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests org.springframework.test.context.junit.jupiter.nested.TestExecutionListenersNestedTests$InheritedAndExtendedConfigTests$DoubleNestedWithOverriddenConfigTests$TripleNestedWithInheritedConfigAndTestInterfaceTests org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests org.springframework.test.context.junit4.AbsolutePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ClassPathResourceSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ConcreteTransactionalJUnit4SpringContextTests org.springframework.test.context.junit4.InheritedConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.MultipleResourcesSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests org.springframework.test.context.junit4.RelativePathSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.SpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.annotation.AnnotationConfigSpringJUnit4ClassRunnerAppCtxTests org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests org.springframework.test.context.testng.ConcreteTransactionalTestNGSpringContextTests See gh-29122

See gh-29122

sbrannen · 2023-10-22T11:54:04Z

Now that we have AOT testing support in place, we should introduce a way to run end-to-end tests for all Spring TestContext Framework integration tests in the spring-test module.

For the sake of simplicity, we have not introduced a dedicated build for this purpose.

Instead, we currently rely on manual runs of AotIntegrationTests#endToEndTestsForEntireSpringTestModule() within the spring-test module.

Several tests are now annotated with the newly introduced @DisabledInAotMode annotation. Note, however, that @DisabledInAotMode has only been applied for use cases that will never be supported in AOT mode (or are unlikely to be supported in AOT mode, such as @ContextHierarchy and @EJB).

Current results

Test run finished after 5887 ms
[       399 containers found      ]
[        71 containers skipped    ]
[       328 containers started    ]
[         0 containers aborted    ]
[       326 containers successful ]
[         2 containers failed     ]
[       745 tests found           ]
[       123 tests skipped         ]
[       621 tests started         ]
[         1 tests aborted         ]
[       590 tests successful      ]
[        30 tests failed          ]

Failing Test Classes:
org.springframework.test.context.groovy.AbsolutePathGroovySpringContextTests
org.springframework.test.context.groovy.GroovySpringContextTests
org.springframework.test.context.groovy.RelativePathGroovySpringContextTests
org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests
org.springframework.test.context.junit4.AbsolutePathSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.ClassPathResourceSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.ConcreteTransactionalJUnit4SpringContextTests
org.springframework.test.context.junit4.InheritedConfigSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.MultipleResourcesSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests
org.springframework.test.context.junit4.RelativePathSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.SpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.annotation.AnnotationConfigSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests
org.springframework.test.context.testng.ConcreteTransactionalTestNGSpringContextTests

Failure Categories

Uses `@Resource` in test class

To be addressed by Support the use of @Resource in test classes in AOT mode #31733
org.springframework.test.context.groovy.AbsolutePathGroovySpringContextTests
org.springframework.test.context.groovy.GroovySpringContextTests
org.springframework.test.context.groovy.RelativePathGroovySpringContextTests
org.springframework.test.context.junit4.AbsolutePathSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.ClassPathResourceSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.ConcreteTransactionalJUnit4SpringContextTests
org.springframework.test.context.junit4.InheritedConfigSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.MultipleResourcesSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.RelativePathSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.SpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.junit4.annotation.AnnotationConfigSpringJUnit4ClassRunnerAppCtxTests
org.springframework.test.context.testng.ConcreteTransactionalTestNGSpringContextTests

Uses `@PersistenceContext` in test class

To be addressed by Support @PersistenceContext and @PersistenceUnit in test classes in AOT #31442
org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests

Uses `<mvc:annotation-driven />`

To be addressed by <mvc:annotation-driven> should not instantiate bean property values #31472 and ConstructorResolver rejects a candidate if the parameter value is null #31495
org.springframework.test.context.junit4.spr9799.Spr9799XmlConfigTests

Unknown Reason

The following test class passes in AOT mode when run by itself but fails when run with all integration tests in the spring-test module.

org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests

snicoll · 2023-10-23T08:49:02Z

Note, however, that @DisabledInAotMode has only been applied for use cases that will never be supported in AOT mode (or are unlikely to be supported in AOT mode, such as @ContextHierarchy and @ejb).

If that's the case, then we need to revisit #29030 (comment)

sbrannen · 2024-01-05T16:15:11Z

Now that #31733 has been resolved, we only have 2 remaining test classes that fail in AOT mode.

Test run finished after 6534 ms
[       399 containers found      ]
[        71 containers skipped    ]
[       328 containers started    ]
[         0 containers aborted    ]
[       327 containers successful ]
[         1 containers failed     ]
[       745 tests found           ]
[       123 tests skipped         ]
[       622 tests started         ]
[         1 tests aborted         ]
[       615 tests successful      ]
[         6 tests failed          ]

Failing Test Classes:
org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests
org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests

JpaEntityListenerTests still fails because #31442 has not yet been addressed.
ParameterizedDependencyInjectionTests still passes in AOT mode when run by itself but fails when run with all integration tests in the spring-test module. 🤷‍♂️

sbrannen · 2024-05-29T11:44:20Z

Update

Test run finished after 6250 ms
[       425 containers found      ]
[        71 containers skipped    ]
[       353 containers started    ]
[         0 containers aborted    ]
[       346 containers successful ]
[         7 containers failed     ]
[       841 tests found           ]
[       123 tests skipped         ]
[       698 tests started         ]
[         1 tests aborted         ]
[       684 tests successful      ]
[        13 tests failed          ]

Failing Test Classes:
org.springframework.test.context.bean.override.convention.TestBeanByTypeIntegrationTests
org.springframework.test.context.bean.override.convention.TestBeanInheritanceIntegrationTests$ConcreteTestBeanIntegrationTests
org.springframework.test.context.bean.override.convention.TestBeanIntegrationTests
org.springframework.test.context.bean.override.convention.TestBeanIntegrationTests$TestBeanNested
org.springframework.test.context.bean.override.convention.TestBeanIntegrationTests$TestBeanNested2
org.springframework.test.context.bean.override.mockito.MockitoBeanIntegrationTests
org.springframework.test.context.bean.override.mockito.MockitoSpyBeanIntegrationTests
org.springframework.test.context.bean.override.mockito.MockitoSpyBeanIntegrationTests$MockitoBeanNested
org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests
org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests

Bean Override tests are currently not supported in AOT mode.
JpaEntityListenerTests still fails because #31442 has not yet been addressed.
ParameterizedDependencyInjectionTests still passes in AOT mode when run by itself but fails when run with all integration tests in the spring-test module. 🤷‍♂️

See gh-29122

This commit also ensures that @⁠Bean methods are declared within @⁠Configuration classes instead of within test classes. See gh-29122

As a follow up to the previous commit (31f8e12), this commit polishes bean override tests and revises them with a focus on AOT testing support and simplified maintenance. - Introduce EngineTestKitUtils to simplify working with JUnit's EngineTestKit. - Use idiomatic EngineTestKit APIs to simplify assertions on EngineTestKit results. - Introduce BeanOverrideTestSuite to simplify running all bean override tests within the IDE. - Separate failure and success scenario tests, so that failure tests do not launch the JUnit Platform to run tests using the Spring TestContext Framework (TCF) within a test class that itself uses the TCF. - Make AbstractTestBeanIntegrationTestCase actually abstract. - Rename test case classes to give them meaningful names and simplify understanding of what's being tested. - Ensure tests for @⁠MockitoSpyBean functionality use @⁠MockitoSpyBean instead of @⁠MockitoBean. - Declare @⁠Configuration classes local to @⁠SpringJUnitConfig test classes whenever possible. See gh-29122 See gh-32925

sbrannen · 2024-05-31T10:32:46Z

Update

After recent revisions to our bean override tests, the results are now as follows.

Test run finished after 5842 ms
[       427 containers found      ]
[        71 containers skipped    ]
[       355 containers started    ]
[         0 containers aborted    ]
[       350 containers successful ]
[         5 containers failed     ]
[       838 tests found           ]
[       123 tests skipped         ]
[       700 tests started         ]
[         1 tests aborted         ]
[       693 tests successful      ]
[         6 tests failed          ]

Failing Test Classes:
org.springframework.test.context.bean.override.convention.TestBeanByTypeIntegrationTests
org.springframework.test.context.bean.override.convention.TestBeanInheritanceIntegrationTests$ConcreteTestBeanIntegrationTests
org.springframework.test.context.bean.override.mockito.MockitoBeanByTypeIntegrationTests
org.springframework.test.context.bean.override.mockito.MockitoBeanIntegrationTests
org.springframework.test.context.junit.jupiter.orm.JpaEntityListenerTests
org.springframework.test.context.junit4.ParameterizedDependencyInjectionTests

AOT issues with Bean Override tests should be addressed in #32925.
JpaEntityListenerTests still fails because #31442 has not yet been addressed.
ParameterizedDependencyInjectionTests still passes in AOT mode when run by itself but fails when run with all integration tests in the spring-test module. 🤷‍♂️

sbrannen added in: test Issues in the test module type: task A general task theme: aot An issue related to Ahead-of-time processing labels Sep 9, 2022

sbrannen added this to the 6.0.0-M6 milestone Sep 9, 2022

sbrannen self-assigned this Sep 9, 2022

sbrannen added a commit that referenced this issue Sep 10, 2022

Improve tips for using endToEndTestsForEntireSpringTestModule()

7605ed7

See gh-29122

sbrannen modified the milestones: 6.0.0-M6, 6.0.0-RC1 Sep 10, 2022

sbrannen modified the milestones: 6.0.0-RC1, 6.0.0-RC2 Oct 4, 2022

sbrannen modified the milestones: 6.0.0-RC2, 6.0.0 Oct 12, 2022

sbrannen modified the milestones: 6.0.0, 6.0.x Nov 13, 2022

sbrannen modified the milestones: 6.0.x, 6.x Backlog Mar 7, 2023

sbrannen modified the milestones: 6.x Backlog, 6.1.x Oct 16, 2023

sbrannen added a commit that referenced this issue Oct 16, 2023

Apply @⁠DisabledInAotMode to remaining @⁠ContextHierarchy integration…

0adec6d

… tests See gh-29122

sbrannen added a commit that referenced this issue Oct 16, 2023

Consistently use generated name in embedded databases

9cac5ee

... to avoid database name conflicts when test classes are executed in a different order. See gh-29122

sbrannen added a commit that referenced this issue Oct 16, 2023

Filter out TestNG @⁠EJB tests in AOT end-to-end tests

88cd070

See gh-29122

sbrannen added a commit that referenced this issue Oct 22, 2023

Ensure integration tests pass in AOT mode

563112c

See gh-29122

sbrannen changed the title ~~Verify AOT testing support for all tests in the spring-test module~~ Verify AOT testing support for all tests in the spring-test module Oct 22, 2023

sbrannen modified the milestones: 6.1.x, 6.2.x Jan 5, 2024

bclozel changed the title ~~Verify AOT testing support for all tests in the spring-test module~~ Verify AOT testing support for all tests in the spring-test module Feb 14, 2024

sbrannen changed the title ~~Verify AOT testing support for all tests in the spring-test module~~ Verify AOT testing support for all tests in the spring-test module Feb 14, 2024

sbrannen modified the milestones: 6.2.x, 6.2.0-M1, 6.2.0-M2 Mar 18, 2024

sbrannen modified the milestones: 6.2.0-M2, 6.2.x May 4, 2024

sbrannen added a commit that referenced this issue May 29, 2024

Ensure MockMvcTesterIntegrationTests runs in AOT mode

cade3c7

See gh-29122

sbrannen added a commit that referenced this issue May 29, 2024

Demonstrate how to disable Bean Override tests in AOT mode

1f537b9

See gh-29122

sbrannen added a commit that referenced this issue May 29, 2024

Separate failure and success scenario tests for @⁠TestBean

31f8e12

This commit also ensures that @⁠Bean methods are declared within @⁠Configuration classes instead of within test classes. See gh-29122

This was referenced Jun 1, 2024

Introduce dedicated AOT tests for bean override support in the TCF #32932

Closed

Support bean override features with AOT #32933

Open

@TestBean and @MockitoBean tests fail in AOT mode #32925

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Verify AOT testing support for all tests in the `spring-test` module #29122

Verify AOT testing support for all tests in the `spring-test` module #29122

sbrannen commented Sep 9, 2022 •

edited

sbrannen commented Oct 22, 2023 •

edited

snicoll commented Oct 23, 2023

sbrannen commented Jan 5, 2024 •

edited

sbrannen commented May 29, 2024 •

edited

sbrannen commented May 31, 2024 •

edited

Verify AOT testing support for all tests in the spring-test module #29122

Verify AOT testing support for all tests in the spring-test module #29122

Comments

sbrannen commented Sep 9, 2022 • edited

Overview

Related Issues

sbrannen commented Oct 22, 2023 • edited

Current results

Failure Categories

Uses @Resource in test class

Uses @PersistenceContext in test class

Uses <mvc:annotation-driven />

Unknown Reason

snicoll commented Oct 23, 2023

sbrannen commented Jan 5, 2024 • edited

sbrannen commented May 29, 2024 • edited

Update

sbrannen commented May 31, 2024 • edited

Update

Verify AOT testing support for all tests in the `spring-test` module #29122

Verify AOT testing support for all tests in the `spring-test` module #29122

sbrannen commented Sep 9, 2022 •

edited

sbrannen commented Oct 22, 2023 •

edited

Uses `@Resource` in test class

Uses `@PersistenceContext` in test class

Uses `<mvc:annotation-driven />`

sbrannen commented Jan 5, 2024 •

edited

sbrannen commented May 29, 2024 •

edited

sbrannen commented May 31, 2024 •

edited