Skip to content

No early bean initialization from EmbeddedKafkaContextCustomizer #2870

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
garyrussell opened this issue Oct 31, 2023 · 6 comments · Fixed by #2872
Closed

No early bean initialization from EmbeddedKafkaContextCustomizer #2870

garyrussell opened this issue Oct 31, 2023 · 6 comments · Fixed by #2872
Assignees
Milestone

Comments

@garyrussell
Copy link
Contributor

See spring-projects/spring-framework#31453 (comment)

While we don't support the embedded broker in native images; the problem seems to be that the broker is not shut down properly.

@garyrussell garyrussell added this to the Backlog milestone Oct 31, 2023
@snicoll
Copy link
Member

snicoll commented Oct 31, 2023

The problem is rather that the broker is starting "manually" as part of a callback that should not start any bean, see

beanFactory.initializeBean(embeddedKafkaBroker, EmbeddedKafkaBroker.BEAN_NAME);
beanFactory.registerSingleton(EmbeddedKafkaBroker.BEAN_NAME, embeddedKafkaBroker);

Customize the supplied ConfigurableApplicationContext after bean definitions have been loaded into the context but before the context has been refreshed.

I think it should rather contribute a RootBeanDefinition. Unfortunately, the contract of this method is a bit strange as it should provide a registry, not a context since it has not been refreshed yet.

Here is an example in Spring Boot of what I mean: https://github.com/spring-projects/spring-boot/blob/8f2ec227389391fdd173db0ab64f26abd2752f20/spring-boot-project/spring-boot-test/src/main/java/org/springframework/boot/test/web/client/TestRestTemplateContextCustomizer.java#L62

@artembilan
Copy link
Member

Just debugged one of the test using @EmbeddedKafka. The EmbeddedKafkaZKBroker.destroy() is called and all the brokers and Zookeeper are shouted down in normal JVM.
I fail to find that @DisabledInAotMode to mark our @EmbeddedKafka, but I see that mentioned TestRestTemplateContextCustomizer uses if (AotDetector.useGeneratedArtifacts()) {.
Maybe that would be enough for us as well.
At the same time I don't see too much difference with what we have so far with a ((DefaultSingletonBeanRegistry) beanFactory).registerDisposableBean(EmbeddedKafkaBroker.BEAN_NAME, embeddedKafkaBroker); and what Spring Boot does:

	if (beanFactory instanceof BeanDefinitionRegistry registry) {
			registerTestRestTemplate(registry);
		}

Even if we are about migrating to the RootBeanDefinition, we still need a BeanDefinitionRegistry registry.
So, I don't understand what we are pursuing here.

Thanks

@snicoll
Copy link
Member

snicoll commented Oct 31, 2023

Again, this isn't a problem with shutdown. It's not even a problem with AOT actually even if it surfaces only when one tries to process the test context using Spring AOT.

I don't see too much difference with what we have [...] and what Spring Boot does

There's a huge difference. Spring Kafka creates a singleton and force start it manually at a time where only the context can be customized before the refresh (see javadoc). Spring Boot registers a bean definition that's going to be handled just like any other bean at the appropriate time.

Even if we are about migrating to the RootBeanDefinition, we still need a BeanDefinitionRegistry registry.

Correct and there's nothing in the issue I can see that's asking to get a way from casting the registry. I've tried to explain that with "Unfortunately, the contract of this method is a bit strange as it should provide a registry, not a context since it has not been refreshed yet.".

Within customizeContext you should not start any bean as this isn't the appropriate time to do so. The ask is to change the implementation so that it stops doing that.

@artembilan
Copy link
Member

Still not clear why is that harmful.
The EmbeddedKafkaBroker instance does not have any dependencies, therefore that beanFactory.initializeBean(embeddedKafkaBroker, EmbeddedKafkaBroker.BEAN_NAME); is thing by itself just to call its afterPropertiesSet() to start embedded brokers before the real test suite.
And therefore changing it to the:

((BeanDefinitionRegistry) beanFactory).registerBeanDefinition(EmbeddedKafkaBroker.BEAN_NAME, 
				new RootBeanDefinition(EmbeddedKafkaBroker.class, () -> embeddedKafkaBroker));

won't have too much difference for the end result.
(Assuming that we agreed we don't need to rework it to plain BeanDefinition configuration since no AOT support for this type of tests.)

@garyrussell
Copy link
Contributor Author

The difference is initializeBean() creates the broker immediately, whereas registering a bean definition defers the creation until the context is refreshed.

@snicoll
Copy link
Member

snicoll commented Oct 31, 2023

Still not clear why is that harmful.

Because the code starts a bean at a time where the ApplicationContext is configured. It should not do that and this has nothing to do with AOT.

@artembilan artembilan self-assigned this Oct 31, 2023
@artembilan artembilan modified the milestones: Backlog, 3.1.0 Oct 31, 2023
@artembilan artembilan changed the title Investigate Possible Embedded Broker Shutdown Issue No early bean initialziation from EmbeddedKafkaContextCustomizer Oct 31, 2023
artembilan added a commit to artembilan/spring-kafka that referenced this issue Oct 31, 2023
Fixes spring-projects#2870

The `EmbeddedKafkaContextCustomizer` uses `beanFactory.initializeBean()`
which is too early according to the `ApplicationContext` lifecycle
since it is not refreshed yet for `ContextCustomizer`

* Rework the logic in the `EmbeddedKafkaContextCustomizer` to register a `BeanDefinition`
for an `EmbeddedKafkaBroker` to include it into standard `ApplicationContext` lifecycle
* Also mark `@EmbeddedKafka` with a `@DisabledInAotMode` to disallow this kind of tests
in native images
artembilan added a commit to artembilan/spring-kafka that referenced this issue Oct 31, 2023
Fixes spring-projects#2870

The `EmbeddedKafkaContextCustomizer` uses `beanFactory.initializeBean()`
which is too early according to the `ApplicationContext` lifecycle
since it is not refreshed yet for `ContextCustomizer`

* Rework the logic in the `EmbeddedKafkaContextCustomizer` to register a `BeanDefinition`
for an `EmbeddedKafkaBroker` to include it into standard `ApplicationContext` lifecycle
* Also mark `@EmbeddedKafka` with a `@DisabledInAotMode` to disallow this kind of tests
in native images
@garyrussell garyrussell changed the title No early bean initialziation from EmbeddedKafkaContextCustomizer No early bean initialization from EmbeddedKafkaContextCustomizer Nov 13, 2023
garyrussell pushed a commit that referenced this issue Nov 16, 2023
* GH-2870: EmbeddedKafka: register BeanDefinition

Fixes #2870

The `EmbeddedKafkaContextCustomizer` uses `beanFactory.initializeBean()`
which is too early according to the `ApplicationContext` lifecycle
since it is not refreshed yet for `ContextCustomizer`

* Rework the logic in the `EmbeddedKafkaContextCustomizer` to register a `BeanDefinition`
for an `EmbeddedKafkaBroker` to include it into standard `ApplicationContext` lifecycle
* Also mark `@EmbeddedKafka` with a `@DisabledInAotMode` to disallow this kind of tests
in native images

* * Call `embeddedKafkaBroker.afterPropertiesSet()` explicitly in the `EmbeddedKafkaContextCustomizer`
since we need to have broker started and system props initialized before the rest of `ApplicationContext`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants