Component scan fails to find jar entries in WEB-INF/classes with embedded Tomcat

[osnsergey]

Description
Component Scan can't find annotated beans in the Web application.

Reproducer
A small sample application for the issue has been created: GitHub Repository (Java 17, Gradle).

Exception Stack Trace:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'beanFromXML' defined in ServletContext resource [/WEB-INF/applicationContext.xml]: Cannot resolve reference to bean 'beanFromScan' while setting bean property 'beanFromScan'
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:377)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:135)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1711)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1460)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:307)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.instantiateSingleton(DefaultListableBeanFactory.java:1122)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingleton(DefaultListableBeanFactory.java:1093)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:1030)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:987)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:627)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:394)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:274)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:126)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4008)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4436)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:772)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:203)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:415)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:870)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.startup.Tomcat.start(Tomcat.java:437)
        at com.test.app.runner.AppRunner.run(AppRunner.java:42)
        at com.test.app.runner.AppRunner.main(AppRunner.java:131)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at com.test.app.runner.JarLauncher.main(JarLauncher.java:38)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'beanFromScan' available
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:925)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1361)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:365)
        ... 51 common frames omitted

How to reproduce
Clone https://github.com/osnsergey/Spring62TestWebApp, use README.md commands to run the sample.

[jhoeller]

@osnsergey this looks like a highly custom launcher which makes it hard to reason about on my end. There is nothing wrong with such a custom setup, I would just appreciate some debugging on your end: specifically, what kind of resources PathMatchingResourcePatternResolver sees in its code path and why it does not match the class file that you expect it to match.

PathMatchingResourcePatternResolver has been revised quite a bit in 6.2, so I'm not surprised by a side effect here. We've fixed a few things in 6.2.2 already, largely around jar caching. If you could point me to where PathMatchingResourcePatternResolver still mismatches in your scenario (even just a reasonable guess), that would make it much easier to fix this for 6.2.3 (due in two weeks).

[osnsergey]

@jhoeller the debugging shows the following:

1. rootDirResources is filled by ParallelWebappClassLoader from embedded Tomcat (in PathMatchingResourcePatternResolver).

412:	protected Set<Resource> doFindAllClassPathResources(String path) throws IOException {
		Set<Resource> result = new LinkedHashSet<>(16);
		ClassLoader cl = getClassLoader();
		Enumeration<URL> resourceUrls = (cl != null ? cl.getResources(path) : ClassLoader.getSystemResources(path));
		while (resourceUrls.hasMoreElements()) {
			URL url = resourceUrls.nextElement();
			result.add(convertClassLoaderURL(url));
		}
		if (!StringUtils.hasLength(path)) {
			// The above result is likely to be incomplete, i.e. only containing file system references.
			// We need to have pointers to each of the jar files on the class path as well...
			addAllClassLoaderJarRoots(cl, result);
		}
		return result;
	}

cl (default web app class loader) is object of class ParallelWebappClassLoader.

rootDirResources = getResources(rootDirPath); // line 689 in PathMatchingResourcePatternResolver

the resource URL for the base path resource in the rootDirResources contains ! at the end of WEB-INF/classes:

URL [jar:file:/C:/Work//Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/classes!/com/test/app/]

this URL where the Component Scan should get annotated bean candidates.

2. jarEntriesCache is filled by Spring

this.jarEntriesCache.put(jarFileUrl, entriesCache); //line 900 in PathMatchingResourcePatternResolver

the entriesCache doesn't contain ! at the end of WEB-INF/classes:

0 = "META-INF/"
1 = "META-INF/MANIFEST.MF"
2 = "META-INF/context.xml"
3 = "META-INF/services/"
4 = "META-INF/services/org.apache.juli.logging.Log"
5 = "WEB-INF/"
6 = "WEB-INF/applicationContext.xml"
7 = "WEB-INF/classes/"
8 = "WEB-INF/classes/com/"
9 = "WEB-INF/classes/com/test/"
10 = "WEB-INF/classes/com/test/app/"
11 = "WEB-INF/classes/com/test/app/runner/"

3. Here (in PathMatchingResourcePatternResolver):

810:		if (separatorIndex >= 0) {
			jarFileUrl = urlFile.substring(0, separatorIndex);
			rootEntryPath = urlFile.substring(separatorIndex + 2);  // both separators are 2 chars
			NavigableSet<String> entriesCache = this.jarEntriesCache.get(jarFileUrl);
			if (entriesCache != null) {
				Set<Resource> result = new LinkedHashSet<>(64);
				// Search sorted entries from first entry with rootEntryPath prefix
817:				for (String entryPath : entriesCache.tailSet(rootEntryPath, false)) {
818:					if (!entryPath.startsWith(rootEntryPath)) {
						// We are beyond the potential matches in the current TreeSet.
820:						break;
					}

urlFile contains file:/C:/Work/Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/classes!/com/test/app/
jarFileUrl contains file:/C:/Work/Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar
rootEntryPath contains WEB-INF/classes!/com/test/app/
entryPath contains WEB-INF/classes/

The code on lines 817 - 820 skips bean candidates from the com/test/app base path because rootEntryPath contains ! after WEB-INF/classes but the entriesCache doesn't contain the ! after WEB-INF/classes.

[jhoeller]

Thanks for the thorough analysis! I'll try to cover this scenario for 6.2.3.

[jhoeller]

It seems that we simply need to clean the path before matching it against jar entries in the cache. I haven't seen any other encounters of that difference in the common code paths there, so I'm going with the simplest possible change. This will be available in the upcoming 6.2.3 snapshot; it makes your repro setup pass for me locally.

[jhoeller]

@osnsergey it'd be great if you could give a 6.2.3 snapshot a try with your full application as well (in advance of the 6.2.3 release on Feb 13), just to make sure we're not missing anything that keeps breaking your setup.

[osnsergey]

@jhoeller , I've tried to run full application on 6.2.3-SNAPSHOT build and the issue with component scan looks resolved. Thank you for the quick fix!