Component scan fails to find bean candidates in the embedded jar file in META-INF/context.xml for embedded Tomcat application

[osnsergey]

Description
Component Scan can't find annotated beans in the Web application if beans are located in the embedded .jar file.

Reproducer
A small sample application for the issue has been created: GitHub Repository (Java 17, Gradle).

Exception Stack Trace:

21:42:59.048 [main] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory -- Creating shared instance of singleton bean 'localizationController'
21:42:59.070 [main] WARN org.springframework.web.context.support.XmlWebApplicationContext -- Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'localizationController': Unsatisfied dependency expressed through field 'langPackageLoader': No qualifying bean of type 'com.my.app.localization.LangPackageLoader' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
21:42:59.076 [main] ERROR org.springframework.web.context.ContextLoader -- Context initialization failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'localizationController': Unsatisfied dependency expressed through field 'langPackageLoader': No qualifying bean of type 'com.my.app.localization.LangPackageLoader' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:788)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:768)
        at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:146)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:509)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1445)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:339)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:346)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:337)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.instantiateSingleton(DefaultListableBeanFactory.java:1155)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingleton(DefaultListableBeanFactory.java:1121)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:1056)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:987)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:627)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:394)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:274)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:126)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4008)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4436)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:772)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:203)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:415)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:870)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.startup.Tomcat.start(Tomcat.java:437)
        at com.test.app.runner.AppRunner.run(AppRunner.java:42)
        at com.test.app.runner.AppRunner.main(AppRunner.java:131)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at com.test.app.runner.JarLauncher.main(JarLauncher.java:38)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.my.app.localization.LangPackageLoader' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

        at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:2177)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1627)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1552)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:785)
        ... 52 common frames omitted

How to reproduce
Clone https://github.com/osnsergey/Spring62TestWebApp2, use README.md commands to run the sample.

Bean definitions can't be found in the following jar:

spring62webapp.jar!\WEB-INF\lib\localization.jar!\

[jhoeller]

@osnsergey any insight into where PathMatchingResourcePatternResolver goes wrong in this follow-up? I'll look at it in depth tomorrow morning, with the aim to resolve it for the 6.2.4 release which we intend to publish in about 24 hours.

[osnsergey]

@jhoeller , no, I didn't analyze it deeply yet...

[osnsergey]

In Spring 6.1

Stack

"main@1" prio=5 tid=0x1 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.doFindPathMatchingJarResources(PathMatchingResourcePatternResolver.java:744)
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.findPathMatchingResources(PathMatchingResourcePatternResolver.java:601)
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.getResources(PathMatchingResourcePatternResolver.java:335)
	  at org.springframework.context.support.AbstractApplicationContext.getResources(AbstractApplicationContext.java:1520)
	  at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.scanCandidateComponents(ClassPathScanningCandidateComponentProvider.java:457)
 ...

protected Set<Resource> doFindPathMatchingJarResources(Resource rootDirResource, URL rootDirUrl, String subPattern)
			throws IOException {
...
ln745.                  Set<Resource> result = new LinkedHashSet<>(64);
ln746.			for (Enumeration<JarEntry> entries = jarFile.entries(); entries.hasMoreElements();) {
				JarEntry entry = entries.nextElement();
				String entryPath = entry.getName();
				if (entryPath.startsWith(rootEntryPath)) {
					String relativePath = entryPath.substring(rootEntryPath.length());
					if (getPathMatcher().match(subPattern, relativePath)) {
ln751.					        result.add(rootDirResource.createRelative(relativePath));
					}
				}
			}
...
}

on ln746: jarFile = C:\Work\Spring62TestWebApp2\spring62webapp\result\release\spring62webapp.jar!/WEB-INF/lib/localization.jar

rootDirResource = URL [jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/]

rootDirUrl = jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/

subPattern = **/*.class

on ln751 result is filled with all classes from the localization.jar:

result = {LinkedHashSet@3528}  size = 3
 0 = {UrlResource@3927} "URL [jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/LangPackage.class]"
 1 = {UrlResource@3928} "URL [jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/LangPackageLoader.class]"
 2 = {UrlResource@3929} "URL [jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/LangPackageLoaderImpl.class]"

Eventually in the

	private Set<BeanDefinition> scanCandidateComponents(String basePackage) {
		Set<BeanDefinition> candidates = new LinkedHashSet<>();
		try {
			String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX +
					resolveBasePackage(basePackage) + '/' + this.resourcePattern;
ln457.		        Resource[] resources = getResourcePatternResolver().getResources(packageSearchPath);

On ln457 the result from the doFindPathMatchingJarResources is then processed for bean candidates.

---

In Spring 6.2

Stack

"main@1" prio=5 tid=0x1 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.doFindPathMatchingJarResources(PathMatchingResourcePatternResolver.java:803)
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.findPathMatchingResources(PathMatchingResourcePatternResolver.java:713)
	  at org.springframework.core.io.support.PathMatchingResourcePatternResolver.getResources(PathMatchingResourcePatternResolver.java:351)
	  at org.springframework.context.support.AbstractApplicationContext.getResources(AbstractApplicationContext.java:1549)
	  at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.scanCandidateComponents(ClassPathScanningCandidateComponentProvider.java:457)

	protected Set<Resource> doFindPathMatchingJarResources(Resource rootDirResource, URL rootDirUrl, String subPattern)
			throws IOException {
...
		if (separatorIndex >= 0) {
ln811.		        jarFileUrl = urlFile.substring(0, separatorIndex);
			rootEntryPath = urlFile.substring(separatorIndex + 2);  // both separators are 2 chars
			NavigableSet<String> entriesCache = this.jarEntriesCache.get(jarFileUrl);
			if (entriesCache != null) {
				Set<Resource> result = new LinkedHashSet<>(64);
				// Clean root entry path to match jar entries format without "!" separators
ln817.			        rootEntryPath = rootEntryPath.replace(ResourceUtils.JAR_URL_SEPARATOR, "/");
				// Search sorted entries from first entry with rootEntryPath prefix
ln820.			        for (String entryPath : entriesCache.tailSet(rootEntryPath, false)) {
					if (!entryPath.startsWith(rootEntryPath)) {
						// We are beyond the potential matches in the current TreeSet.
ln822.				        	break;
					}
					String relativePath = entryPath.substring(rootEntryPath.length());
					if (getPathMatcher().match(subPattern, relativePath)) {
						result.add(rootDirResource.createRelative(relativePath));
					}
				}
ln829.			        return result;

...

}

rootDirResource = URL [jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/]

rootDirUrl = jar:file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/lib/localization.jar!/com/my/app/localization/

subPattern = **/*.class

on ln811: jarFileUrl = file:/C:/Work/Spring62TestWebApp2/spring62webapp/result/release/spring62webapp.jar

on ln817: rootEntryPath = WEB-INF/lib/localization.jar/com/my/app/localization/

on ln820: entryPath = WEB-INF/lib/log4j-over-slf4j-2.0.12.jar

on ln822 the loop breaks and result isn't filled at all.

Maybe the unconditional return on ln829 is a defect because the code below looks like in Spring 6.1 and could work... But that's just a guess...

[jhoeller]

@osnsergey thanks for the analysis, I'll do a deep dive ASAP.

As for the reproducer, is java -jar spring62webapp.jar meant to fail there? It starts up fine for me locally, as-is.

[osnsergey]

@jhoeller , it should show an exception from the issue description. The application itself doesn't stop.

[jhoeller]

@osnsergey the app seems to bootstrap fine, no exception shown except a Tomcat debug log entry, and with both beans logged as present. This could be order-dependent, maybe my local setup leads to a different order of jar file introspection?

No hurries though, I'm currently investigating how the entriesCache could possibly mismatch as you have shown above. We generally seem to be unable to handle multiple nested jars there, our TreeSet logic expects package entries from a single jar there (which is why it backs out early once the entries do not match anymore). I'll revise this.

[jhoeller]

I'm about to push a revision that falls back to the regular search algorithm if no matching root entry path has been founded in the cache at all. According to some local tests, this addresses your scenario.

[jhoeller]

@osnsergey please give the upcoming 6.2.4 snapshot a try! I hope we have sorted this out for good now.

[osnsergey]

@jhoeller , just checked on test app, the issue looks fixed now. Thanks!

[jhoeller]

Good to hear, thanks for the immediate turnaround!

[osnsergey]

@jhoeller please clarify the 6.2.4 release date.

[bclozel]

@osnsergey it's been rescheduled to March 13th (as you can see on the right hand side column on this page).
We had a chance to release 6.2.4 right before the Spring Boot releases went out last week but missed it because we didn't have enough time/feedback.