Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AsyncRequestNotUsableException thrown and logged after upgrade from Spring Boot 3.2.3 to Spring Boot 3.2.4 #32509

Closed
vap78 opened this issue Mar 21, 2024 · 8 comments
Closed

AsyncRequestNotUsableException thrown and logged after upgrade from Spring Boot 3.2.3 to Spring Boot 3.2.4 #32509

vap78 opened this issue Mar 21, 2024 · 8 comments
Assignees
@rstoyanchev
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket)

Comments

@vap78
Copy link

vap78 commented Mar 21, 2024 •
edited by wilkinsona

Hi

after upgrading to spring boot 3.2.4 from 3.2.3 our applications start logging this exception over and over (I suspect on every request):

org.springframework.web.context.request.async.AsyncRequestNotUsableException: ServletOutputStream failed to write: java.io.IOException: Connection reset by peer
        at org.springframework.web.context.request.async.StandardServletAsyncWebRequest$LifecycleHttpServletResponse.handleIOException(StandardServletAsyncWebRequest.java:320)
        at org.springframework.web.context.request.async.StandardServletAsyncWebRequest$LifecycleServletOutputStream.write(StandardServletAsyncWebRequest.java:378)
        at org.springframework.util.StreamUtils$NonClosingOutputStream.write(StreamUtils.java:264)
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2203)
        at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:526)
        at nl.collectcar.api.book.model.dto.serialize.JsonUuidSerializer.serialize(JsonUuidSerializer.java:15)
        at nl.collectcar.api.book.model.dto.serialize.JsonUuidSerializer.serialize(JsonUuidSerializer.java:12)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
        at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
        at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
        at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
        at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:479)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:399)
        at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1568)
        at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1061)
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:483)
        at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:114)
        at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:297)
        at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor.handleReturnValue(HttpEntityMethodProcessor.java:245)
        at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:78)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:136)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:925)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:830)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.servlet.v6_0.OpenTelemetryHandlerMappingFilter.doFilter(OpenTelemetryHandlerMappingFilter.java:76)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:735)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at java.base/java.lang.VirtualThread.run(VirtualThread.java:309)
        Suppressed: org.springframework.web.context.request.async.AsyncRequestNotUsableException: Response not usable after response errors.
                at org.springframework.web.context.request.async.StandardServletAsyncWebRequest$LifecycleHttpServletResponse.obtainLockAndCheckState(StandardServletAsyncWebRequest.java:314)
                at org.springframework.web.context.request.async.StandardServletAsyncWebRequest$LifecycleServletOutputStream.write(StandardServletAsyncWebRequest.java:373)
                at org.springframework.util.StreamUtils$NonClosingOutputStream.write(StreamUtils.java:264)
                at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2203)
                at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1227)
                at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:452)
                ... 56 common frames omitted
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Connection reset by peer
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:344)
        at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:773)
        at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:676)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:379)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:357)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:97)
        at org.springframework.web.context.request.async.StandardServletAsyncWebRequest$LifecycleServletOutputStream.write(StandardServletAsyncWebRequest.java:375)
        ... 84 common frames omitted
Caused by: java.io.IOException: Connection reset by peer
        at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
        at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
        at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:137)
        at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:102)
        at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:58)
        at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:542)
        at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:118)
        at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1381)
        at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:764)
        at org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(SocketWrapperBase.java:589)
        at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:533)
        at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:540)
        at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:112)
        at org.apache.coyote.http11.Http11OutputBuffer.doWrite(Http11OutputBuffer.java:193)
        at org.apache.coyote.Response.doWrite(Response.java:616)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:331)
        ... 90 common frames omitted

this did not happen with the 3.2.3 version and after a rollback the exceptions are no longer logged.

The exceptions themselves do not seem to have any functional impact.

Used JVM: Amazon Corretto JDK 21.0.2

Best regards
Vasil
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Mar 21, 2024
@wilkinsona
Copy link
Member

The change in behavior is due to #32340. We'll transfer this to Spring Framework so that they can investigate further.

@bclozel bclozel transferred this issue from spring-projects/spring-boot Mar 21, 2024
@snicoll snicoll added the in: web Issues in web modules (web, webmvc, webflux, websocket) label Mar 21, 2024
@rstoyanchev
Copy link
Contributor

What's actually logging the exception?

Previously, this would have been org.apache.catalina.connector.ClientAbortException. It is now wrapped with AsyncRequestNotUsableException, and that should be handled in DefaultHandlerExceptionResolver by marking it resolved without further handling.

@vap78
Copy link
Author

vap78 commented Mar 23, 2024 •
edited

Hi

looks like this is traced in a custom exception handler annotated with @ControllerAdvice.

What is the recommendation here?

I could simply ignore this specific exception, but what would be the performance impact of exceptions being generated/thrown on every request?

Best regards
Vasil

@rstoyanchev
Copy link
Contributor

In this case the exception only wraps the IOException from the Servlet response. In other words there is already an exception, and such exceptions should be raised only when something goes wrong with the network, not for every request.

The exception implies the response is no longer usable, and you can't send any more data. You can treat it in the same way that you would the underlying ClientAbortException or any IOException from the response.

I'm closing as there is nothing for us to do, but feel free to comment if necessary.

@rstoyanchev rstoyanchev closed this as not planned Won't fix, can't repro, duplicate, stale Mar 25, 2024
@rstoyanchev rstoyanchev removed the status: waiting-for-triage An issue we've not yet triaged or decided on label Mar 25, 2024
@hw207165
Copy link

Hi

looks like this is traced in a custom exception handler annotated with @ControllerAdvice.

What is the recommendation here?

I could simply ignore this specific exception, but what would be the performance impact of exceptions being generated/thrown on every request?

Best regards Vasil

How did you solve it？I am facing the same problem. Thanks.

@danwashusen
Copy link

danwashusen commented Apr 28, 2024 •
edited

We are also seeing this error after an upgrade, it doesn't seem to happen for every request, just those that have used response.sendRedirect(...). In our case we just see 'Response not usable after response errors.' logged with a nasty error message (no IO errors)... the client gets the redirect, so the issue is only server side...

@vap78
Copy link
Author

vap78 commented May 17, 2024

Hi
looks like this is traced in a custom exception handler annotated with @ControllerAdvice.
What is the recommendation here?
I could simply ignore this specific exception, but what would be the performance impact of exceptions being generated/thrown on every request?
Best regards Vasil

How did you solve it？I am facing the same problem. Thanks.

We didn't .

The only solution I see is to suppress the exception in the custom exception handler.
But I don't understand, why is it being thrown as we did not changes, except the Spring boot upgrade.

@vap78
Copy link
Author

vap78 commented May 17, 2024

From what I see what has changed is the exception type thrown.

Before spring-web:6.1.5 a ClientAbortException was thrown. Since 6.1.5 a AsyncRequestNotUsableException.

So adding AsyncRequestNotUsableException to the exception handler will restore the old state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rstoyanchev rstoyanchev

Labels
in: web Issues in web modules (web, webmvc, webflux, websocket)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@danwashusen @rstoyanchev @snicoll @wilkinsona @vap78 @spring-projects-issues @hw207165