Nissim and Bryan, On closer evaluation, this appears to be a threading issue with the new StandardProxiedEntityEncoder and its use of CharsetEncoder.canEncode(). With CharsetEncoder being stateful, that explains why this can happen intermittently.
I will cancel this release candidate build and put together a Jira and pull request to correct behavior. Thanks for reporting this issue! Regards, David Handermann On Fri, Dec 20, 2024 at 10:40 AM Bryan Bende <bbe...@gmail.com> wrote: > > I just got the same error while on the primary/coordinator node, > investigating... > > On Fri, Dec 20, 2024 at 10:34 AM David Handermann > <exceptionfact...@apache.org> wrote: > > > > Hi Nissim, > > > > Thanks for providing the stack trace. This stack trace seems to point > > to some other issue, as the failure itself relates to character set > > parsing. It sounds like one node is returning some type of error > > response from another node during the request replication process. > > > > If you can look at the user and app logs across all cluster nodes, > > that should provide some additional background for the source of the > > error. > > > > Regards, > > David Handermann > > > > On Fri, Dec 20, 2024 at 9:12 AM Nissim Shiman <nshi...@yahoo.com.invalid> > > wrote: > > > > > > Hello Team, > > > > > > I am running into an issue when running on a three node cluster. If the > > > ui is brought up on one of the non-primary nodes, and refreshed, it give > > > an error: > > > > > > Invalid State > > > Current state = CODING_END, new state = CODING > > > > > > I am interested if others see anything similar. > > > > > > Code was compiled and run on RHEL8 with java-21-openjdk-21.0.5 > > > > > > The nifi-user log has this: > > > > > > 2024-12-20 14:56:55,456 WARN [NiFi Web Server-156] > > > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > > > Current state = CODING_END, new state = CODING. Returning Conflict > > > response.java.lang.IllegalStateException: Current state = CODING_END, new > > > state = CODING at > > > java.base/java.nio.charset.CharsetEncoder.throwIllegalStateException(CharsetEncoder.java:1012) > > > at > > > java.base/java.nio.charset.CharsetEncoder.canEncode(CharsetEncoder.java:924) > > > at > > > java.base/java.nio.charset.CharsetEncoder.canEncode(CharsetEncoder.java:1005) > > > at > > > org.apache.nifi.security.proxied.entity.StandardProxiedEntityEncoder.getSanitizedIdentity(StandardProxiedEntityEncoder.java:76) > > > at > > > org.apache.nifi.security.proxied.entity.StandardProxiedEntityEncoder.getEncodedEntity(StandardProxiedEntityEncoder.java:64) > > > at > > > java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) > > > at > > > java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) > > > at > > > java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) > > > at > > > java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) > > > at > > > java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) > > > at > > > java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > > > at > > > java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) > > > at > > > org.apache.nifi.web.security.ProxiedEntitiesUtils.buildProxiedEntitiesChainString(ProxiedEntitiesUtils.java:114) > > > at > > > org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.updateRequestHeaders(ThreadPoolRequestReplicator.java:249) > > > at > > > org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.forwardToCoordinator(ThreadPoolRequestReplicator.java:333) > > > at > > > org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.forwardToCoordinator(ThreadPoolRequestReplicator.java:324) > > > at > > > org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1039) > > > at > > > org.apache.nifi.web.api.ApplicationResource.replicateNodeResponse(ApplicationResource.java:1008) > > > at > > > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:999) > > > at > > > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:983) > > > at > > > org.apache.nifi.web.api.ApplicationResource.replicate(ApplicationResource.java:958) > > > at > > > org.apache.nifi.web.api.FlowResource.getFlow(FlowResource.java:499) > > > at > > > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > > > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > > > at > > > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) > > > at > > > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) > > > at > > > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) > > > at > > > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) > > > at > > > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) > > > at > > > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274) > > > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) > > > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) > > > at org.glassfish.jersey.internal.Errors.process(Errors.java:292) > > > at org.glassfish.jersey.internal.Errors.process(Errors.java:274) > > > at org.glassfish.jersey.internal.Errors.process(Errors.java:244) > > > at > > > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266) > > > at > > > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:253) > > > at > > > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:696) > > > at > > > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:397) > > > at > > > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:349) > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358) > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312) > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1379) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHolder.handle(ServletHolder.java:736) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1619) > > > at > > > org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:365) > > > at > > > org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:101) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) > > > at > > > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.apache.nifi.web.security.log.AuthenticationUserFilter.doFilterInternal(AuthenticationUserFilter.java:57) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:145) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:94) > > > at > > > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:56) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.apache.nifi.web.security.csrf.CsrfCookieFilter.doFilterInternal(CsrfCookieFilter.java:43) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:117) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.apache.nifi.web.security.csrf.SkipReplicatedCsrfFilter.doFilterInternal(SkipReplicatedCsrfFilter.java:59) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) > > > at > > > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233) > > > at > > > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191) > > > at > > > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:362) > > > at > > > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:278) > > > at > > > org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1591) > > > at > > > org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:44) > > > at > > > org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:208) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1591) > > > at > > > org.eclipse.jetty.ee10.servlets.DoSFilter.doFilterChain(DoSFilter.java:462) > > > at > > > org.apache.nifi.web.server.filter.DataTransferExcludedDoSFilter.doFilterChain(DataTransferExcludedDoSFilter.java:51) > > > at > > > org.eclipse.jetty.ee10.servlets.DoSFilter.doFilter(DoSFilter.java:317) > > > at > > > org.eclipse.jetty.ee10.servlets.DoSFilter.doFilter(DoSFilter.java:282) > > > at > > > org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1591) > > > at > > > org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) > > > at > > > org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1591) > > > at > > > org.apache.nifi.web.server.log.RequestAuthenticationFilter.doFilterInternal(RequestAuthenticationFilter.java:59) > > > at > > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) > > > at > > > org.eclipse.jetty.ee10.servlet.FilterHolder.doFilter(FilterHolder.java:205) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1591) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler$MappedServlet.handle(ServletHandler.java:1552) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletChannel.dispatch(ServletChannel.java:819) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:436) > > > at > > > org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:469) > > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575) > > > at > > > org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717) > > > at > > > org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060) > > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:181) > > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:181) > > > at org.eclipse.jetty.server.Server.handle(Server.java:182) > > > at > > > org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662) > > > at > > > org.eclipse.jetty.util.thread.Invocable$ReadyTask.run(Invocable.java:175) > > > at > > > org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2$1.run(HttpStreamOverHTTP2.java:135) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:195) > > > at > > > org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:209) > > > at > > > org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:156) > > > at > > > org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:442) > > > at > > > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) > > > at > > > org.eclipse.jetty.io.ssl.SslConnection$SslEndPoint.onFillable(SslConnection.java:575) > > > at > > > org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:390) > > > at > > > org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:150) > > > at > > > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) > > > at > > > org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) > > > at > > > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) > > > at > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) > > > at java.base/java.lang.Thread.run(Thread.java:1583) > > > > > > > > > On Thursday, December 19, 2024 at 09:02:34 PM UTC, David Handermann > > > <exceptionfact...@apache.org> wrote: > > > > > > Team, > > > > > > I am pleased to be calling this vote for the source release of Apache > > > NiFi 2.1.0. > > > > > > Please review the following guide for how to verify a release candidate > > > build: > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification > > > > > > The source being voted on the and the convenience binaries are > > > available on the Apache Distribution Repository: > > > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-2.1.0 > > > > > > The build artifacts are available on the Apache Nexus Repository: > > > > > > https://repository.apache.org/content/repositories/orgapachenifi-1303/ > > > > > > Git Tag: nifi-2.1.0-RC2 > > > Git Commit ID: 2f91793272f48efb80d18a01d7086f406b6d9968 > > > GitHub Commit Link: > > > https://github.com/apache/nifi/commit/2f91793272f48efb80d18a01d7086f406b6d9968 > > > > > > Checksums of nifi-2.1.0-source-release.zip > > > > > > SHA512: > > > 374227bbdf9df45ce30266d1e5d86fbd022f88876ac765a29b65dde3f9da7f675cf4e4e313e9b6f77067b73ea7a7cc82ce4599cac04b7a38175c81e46ef1ed2d > > > > > > Release artifacts are signed with the following key: > > > > > > https://people.apache.org/keys/committer/exceptionfactory.asc > > > > > > KEYS file is available on the Apache Distribution Repository: > > > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS > > > > > > Issues resolved for this version: 88 > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12355215 > > > > > > Release note highlights can be found on the project wiki: > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.1.0 > > > > > > The vote will be open for 72 hours. > > > > > > Please download the release candidate and evaluate the necessary items > > > including checking hashes, signatures, build from source, and test. > > > Then please vote: > > > > > > [] +1 Release this package as nifi-2.1.0 > > > [] +0 no opinion > > > [] -1 Do not release this package because... > > >
