Joseph Percivall created NIFI-3839:
--------------------------------------
Summary: Problem accessing provenance initially when upgrading
from 1.1.2 to 1.2.0
Key: NIFI-3839
URL: https://issues.apache.org/jira/browse/NIFI-3839
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.2.0
Environment: centos 7, docker instance, OpenJDK 1.8
Reporter: Joseph Percivall
Priority: Minor
Upgraded a long running dev docker 1.1.2 instance to 1.2.0 RC1. Did not change
any configuration. After leaving it running for a couple days, when I attempted
to access provenance either by right-clicking the component or by accessing it
through the hamburger menu I got a 500 error due to a NPE. I see the stacktrace
in the window that should have the provenance events (stacktraces from nifi-app
log below). Upgraded the version to RC2 (restarting the container) and
afterwards it worked.
There are no tickets resolved in RC2 that appear to relate to this issue.
One thing of note relating to provenance, there would have been events that
expired between being started with 1.2.0-RC1 and the provenance being queried.
The docker container was running the latest version of Centos7 with OpenJdk
java 1.8.0 and the provenance volume was mounted (so it persisted through
restarts). Also, the instance is secure with certs and I had the correct
policies to access the provenance.
I have not been able to reproduce and restarting the instance appeared to
resolve the issue. I am logging an issue relating to the RC because I believe
there may be an issue related to upgrading 1.1.2 -> 1.2.0 and provenance events
expiring while keeping an old prov repo but is solved by restarting.
For the stacktraces, I pulled a couple lines before/after the actual stacktrace
in order to provide context and maybe some pertinent context.
Stacktrace 1:
{code}
2017-05-08 14:05:36,943 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
Event Repository - Minimum Event ID now 589858079
2017-05-08 14:05:36,943 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action
org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on Provenance
Event file ./provenance_repository/589858075.prov.gz in 1 millis
2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
./provenance_repository/589858075.prov.gz
2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Table-of-Contents
file ./provenance_repository/toc/589858075.toc
2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action
org.apache.nifi.provenance.expiration.FileRemovalAction@16a8507e on Provenance
Event file ./provenance_repository/589858075.prov.gz in 402049 nanos
2017-05-08 14:05:38,879 WARN [NiFi Web Server-18211]
org.eclipse.jetty.server.HttpChannel /nifi/provenance
javax.servlet.ServletException: org.eclipse.jetty.servlet.ServletHolder$1:
java.lang.NullPointerException
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:138)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
at
org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at
org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:122)
at
org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.invoke(ExecutingExecutionStrategy.java:58)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:201)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:133)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.servlet.ServletHolder$1:
java.lang.NullPointerException
at
org.eclipse.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:596)
at
org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:655)
at
org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:498)
at
org.eclipse.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:785)
at
org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:770)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:538)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
at
org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
... 3 common frames omitted
Caused by: java.lang.NullPointerException: null
at
org.apache.jasper.servlet.JspServlet.handleMissingResource(JspServlet.java:397)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:387)
at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:138)
at
org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
... 32 common frames omitted
2017-05-08 14:05:39,462 INFO [Provenance Maintenance Thread-1]
o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
Event Repository - Minimum Event ID now 589858259
2017-05-08 14:05:39,462 INFO [Provenance Maintenance Thread-1]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action
org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on Provenance
Event file ./provenance_repository/589858080.prov.gz in 10 millis
{code}
Stacktrace 2:
{code}
2017-05-08 14:07:21,100 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
Event Repository - Minimum Event ID now 589873378
2017-05-08 14:07:21,100 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action
org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on Provenance
Event file ./provenance_repository/589873199.prov.gz in 11 millis
2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
./provenance_repository/589873199.prov.gz
2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Table-of-Contents
file ./provenance_repository/toc/589873199.toc
2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action
org.apache.nifi.provenance.expiration.FileRemovalAction@16a8507e on Provenance
Event file ./provenance_repository/589873199.prov.gz in 562266 nanos
2017-05-08 14:07:21,379 WARN [NiFi Web Server-18263]
org.eclipse.jetty.server.HttpChannel /nifi/provenance
javax.servlet.ServletException: org.eclipse.jetty.servlet.ServletHolder$1:
java.lang.NullPointerException
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:138)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
at
org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at
org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:122)
at
org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.invoke(ExecutingExecutionStrategy.java:58)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:201)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:133)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.servlet.ServletHolder$1:
java.lang.NullPointerException
at
org.eclipse.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:596)
at
org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:655)
at
org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:498)
at
org.eclipse.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:785)
at
org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:770)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:538)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
at
org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
... 3 common frames omitted
Caused by: java.lang.NullPointerException: null
at
org.apache.jasper.servlet.JspServlet.handleMissingResource(JspServlet.java:397)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:387)
at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:138)
at
org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
... 32 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
