Hi, All
I am running a trident topo, and having such error, and it is strange that
I can't see the error on storm UI, but I see it on worker-6703.log of one
node,
2015-03-05T17:29:36.859-0800 b.s.m.n.Client [INFO] failed to send requests
to visible-alley/10.100.98.105:6703:
java.nio.channels.ClosedChannelException: null
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:405)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:373)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.NioWorker.read(NioWorker.java:93)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[storm-core-0.9.3.jar:0.9.3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_65]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
2015-03-05T17:29:36.859-0800 b.s.m.n.Client [INFO] failed to send requests
to visible-alley/10.100.98.105:6703:
java.nio.channels.ClosedChannelException: null
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:405)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:373)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.NioWorker.read(NioWorker.java:93)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[storm-core-0.9.3.jar:0.9.3]
at
org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[storm-core-0.9.3.jar:0.9.3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_65]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
and such error on another node:
2015-03-05T17:33:52.892-0800 b.s.d.worker [ERROR] Error on initialization
of server mk-worker
java.io.FileNotFoundException: File
'/srv/tmpvar/storm/data/supervisor/stormdist/eventsStreamerv1-65-1425604313/stormconf.ser'
does not exist
at
org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
~[commons-io-2.4.jar:2.4]
at
org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763)
~[commons-io-2.4.jar:2.4]
at
backtype.storm.config$read_supervisor_storm_conf.invoke(config.clj:212)
~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.worker$worker_data.invoke(worker.clj:182)
~[storm-core-0.9.3.jar:0.9.3]
at
backtype.storm.daemon.worker$fn__3743$exec_fn__1108__auto____3744.invoke(worker.clj:363)
~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.applyToHelper(AFn.java:185)
[clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at clojure.core$apply.invoke(core.clj:617) ~[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.worker$fn__3743$mk_worker__3799.doInvoke(worker.clj:354)
[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.RestFn.invoke(RestFn.java:512)
[clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:461)
[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.applyToHelper(AFn.java:172)
[clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source)
[storm-core-0.9.3.jar:0.9.3]
2015-03-05T17:33:52.902-0800 b.s.util [ERROR] Halting process: ("Error on
initialization")
java.lang.RuntimeException: ("Error on initialization")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325)
[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.RestFn.invoke(RestFn.java:423)
[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.worker$fn__3743$mk_worker__3799.doInvoke(worker.clj:354)
[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.RestFn.invoke(RestFn.java:512)
[clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:461)
[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.AFn.applyToHelper(AFn.java:172)
[clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source)
[storm-core-0.9.3.jar:0.9.3]
This is my yaml,
storm.zookeeper.servers:
- "10.100.98.100"
- "10.100.98.101"
- "10.100.98.102"
- "10.100.98.103"
- "10.100.98.104"
- "10.100.98.105"
- "10.100.98.106"
storm.zookeeper.port: 2181
#nimbus.host: "localhost"
nimbus.thrift.port: 6627
ui.port: 8080
nimbus.host: "10.100.98.103"
storm.local.dir: "/srv/tmpvar/storm/data"
#java.library.path: "/usr/local/lib"
java.library.path: "/usr/lib/jvm/java-7-openjdk-amd64"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"
supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
worker.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"
Do I need to add netty stuff?
thanks
AL
