Oops, I saw "EOFException" and jumped to "scrub." But your EOF is coming from TCP. Something (almost certainly a non-cassandra process) is connecting to the internal Cassandra communication port (the one that defaults to 7000) and disconnecting.
On Mon, Apr 4, 2011 at 4:14 AM, Kazuo YAGI <ky...@zynga.co.jp> wrote: >>> Solution: upgrade to 0.7.4, run scrub > > Although I upgraded all my cassandra nodes from 0.7.0 to 0.7.4 and > ran nodetool scrub to all keyspaces, this EOFException error messages > didn't go away. Do you have any ideas how to deal with it next? > > Besides, it would be really useful if I could know whether or not > these error messages are ignorable, because our application has been > working well before and after upgrading. > > Thanks, > Kazuo > > (11/03/28 11:19), Kazuo YAGI wrote: >> >> I really thank you for your information. >> I'll try to upgrade 0.7.4 and run scrub. >> >> (11/03/25 22:39), Jonathan Ellis wrote: >>> >>> from NEWS.txt: >>> >>> Upgrading >>> --------- >>> - 0.7.1 and 0.7.2 shipped with a bug that caused incorrect row-level >>> bloom filters to be generated when compacting sstables generated >>> with earlier versions. This would manifest in IOExceptions during >>> column name-based queries. 0.7.3 provides "nodetool scrub" to >>> rebuild sstables with correct bloom filters, with no data lost. >>> (If your cluster was never on 0.7.0 or earlier, you don't have to >>> worry about this.) Note that nodetool scrub will snapshot your >>> data files before rebuilding, just in case. >>> >>> Solution: upgrade to 0.7.4, run scrub >>> >>> 2011/3/25 Kazuo YAGI<ky...@zynga.co.jp>: >>>> >>>> Hi everyone, >>>> >>>> I'm running 10 cassandra nodes in total and updated 5 nodes >>>> from 0.7.0 to 0.7.2 one by one like below. >>>> >>>> # sudo /etc/init.d/cassandra stop >>>> # sudo yum update cassandra-0.7.2-4 >>>> # sudo /etc/init.d/cassandra start >>>> >>>> After updating, those 5 updated nodes keep outputting error messages to >>>> /var/lib/cassandra/log/system.log every exactly 5 minutes. >>>> >>>> Do you have any idea about this error message? Thanks for your support. >>>> >>>> /var/lib/cassandra/log/system.log >>>> ------ >>>> ERROR [Thread-45] 2011-03-23 21:21:29,125 >>>> AbstractCassandraDaemon.java (line 114) Fatal exception in thread >>>> Thread[Thread-45,5,main] >>>> java.io.IOError: java.io.EOFException >>>> at >>>> >>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:73) >>>> >>>> Caused by: java.io.EOFException >>>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>>> at >>>> >>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:61) >>>> >>>> ERROR [Thread-51] 2011-03-23 21:26:29,227 >>>> AbstractCassandraDaemon.java (line 114) Fatal exception in thread >>>> Thread[Thread-51,5,main] >>>> java.io.IOError: java.io.EOFException >>>> at >>>> >>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:73) >>>> >>>> Caused by: java.io.EOFException >>>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>>> at >>>> >>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:61) >>>> >>>> ------ >>>> >>>> Best, >>>> Kazuo YAGI >>>> >>> >>> >>> >> >> > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
