On Wed, Apr 6, 2011 at 11:49 PM, Ross Black <[email protected]> wrote:
> Hi, > > I am using the thrift client batch_mutate method with Cassandra 0.7.0 on > Ubuntu 10.10. > > When the size of the mutations gets too large, the client fails with the > following exception: > > Caused by: org.apache.thrift.transport.TTransportException: > java.net.SocketException: Connection reset > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) > at > org.apache.thrift.transport.TFramedTransport.flush(TFramedTransport.java:157) > at > org.apache.cassandra.thrift.Cassandra$Client.send_batch_mutate(Cassandra.java:901) > at > org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:889) > at > com.cxense.cxad.core.persistence.cassandra.store.BatchMutationTask.apply(BatchMutationTask.java:78) > at > com.cxense.cxad.core.persistence.cassandra.store.BatchMutationTask.apply(BatchMutationTask.java:30) > at > com.cxense.cassandra.conn.DefaultCassandraConnectionTemplate.execute(DefaultCassandraConnectionTemplate.java:316) > at > com.cxense.cassandra.conn.DefaultCassandraConnectionTemplate.execute(DefaultCassandraConnectionTemplate.java:257) > at > com.cxense.cxad.core.persistence.cassandra.store.AbstractCassandraStore.writeMutations(AbstractCassandraStore.java:492) > ... 39 more > Caused by: java.net.SocketException: Connection reset > at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96) > at java.net.SocketOutputStream.write(SocketOutputStream.java:136) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105) > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) > ... 47 more > > > It took a while for me to discover that this obscure error message was as a > result of the thrift message exceeding the maximum frame size specified for > the Cassandra server. (default 15MB) > [Using TFastFramedTransport with a max frame size of 15728640 bytes.] > > The poor error message looks like a bug in Thrift server code that assumes > that any transport exception is a connection failure which should drop the > connection. > > * > *My main problem is how to ensure that this does not occur again in > running code. > I could configure the server with a larger frame size, but this size is > effectively arbitrary so there is no guarantee in our code would stop a very > large mutation being sent occasionally. > My current work-around is to break the mutation list into multiple parts, > but to do this correctly I need to track the size of each mutation which is > fairly messy. > * > Is there some way to configure Thrift or Cassandra to deal with messages > that are larger than the max frame size (at either client or server) ?* The only way to do that is to set the frame size higher. Messages cannot be bigger than the maximum frame size. -ryan
