Re: Many "Unrecognized message" errors in the logs while nodes are leaving

2014-09-29 Thread Chris Read
We were getting upto ~20 messages per second while nodes were leaving the cluster. We went from 19 nodes down to 15. Ring size is 64, using bitcask as the store. Requests on this cluster were averaging around 10k/sec, spiking to 25k/sec occasionally. Request mix is 99% GET with a few PUT and DEL.

Re: Many "Unrecognized message" errors in the logs while nodes are leaving

2014-09-27 Thread Sean Cribbs
Hi Chris, Those log messages are mostly harmless -- they don't indicate any specific thing going wrong. They only indicate that the process handling the client connection decided to stop waiting for additional internal protocol messages and replied to the client. Since the protocol state machine i

Many "Unrecognized message" errors in the logs while nodes are leaving

2014-09-26 Thread Chris Read
Greetings... I started removing a few nodes from a cluster today, and started to get a lot of the following errors in the logs: @riak_api_pb_server:handle_info:331 Unrecognized message Message is either '{,ok}' or '{,{error,timeout}}' We're running Riak 2.0 on Erlang R16B02-basho5, both built f