Running on Centos with Kafka 0.8.2.1 I’m experiencing periodic disconnects. I’ve enabled keepalives, though I don’t think that’s related to the issue.
The fact that this happens at granularity of 5 minutes and always at 45 seconds after is interesting. This may be some internal network issue, but wanted to see if anyone else had encountered this. [paul.jensen@SLX029 ceres (feature-CERES-1873-kafka-vols *%)]$ grep Disconnected logs/20160217.log | grep -v bootstrap 17:05:45.679 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 18:05:45.674 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 18:40:46.281 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 18:55:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 19:35:45.673 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 19:50:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 20:10:46.286 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 20:50:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 21:00:45.980 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 22:15:45.678 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 22:35:45.673 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected 23:35:45.676 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): CHIVLXSTG073.spottrading.com:9092/1: Receive failed: Disconnected [paul.jensen@SLX029 ceres (feature-CERES-1873-kafka-vols *%)]$ grep Disconnected logs/20160217.log | grep bootstrap 17:15:45.984 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 18:20:45.674 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 18:30:45.983 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 19:15:45.673 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 20:00:45.987 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 20:25:45.676 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 20:35:45.982 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 21:15:45.678 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 21:35:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 21:50:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 22:00:45.978 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 22:50:45.672 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnected 23:50:45.674 INFO [Applications::Risk::Ceres::KafkaProducer::event_cb(176)] Kafka Log ERROR fac:FAIL): chivlxstg073:9092/bootstrap: Receive failed: Disconnectedd Thanks, Paul ________________________________ Paul Jensen Software Developer IV Spot Trading L.L.C 440 South LaSalle St., Suite 2800 Chicago, IL 60605 Office: 312.362.4550 Direct: 312-362-4711 Fax: 312.362.4551 paul.jen...@spottradingllc.com www.spottradingllc.com<http://www.spottradingllc.com/> ________________________________ The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Spot Trading, LLC