I can see the entries processed in the table very fast but after that it takes a long time for the checkpoint update.
Haven't tried other methods of checkpointing yet, we are using DSE on Azure. Thanks, Sourabh On Fri, Oct 2, 2015 at 6:52 AM, Cody Koeninger <c...@koeninger.org> wrote: > Why are you sure it's checkpointing speed? > > Have you compared it against checkpointing to hdfs, s3, or local disk? > > On Fri, Oct 2, 2015 at 1:17 AM, Sourabh Chandak <sourabh3...@gmail.com> > wrote: > >> Hi, >> >> I have a receiverless kafka streaming job which was started yesterday >> evening and was running fine till 4 PM today. Suddenly post that writing of >> checkpoint has slowed down and it is now not able to catch up with the >> incoming data. We are using the DSE stack with Spark 1.2 and Cassandra for >> checkpointing. Spark streaming is done using a backported code. >> >> Running nodetool shows that the Read latency of the cfs keyspace is ~8.5 >> ms. >> >> Can someone please help me resolve this? >> >> Thanks, >> Sourabh >> >> >
