Re: Data loss in spark job

2018-02-27 Thread Faraz Mateen
lose some data. Can anyone please explain how I can further debug this? Thanks, Faraz On Mon, Feb 26, 2018 at 4:46 PM, Faraz Mateen wrote: > Hi, > > I think I have a situation where spark is silently failing to write data > to my Cassandra table. Let me explain my current situati

Data loss in spark job

2018-02-26 Thread Faraz Mateen
Hi, I think I have a situation where spark is silently failing to write data to my Cassandra table. Let me explain my current situation. I have a table consisting of around 402 million records. The table consists of 84 columns. Table schema is something like this: *id (text) | datetime (tim