Re: Spark Streaming not working

2020-04-14 Thread Gerard Maas
Hi, Could you share the code that you're using to configure the connection to the Kafka broker? This is a bread-and-butter feature. My first thought is that there's something in your particular setup that prevents this from working. kind regards, Gerard. On Fri, Apr 10, 2020 at 7:34 PM Debabrat

Re: Spark Streaming not working

2020-04-14 Thread Gabor Somogyi
ption in task 0.5 in stage 0.0 >> (TID 24) >> > java.lang.AssertionError: assertion failed: Failed to get records for >> spark-executor-service-spark-ingestion dice-ingestion 11 0 after polling >> for 12 >> >> Cheers, >> -z >> >> ___

Re: Spark Streaming not working

2020-04-14 Thread Gabor Somogyi
hosh > Sent: Saturday, April 11, 2020 2:25 > To: user > Subject: Re: Spark Streaming not working > > Any solution please ? > > On Fri, Apr 10, 2020 at 11:04 PM Debabrata Ghosh <mailto:mailford...@gmail.com>> wrote: > Hi, > I have a spark streaming applica

Re: Spark Streaming not working

2020-04-14 Thread ZHANG Wei
e 0.0 (TID 24) > java.lang.AssertionError: assertion failed: Failed to get records for > spark-executor-service-spark-ingestion dice-ingestion 11 0 after polling for > 12 Cheers, -z From: Debabrata Ghosh Sent: Saturday, April 11, 2020 2:25 To: user Subjec

Re: Spark Streaming not working

2020-04-10 Thread Debabrata Ghosh
Any solution please ? On Fri, Apr 10, 2020 at 11:04 PM Debabrata Ghosh wrote: > Hi, > I have a spark streaming application where Kafka is producing > records but unfortunately spark streaming isn't able to consume those. > > I am hitting the following error: > > 20/04/10 17:28:04 ERROR E

Re: Spark Streaming not working

2020-04-10 Thread Chenguang He
unsubscribe

Re: Spark Streaming not working

2020-04-10 Thread Debabrata Ghosh
Yes the Kafka producer is producing records from the same host - Rechecked Kafka connection and the connection is there. Came across this URL but unable to understand it https://stackoverflow.com/questions/42264669/spark-streaming-assertion-failed-failed-to-get-records-for-spark-executor-a-gro On

Re: Spark Streaming not working

2020-04-10 Thread Srinivas V
Check if your broker details are correct, verify if you have network connectivity to your client box and Kafka broker server host. On Fri, Apr 10, 2020 at 11:04 PM Debabrata Ghosh wrote: > Hi, > I have a spark streaming application where Kafka is producing > records but unfortunately spa

Re: Spark Streaming not working in YARN mode

2014-11-20 Thread Akhil Das
Cool On 20 Nov 2014 22:01, "kam lee" wrote: > Yes, fixed by setting --executor-cores to 2 or higher. > > Thanks a lot! Really appreciate it! > cloud > > On Wed, Nov 19, 2014 at 10:48 PM, Akhil Das > wrote: > >> Make sure the executor cores are set to a value which is >= 2 while >> submitting the

Re: Spark Streaming not working in YARN mode

2014-11-19 Thread Akhil Das
Make sure the executor cores are set to a value which is >= 2 while submitting the job. Thanks Best Regards On Thu, Nov 20, 2014 at 10:36 AM, kam lee wrote: > I created a simple Spark Streaming program - it received numbers and > computed averages and sent the results to Kafka. > > It worked pe