Re: Kinesis Receiver not respecting spark.streaming.receiver.maxRate

Hi Sai, I am running in local mode and there is only one receiver. Verified that. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kinesis-Receiver-not-respecting-spark-streaming-receiver-maxRate-tp27754p27760.html Sent from the Apache Spark User List mailing

Configuring Kinesis max records limit in KinesisReceiver

I use `KinesisUtil.createStream` to create a DStream from a kinesis stream. By default my spark receiver receives 1 events from the stream. I see that the default value for KCL comes from KCL Configuration

Help required in validating an architecture using Structured Streaming

Hi, We are building an internal analytics application. Kind of an event store. We have all the basic analytics use cases like filtering, aggregation, segmentation etc. So far our architecture used ElasticSearch extensively but that is not scaling anymore. One unique requirement we have is an event

Spark streaming uses lesser number of executors

Hi, I am using spark streaming process some events. It is deployed in standalone mode with 1 master and 3 workers. I have set number of cores per executor to 4 and total num of executors to 24. This means totally 6 executors will be spawned. I have set spread-out to true. So each worker machine get