Hi Spark devs,
I have a new feature to propose and hear opinions on community. Not sure it
is such a big change to worth to step on SPIP, so posting to dev mailing
list instead.
> Feature
Reconfigurable number of partitions on state operators in Structured
Streaming
> Rationalization
Nowadays
Short answer is it isn't necessary.
Long answer is that you aren't just changing from 08 to 10, you're
changing from the receiver based implementation to the direct stream.
Read these:
https://github.com/koeninger/kafka-exactly-once
http://spark.apache.org/docs/latest/streaming-kafka-0-8-integrat
Hi All,
Recently I started migrating the code from kafka08 to kafka010.
in 08 *topics * argument takes care of consuming number of partitions for
each topic.
def createStream(
ssc: StreamingContext,
zkQuorum: String,
groupId: String,
topics: Map[String, Int],
sto
