Ok that worked thanks for the suggestion.
Sent from my iPhone
> On May 24, 2019, at 11:53 AM, SNEHASISH DUTTA
> wrote:
>
> Hi,
> All the keys are similar so they are going to same partition.
> Key->Partition distribution is dependent upon hash calculation add some
> random number to your key
I have Spark code that writes a batch to Kafka as specified here:
https://spark.apache.org/docs/2.4.0/structured-streaming-kafka-integration.html
The code looks like the following:
df.selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)")
\
.write \
.format("kafka") \
.option("
