So you need some state between messages in a partition. You can use
mapPartitions or foreachPartition, which allow you to write code to process
an entire partition.
On Thu, Aug 13, 2015 at 11:48 AM, Priya Ch
wrote:
> Hi Philip,
>
> I have the following requirement -
> I read the streams of data
Hi Philip,
I have the following requirement -
I read the streams of data from various partitions of kafka topic. And then
I union the dstreams and apply hash partitioner so messages of same key
would go into single partition of an rdd, which is ofcourse handled by a
single thread. This way we try
All you'd need to do is *transform* the rdd before writing it, e.g. using
the .map function.
On Thu, Aug 13, 2015 at 11:30 AM, Priya Ch
wrote:
> Hi All,
>
> I have a question in writing rdd to cassandra. Instead of writing entire
> rdd to cassandra, i want to write individual statement into ca
