Not sure if kinesis have such flexibility. What else possibilities are there
at transformations level?
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.or
Any example for this please
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
You can also group by the key in the transformation on each batch. But yes
that's faster/easier if it's already partitioned that way.
On Tue, Mar 9, 2021 at 7:30 AM Ali Gouta wrote:
> Do not know Kenesis, but it looks like it works like kafka. Your producer
> should implement a paritionner that
Do not know Kenesis, but it looks like it works like kafka. Your producer
should implement a paritionner that makes it possible to send your data
with the same key to the same partition. Though, each task in your spark
streaming app will load data from the same partition in the same executor.
I thi
