You need to insert per partition per batch. Normally database drivers meant
for spark have bulk update feature built in. They take a RDD and do a bulk
insert per partition.
In case db driver you are using doesn't provide this feature, you can
aggregate records per partition and then send out to db
by writing code, I suppose :) Jokes apart, I think you need to articulate
the problem with more details for others to help.
Do you mean you want to batch up data in memory and then write as a chunk?
Where do want to insert? Etc etc...
On Fri, Apr 20, 2018 at 1:08 PM, amit kumar singh
wrote:
> H
