Hi,
I am trying to upload avro records to AWS S3 using StreamingFileSink.
Avro file is getting created and uploaded with valid data but I want to add a
Rolling policy which will roll the file after specific time or total part file
size.
With forBulkFormat() I am able to use only CheckpointRolli
Hello,
I am trying to use StreamingFileSink.forBulkFormat() for writing avro to S3.
I have used ‘CheckpointRollingPolicy’ as DefaultRolling cannot be used with
bulk formats.
But when I use this I am facing 2 issues :
1. ‘shouldRollOnEvent’ method is getting called on each record addition but