The StreamingFileSink requires that you have checkpointing enabled. I'm
guessing that you don't have checkpointing enabled, since that would
explain the behavior you are seeing.
The relevant section of the docs [1] explains:
Checkpointing needs to be enabled when using the StreamingFileSink. Part
On my kubernetes cluster when I set the StreamingFileSink to write to a
local instance of S3 (MINIO - 500 GB) it only writes the data after I
execute a savepoint
The expected behavior is to write the data in real-time. I'm guessing the
memory requirements have not been met or a configuration in MI
