Hey Chen-Che Huang, I guess the StreamingFileSink is what you are looking for. It is documented here: https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/streamfile_sink.html I drafted a short example (that is not production ready), which does roughly what you are asking for: https://gist.github.com/rmetzger/7d5dbdaa118c63f5875c8c9520cc311d
Hope this helps! Best, Robert On Thu, Apr 15, 2021 at 11:33 AM Chen-Che Huang <acmic...@gmail.com> wrote: > Hi all, > > We're going to use state processor to make our keyedstate data to be > written to different files based on the keys. More specifically, we want > our data to be written to files key1.txt, key2.txt, ..., and keyn.txt where > the value with the same key is stored in the same file. In each file, the > data may be stored as follows. As far as I know, I need to implement my own > Sink (org.apache.flink.streaming.api.functions.sink.RichSinkFunction) to > meet the requirement. However, I wonder is there a native way to achieve > this without implementing my own Sink because using official solution is > usually more efficient and reliable than doing it by myself. Many thanks > for any comment. > > key1.txt > key1 value11 > key1 value21 > key1 value31 > > key2.txt > key2 value21 > key2 value22 > key2 value23 > > Best wishes, > Chen-Che Huang >
