Markus Holzemer created FLINK-1846:
--------------------------------------
Summary: Sinks inside of iterations
Key: FLINK-1846
URL: https://issues.apache.org/jira/browse/FLINK-1846
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime, Local Runtime, Optimizer
Reporter: Markus Holzemer
Assignee: Markus Holzemer
Priority: Minor
I startet work on adding support for file sinks inside of iterations to flink.
My current plan includes two different modes: one where only the output file of
the last iteration is kept and one where the output file of every single
iteration is kept (the corresponding superstep number is added as a prefix).
I am also thinking about adding a mode where the output of every iteration is
appended to one file or where a "sliding window" of n files is kept for all
iterations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
