Re: flink-git - an experiment in exactly-once semantics

2016-04-26 Thread Maximilian Michels
Hi Eron, Very interesting idea to support exactly once semantics for sinks via Git! I would be curious about the performance of such a sink. Since this currently works on local file systems only (throws an Exception otherwise), I wonder how does it work on failures when the "git-${subtaskIndex}"

flink-git - an experiment in exactly-once semantics

2016-04-22 Thread Eron Wright
Hello, On a long plane trip I had some fun with writing a Flink streaming connector based on Git. https://github.com/EronWright/flink-git Not intended for real application use; flink-git is just an experiment meant for discussion. Flink's Kafka connector provides exactly-once guarantees when ac