Re: Data stream to write to multiple rds instances

2017-03-07 Thread Till Rohrmann
Hi Sathi, if you read data from Kinesis than Flink can offer you exactly once processing guarantees. However, what you see written out to your database depends a little bit on the implementation of your custom sink. If you have synchronous JDBC client which does not lose data and you fail your job

Re: Data stream to write to multiple rds instances

2017-03-02 Thread Sathi Chowdhury
Hi Till, Thanks for your reply.I guess I will have to write a custom sink function that will use JdbcOutputFormat. I have a question about checkpointing support though ..if I am reading a stream from kinesis , streamA and it is transformed to streamB, and that is written to db, as streamB is ch

Re: Data stream to write to multiple rds instances

2017-03-02 Thread Till Rohrmann
Hi Sathi, you can split select or filter your data stream based on the field's value. Then you are able to obtain multiple data streams which you can output using a JDBCOutputFormat for each data stream. Be aware, however, that the JDBCOutputFormat does not give you any processing guarantees since