Re: Dynamic JDBC Sink Support

2021-03-06 Thread Maciej Obuchowski
Hey Rion, I had exactly the same problem and implemented this functionality in my Flink fork with XA sink taken from the development branch. As I see that it's not only my problem, I've created a Jira task for it - FLINK-21643 - and will provide draft PR for it. @David - for traditional relationa

Re: Dynamic JDBC Sink Support

2021-03-05 Thread David Anderson
Rion, A given JdbcSink can only write to one table, but if the number of tables involved isn't unreasonable, you could use a separate sink for each table, and use side outputs [1] from a process function to steer each record to the appropriate sink. I suggest you avoid trying to implement a sink.

Dynamic JDBC Sink Support

2021-03-05 Thread Rion Williams
Hi all, I’ve been playing around with a proof-of-concept application with Flink to assist a colleague of mine. The application is fairly simple (take in a single input and identify various attributes about it) with the goal of outputting those to separate tables in Postgres: object AttributeIdent