Thanks. Idea was to query for 'x' records in last 'n' seconds using an indexed column. Looks like that is not possible?
On Tue, Sep 26, 2017 at 3:24 PM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Mohit, > > no, a JdbcTableSource does not exist yet. However, since there is a > JdbcInputFormat it should not be hard to wrap that in a TableSource. > However, this would rather be a batch TableSource in the sense that it > would just return the data that the query returns. Once all data is read it > would terminate. You can of course wrap the JdbcInputFormat in a > StreamingTableSource, but as I said it would terminate when all data was > read. > > If you are thinking of streaming a changelog stream from a database to the > Table API / SQL, this would not be possible at the moment due to limitation > in the Table API / SQL (these will be removed in the future). > Moreover, not many DBMS expose their changelog (such as PostgreSQL) and > there is no common interface for that such as JDBC. Instead they use custom > formats. There is a tool called Bottled Water that ingests PostgreSQL > streams into Kafka. > > So, to make a long story short: implementing a JDBC TableSource for batch > query should be fairly easy. A true streaming solution that hooks into the > changelog stream of a table is not possible at the moment. > > Cheers, Fabian > > 2017-09-26 15:04 GMT-04:00 Mohit Anchlia <mohitanch...@gmail.com>: > >> We are looking to stream data from the database. Is there already a jdbc >> table source available for streaming? >> > >
