If your column on which you want to perform the split is numeric you can use the NumericBetweenParametersProvider interface that automatically computes the splits for you. This is an example of its usage (at windows of 1000 items at a time) taken from the test class *JDBCInputFormatTest*:
final int *fetchSize* = 1000; final Long *min* = 0L; final Long *max* = 1_000_000L; ParameterValuesProvider pramProvider = new *NumericBetweenParametersProvider*(fetchSize, min, max); jdbcInputFormat = JDBCInputFormat.buildJDBCInputFormat() .setDrivername(DRIVER_CLASS) .setDBUrl(DB_URL) .setQuery(JDBCTestBase.SELECT_ALL_BOOKS_SPLIT_BY_ID) .setRowTypeInfo(rowTypeInfo) .setParametersProvider(pramProvider) .setResultSetType(ResultSet.TYPE_SCROLL_INSENSITIVE) .finish(); I hope this could help, Flavio On Tue, Jan 24, 2017 at 10:57 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi, > > JdbcInputFormat implements the InputFormat interface and is handled exactly > like any other InputFormat. > In contrast to file-based input formats, users must explicitly specify the > input splits by providing an array of parameter values which are injected > into a parameterized query. > This is done because it is not easy to implement a generic method > automatically split a query into multiple (preferably equal-sized) partial > queries. > > Best, Fabian > > 2017-01-24 6:31 GMT+01:00 Pawan Manishka Gunarathna < > pawan.manis...@gmail.com>: > > > Hi, > > Thanks for your help. Since Our data source has database tables > > architecture I have a thought of follow that 'JDBCInputFormat' in Flink. > It > > would be great if you can provide some information regarding how that > > JDBCInputFormat execution happens? > > > > Thanks, > > Pawan > > > > On Mon, Jan 23, 2017 at 4:18 PM, Fabian Hueske <fhue...@gmail.com> > wrote: > > > > > Hi Pawan, > > > > > > I don't this this works. The InputSplits are generated by the > JobManager, > > > i.e., not in parallel by a single process. > > > After the parallel InputFormats have been started on the TaskManagers, > > they > > > request InputSplits and open() them. If there are no InputSplits there > is > > > no work to be done and open will not be called. > > > You can tweak the behavior by implementing your own InputSplits and > > > InputSplitAssigner which assigns exactly one input split to each task. > > > > > > Fabian > > > > > > 2017-01-23 8:44 GMT+01:00 Pawan Manishka Gunarathna < > > > pawan.manis...@gmail.com>: > > > > > > > Hi, > > > > > > > > When we are implementing that Flink *InputFormat* Interface, if we > have > > > > that* > > > > input split creation* part in our data analytics server APIs can we > > > > directly go to the second phase of the flink InputFormat Interface > > > > execution. > > > > > > > > Basically I need to know that can we read those InputSplits directly, > > > > without generating InputSplits inside the InputFormat Interface. So > it > > > > would be great if you can provide any kind of help. > > > > > > > > Thanks, > > > > Pawan > > > > > > > > -- > > > > > > > > *Pawan Gunaratne* > > > > *Mob: +94 770373556* > > > > > > > > > > > > > > > -- > > > > *Pawan Gunaratne* > > *Mob: +94 770373556* > > >
