Hi Pawan, this sounds like you need to implement a custom InputFormat [1]. An InputFormat is basically executed in two phases. In the first phase it generates InputSplits. An InputSplit references a a chunk of data that needs to be read. Hence, InputSplits define how the input data is split to be read in parallel. In the second phase, multiple InputFormats are started and request InputSplits from an InputSplitProvider. Each instance of the InputFormat processes one InputSplit at a time.
It is hard to give general advice on implementing InputFormats because this very much depends on the data source and data format to read from. I'd suggest to have a look at other InputFormats. Best, Fabian [1] https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java 2017-01-16 6:18 GMT+01:00 Pawan Manishka Gunarathna < pawan.manis...@gmail.com>: > Hi, > > we have a data analytics server that has analytics data tables. So I need > to write a custom *Java* implementation for read data from that data source > and do processing (*batch* processing) using Apache Flink. Basically it's > like a new client connector for Flink. > > So It would be great if you can provide a guidance for my requirement. > > Thanks, > Pawan >
