Im using jdts driver to query mssql . I used the ParametersProvider as you suggested but for some reason the job wont run parallel .
[image: flink_in.JPG] Also the sink , a simple print out wont parallel [image: flink_out.JPG] On Tue, Dec 4, 2018 at 10:05 PM Flavio Pompermaier <pomperma...@okkam.it> wrote: > You can pass a ParametersProvider to the jdbc input format in order to > parallelize the fetch. > Of course you don't have to kill the mysql server with too many request > in parallel so you'll probably put a limit to the parallelism of the input > format. > > > On Tue, 4 Dec 2018, 17:31 miki haiat <miko5...@gmail.com wrote: > >> HI , >> I want to query some sql table that contains ~80m rows. >> >> There is a few ways to do that and i wonder what is the best way to do >> that . >> >> >> 1. Using JDBCINPUTFORMAT -> convert to dataset and output it without >> doing any logic in the dataset, passing the full query in the >> JDBCINPUTFORMAT set query parameters. >> 2. Using JDBCINPUTFORMAT select all the data from table then >> desirelaze it ->convert to dataset and preforming logic. >> >> >> Or something else that is much efficient ? >> >> Thanks, >> >> Miki >> >>
