Is it possible that i use checkpoint directory to restart streaming but with modified parameter value in config file (e.g. username/password for db connection) ? Thanks in advance.
Regards, Chandan On Thu, Aug 18, 2016 at 1:10 PM, chandan prakash <chandanbaran...@gmail.com> wrote: > Hi, > I am using direct kafka with checkpointing of offsets same as : > https://github.com/koeninger/kafka-exactly-once/blob/ > master/src/main/scala/example/IdempotentExample.scala > > I need to change some parameters like db connection params : > username/password for db connection . > I stopped streaming gracefully ,changed parameters in config file and > restarted streaming. > *Issue : changed parameters username/password are not being considered.* > > *Question* : > As per my understanding , Checkpointing should only save offsets of kafka > partitions and not the credentials of the db connection. > Why its picking old db connection params ? > > I am declaring params in main method and not in setUpSsc(0 method. > My code is identical to that in the above program link as below: > val jdbcDriver = conf.getString("jdbc.driver") > val jdbcUrl = conf.getString("jdbc.url") > *val jdbcUser = conf.getString("jdbc.user")* > * val jdbcPassword = conf.getString("jdbc.password")* > // while the job doesn't strictly need checkpointing, > // we'll checkpoint to avoid replaying the whole kafka log in case of > failure > val checkpointDir = conf.getString("checkpointDir") > val ssc = StreamingContext.getOrCreate( > checkpointDir, > setupSsc(topics, kafkaParams, jdbcDriver, jdbcUrl, *jdbcUser*, > *jdbcPassword*, checkpointDir) _ > ) > > > > -- > Chandan Prakash > > -- Chandan Prakash
