Sure, I would imagine couple of extra lines within flink.conf ... graphstore.type: customized/zookeeper graphstore.class: org . apache.flink.contrib .MyS3SubmittedJobGraphStoreImp graphstore.endpoint: s3.amazonaws.com graphstore.path.root: s3://my root/
which overwrites initiation of *org.apache.flink.runtime.highavailability.HighAvailabilityServices* /** * Gets the submitted job graph store for the job manager * * @return Submitted job graph store * @throws Exception if the submitted job graph store could not be created */ SubmittedJobGraphStore *getSubmittedJobGraphStore*() throws Exception; In this case, user implemented their own s3 backed job graph store and stores job graphs in s3 instead of zookeeper(high availability) or never(nonha) I find [1] is somehow related and more focus on life cycle and dependency aspect of graph-store and checkpoint-store. FLINK-7106 in this case limited to enable user implemented their own jobgraphstore instead of hardcoded to zookeeper. Thanks, Chen [1] https://issues.apache.org/jira/browse/FLINK-6626 On Thu, Jul 6, 2017 at 2:47 AM, Ted Yu <yuzhih...@gmail.com> wrote: > The sample config entries are broken into multiple lines. > > Can you send the config again with one config on one line ? > > Cheers > > On Wed, Jul 5, 2017 at 10:19 PM, Chen Qin <qinnc...@gmail.com> wrote: > > > Hi there, > > > > I would like to propose/discuss median level refactor work to make > > submittedJobGraphStore configurable and extensible. > > > > The rationale behind is to allow users offload those meta data to durable > > cross dc read after write strong consistency storage and decouple with zk > > quorum. > > > > > > https://issues.apache.org/jira/browse/FLINK-7106 > > > > <https://issues.apache.org/jira/browse/FLINK-7106> > > New configurable setting in flink.conf > > looks like following > > > > g > > raph > > -s > > tore: > > customized/zookeeper > > g > > raph > > -s > > tore.class: xx.yy.MyS3SubmittedJobGraphStoreImp > > > > g > > raph > > -s > > tore. > > endpoint > > : s3.amazonaws.com > > g > > raph > > -s > > tore.path.root: > > s3:/ > > > > / > > my root/ > > > > Thanks, > > Chen > > >
