Hi Fu, That'll be nice.
Thanks. Best, Shimin Dian Fu <dian0511...@gmail.com> 于2019年9月6日周五 下午3:17写道： > Hi Shimin, > > It can be guaranteed to be an atomic operation. This is ensured by the RPC > framework. You could take a look at RpcEndpoint for more details. > > Regards, > Dian > > > 在 2019年9月6日，下午2:35，shimin yang <ysmcl...@gmail.com> 写道： > > > > Hi Fu, > > > > Thank you for the remind. I think it would work in my case as long as > it's > > an atomic operation. > > > > Dian Fu <dian0511...@gmail.com> 于2019年9月6日周五 下午2:22写道： > > > >> Hi Jingsong, > >> > >> Thanks for bring up this discussion. You can try to look at the > >> GlobalAggregateManager to see if it can meet your requirements. It can > be > >> got via StreamingRuntimeContext#getGlobalAggregateManager(). > >> > >> Regards, > >> Dian > >> > >>> 在 2019年9月6日，下午1:39，shimin yang <ysmcl...@gmail.com> 写道： > >>> > >>> Hi Jingsong, > >>> > >>> Big fan of this idea. We faced the same problem and resolved by adding > a > >>> distributed lock. It would be nice to have this feature in JobMaster, > >> which > >>> can replace the lock. > >>> > >>> Best, > >>> Shimin > >>> > >>> JingsongLee <lzljs3620...@aliyun.com.invalid> 于2019年9月6日周五 下午12:20写道： > >>> > >>>> Hi devs: > >>>> > >>>> I try to implement streaming file sink for table[1] like > >> StreamingFileSink. > >>>> If the underlying is a HiveFormat, or a format that updates visibility > >>>> through a metaStore, I have to update the metaStore in the > >>>> notifyCheckpointComplete, but this operation occurs on the task side, > >>>> which will lead to distributed access to the metaStore, which will > >>>> lead to bottleneck. > >>>> > >>>> So I'm curious if we can support notifyOnMaster for > >>>> notifyCheckpointComplete like FinalizeOnMaster. > >>>> > >>>> What do you think? > >>>> > >>>> [1] > >>>> > >> > https://docs.google.com/document/d/15R3vZ1R_pAHcvJkRx_CWleXgl08WL3k_ZpnWSdzP7GY/edit?usp=sharing > >>>> > >>>> Best, > >>>> Jingsong Lee > >> > >> > >
