If you use the StandaloneHaServices, then you have to define the `rest.bind-port` upfront. Hence, picking a random port, won't work.
Cheers, Till On Tue, Aug 6, 2019 at 2:49 AM Zili Chen <wander4...@gmail.com> wrote: > Thanks for your reply. Let me rephrase the question, > is it possible to configure rest.bind-port to "0", > i.e., resolved at the runtime, in standalone mode? > > The problem is that the resolution is after the > construction of standalone ha service which requires > a known bind-port. > > See also(at f75d8e1fbb16ba08ab5a625f50e988c708a8a2bf) > > (port resolved at) > RestServerEndpoint.start:191 (depends on) > AbstractDispatcherResourceManagerComponentFactory.create:162 (requires ha > service) > StandaloneHaService.<init>:70 (requires web monitor address, including the > port) > > Best, > tison. > > > Till Rohrmann <trohrm...@apache.org> 于2019年8月5日周一 下午9:28写道： > > > Not sure whether I fully understood the question but if you are asking > how > > the StandaloneHaServices notify its listeners about a changed REST > endpoint > > address, then this is not possible. The StandaloneHaServices do not > > tolerate failures of the Dispatcher, ResourceManager, JobMaster or > > WebMonitorEndpoint. The contract is that the addresses stored in the > > StandaloneHaServices don't change. Hence, if you want to enable HA for > the > > MiniCluster, then one would have to start a ZooKeeper cluster without the > > EmbeddedLeaderService. > > > > Cheers, > > Till > > > > On Mon, Aug 5, 2019 at 1:52 PM Zili Chen <wander4...@gmail.com> wrote: > > > > > Yes I can understand the idea. > > > > > > One more question, as you mentioned, the EmbeddedLeaderService > > > is not needed. However, from our current codebase, if we configure > > > rest.bind-port as "0", i.e., resolve the port at the runtime, > > > we actually resolve the port on WebMonitorEndpoint#start. > > > But WebMonitorEndpoint#<init> requires HaService and > > > StandaloneHaService#<init> requires pre-configured port. > > > > > > How we fits this scenario into StandaloneHaService instead of > > > EmbeddedLeaderService which publish leader host:port at the > > > runtime? > > > > > > Best, > > > tison. > > > > > > > > > Till Rohrmann <trohrm...@apache.org> 于2019年8月5日周一 下午7:29写道： > > > > > > > The idea of the EmbeddedLeaderService is to give you ZooKeeper like > > high > > > > availability if all components run inside of the same process. That > way > > > it > > > > is also easy to test HA functionality without having to start > > ZooKeeper. > > > > > > > > Technically, the EmbeddedLeaderService is not needed but it is very > > handy > > > > to have. > > > > > > > > Cheers, > > > > Till > > > > > > > > On Mon, Aug 5, 2019 at 10:43 AM Zili Chen <wander4...@gmail.com> > > wrote: > > > > > > > > > Investigated the usage of EmbeddedLeaderService, as > > > > > you said, it is mainly used in MiniCluster. > > > > > > > > > > The difference between EmbeddedLeaderService and > > > > > ZooKeeperLeader(Election|Retrieval)Service is that > > > > > the latter uses component host and port pre-configured > > > > > while the former configured at the runtime. > > > > > > > > > > I'm not sure when proceeding FLINK-10333 whether > > > > > implement such a in memory leader election service > > > > > or let MiniCluster use the standalone one is preferred. > > > > > > > > > > Best, > > > > > tison. > > > > > > > > > > > > > > > Zili Chen <wander4...@gmail.com> 于2019年8月5日周一 下午4:39写道： > > > > > > > > > > > Hi Till, > > > > > > > > > > > > SingleLeaderElectionService is an effectively subclass of > > > > > > EmbeddedLeaderService. The merge work can be done as the > > > > > > minor patch attached. > > > > > > > > > > > > I came into this area when working on FLINK-10333, implementing > > > > > > LeaderElectionService as the document describes. I noticed that > > > > > > besides Standalone and ZooKeeper implementation there is an > > > > > > in memory LeaderElectionService, EmbeddedLeaderService. > > > > > > > > > > > > Before thinking whether it could be replace with the Standalone > > > > > > one, I found SingleLeaderElectionService which is an effectively > > > > > > subclass of it. Thus I started this issue to hear from the > > > > > > community. > > > > > > > > > > > > Best, > > > > > > tison. > > > > > > > > > > > > > > > > > > Till Rohrmann <trohrm...@apache.org> 于2019年8月2日周五 下午5:59写道： > > > > > > > > > > > >> I think at the moment SingleLeaderElectionService is not really > > used > > > > > since > > > > > >> it is part of the YarnIntraNonHaMasterServices. The idea of > these > > ha > > > > > >> services was to offer a different ha implementation which > slightly > > > > > >> different guarantees. Concretely, the > YarnIntraNonHaMasterServices > > > > > handle > > > > > >> operator and TaskManager faults but not master faults. > > > > > >> > > > > > >> The EmbeddedHaServices are only used by the MiniCluster. > > > > > >> > > > > > >> I'm not so sure whether the refactoring is so easy because > > > > > >> EmbeddedLeaderElectionService is an inner class of > > > > EmbeddedLeaderService > > > > > >> and calls methods of this class. Why do you wanna change > anything > > > > there? > > > > > >> > > > > > >> Cheers, > > > > > >> Till > > > > > >> > > > > > >> On Fri, Aug 2, 2019 at 6:33 AM Zili Chen <wander4...@gmail.com> > > > > wrote: > > > > > >> > > > > > >> > Hi devs, > > > > > >> > > > > > > >> > I found that these two classes are quite similar except > > > > > >> > SingleLeaderElectionService has a pre-config leader id. > > > > > >> > > > > > > >> > However, I don't see use points of that leader id. Also > > > > > >> > a random UUID would work as a DEFAULT_LEADER_ID(0). > > > > > >> > I consider whether we could replace > SingleLeaderElectionService > > > > > >> > with EmbeddedLeaderElectionService, or merge. > > > > > >> > > > > > > >> > Best, > > > > > >> > tison. > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
