This is a draft for the BP-38 https://github.com/apache/bookkeeper/pull/2214/files
and this is a draft implementation https://github.com/apache/bookkeeper/pull/2213/files I will ping again when the documents are ready for full review. I have just pushed them in case anyone as time to give early feedback Regards Enrico Il giorno dom 8 dic 2019 alle ore 16:10 Enrico Olivelli <eolive...@gmail.com> ha scritto: > I am preparing a BP with the goal of enabling discovery of every service > exposed by a Bookie using existing metadata service. > > Stay tuned > > Enrico > > Il mar 3 dic 2019, 08:55 Enrico Olivelli <eolive...@gmail.com> ha scritto: > >> Sijie, >> answers inline >> >> Enrico >> >> Il giorno mar 3 dic 2019 alle ore 08:15 Sijie Guo <guosi...@gmail.com> >> ha scritto: >> >>> I am not sure it is a good idea to expose the stats via a RPC endpoint. >>> >>> Because most of the stats collection systems (e.g. prometheus) are >>> collecting stats via an http endpoints. >>> I don't think we should duplicate the work that a stats library is good >>> at. >>> >>> Other thoughts inline. >>> >>> On Sat, Nov 30, 2019 at 4:31 AM Enrico Olivelli <eolive...@gmail.com> >>> wrote: >>> >>> > Hi all, >>> > In my company we are developing a visual tool for Bookkeeper >>> Management and >>> > we would like to have more information about the status of a Bookie >>> just by >>> > using RPC calls. >>> > We already have getBookieInfo that is only reporting about disk space >>> > usage. >>> > >>> > Examples of useful info: >>> > - link to metrics endpoint >>> > - last minor/major gc execution timestamp >>> > - last jvm start timestamp (uptime) >>> > - configuration file >>> > >>> > We already have the http endpoint but it comes with several >>> disadvantages: >>> > - it is a secondary channel (so you have to use two channels to talk >>> to the >>> > bookie) >>> > >>> >>> Why not standardize the management endpoint in http endpoint and build >>> the >>> dashboard on it? >>> >> >> Actually our project is a Web Application (WAR archive) so we can deploy >> it to any Servlet Container (Jetty, Tomcat), so in theory it could be >> deployed inside the bookie itself or at least distributed with BK binary >> distribution >> Once we have a stable release one good plan could be to contribute it to >> ASF BookKeeper Project (as StreamNative did with Pulsar Manager) >> >> https://github.com/diennea/bookkeeper-visual-manager >> >> >> >>> It is a common approach for doing so. An RPC channel is more used for >>> data >>> channel. >>> >>> >>> > - there is no way of getting the http address, as it is not advertised >>> on >>> > metadata discovery service >>> > >>> >>> We should fix this, no? >>> >> >> Yes we could, but actually it is more invasive change. >> I think such kind of enhancements should be done together with the change >> about using some bookieid that is is not host:port. >> >> >> >>> >>> >>> > - it does not support security (auth/tls..) yet >>> > >>> >>> We should add TLS support and authentication to the http endpoint. >>> >> >> Yes we could. >> Some problems: >> - (community) we have multiple implementations of the HTTP endpoint, we >> should decide whether to enhance only one, all of the them... >> - (private) In all of the clusters of my customers we are not using the >> http endpoint provided by the bookie, so actually >> having a new open endpoint will come with some disadvantages (new open >> port, security, network architecture change....) >> - The Bookie http server is not "read only", it can force the bookie to >> do things, we should also change this >> >> >> >> >>> >>> I guess rather than choosing to implement duplicated work in a PRC >>> endpoint, why can't we spend some time enhancing http endpoint? >>> >>> >>> > >>> > Given that I am mostly using Kerberos, >>> > going to the http way will need separate security configurations: for >>> > bookie RPC and for HTTP. >>> > >>> >>> Jia implemented Kerberos for Pulsar http endpoint. Also HDFS supports >>> kerberos at its http endpoint as well. >>> So I don't think that is a problem as well. >>> >> >> That would be interesting. >> >> >>> >>> >>> > >>> > So that's why I would prefer to use the standard binary RPC protocol: >>> only >>> > one client, only one auth, security is already ok, no additional >>> discovery >>> > info. >>> > >>> > If you guys agree on the approach we can send a patch that includes the >>> > support for publishing more information on that RPC >>> > >>> > Cheers >>> > Enrico >>> > >>> >>
