Thanks a lot for your time Yiming and Enrico. :) Regarding the security, we could do it in a separate BP, and make this BP more focus on filling up the useful endpoints. How about it?
On Wed, Sep 13, 2017 at 3:30 PM, Enrico Olivelli <eolive...@gmail.com> wrote: > Hi Jia, > I am OK with the idea of having standard HTTP API, this will help > development of (non-Java) tools. > > It is not clear to me if we are going to add an http API useful for > "managing bookies" or a new HTTP REST-like Client API to BookKeeper. > I am referring to the fact that in the proposal there are API calls to > create ledgers and to read data. > I think we should separate this two aspects and maybe it is better to > address the 'bookie management' first, which is the work that Yiming > started. > [jia] It is targeting for the admin portal. the existence of the ledger api is just to simplify debugging or operations. we can eliminate the ‘create’ endpoint. > > Another point very important to me is that if we are going to introduce > management operations via http we have to take into account security, at > least TLS and some kind of authentication. > About TLS it is very simple to achieve this, every HTTP server supports TLS > (https) > About authentication the problem is not so simple, Kerberos on HTTPs is > very complex, but we need to introduce some auth mechanism. > IMHO We can require the 'http server implementation' to implement security > but out of the box we have to supply basic support at least for one > provider bundled in the distribution package. > [jia] The BP focuses on filling up the useful endpoints. The security will be a separate BP. > > Some API can return very large result sets like 'list ledgers', actually > the HTTP Server subsystem exchanges strings in memory, we will need to > introduce some more smart way because it would be easy to bring down the > bookie just by calling that API multiple times concurrencly (it is just an > example) > [jia] We could add pagination into all the `list` api. > > IMHO The 'create ledger' API is not useful, due to the design of BK you > have to create a ledger and then write immediately to it, I think that such > an API should allow the client to stream the contents of the ledger in the > HTTP body at least. But I think that a more stateful http API needs to be > designed to implement a pure Http client > [jia] Thanks, we are not planning to implement an http client. we will remove ‘create’ here. > > Maybe we should add a more narrowed motivation and add the only useful APIs > to address those issues. > For instance in my company we would like to start creating a BookKeeper Web > UI, so we need some way to talk to bookies and exchange data, but in this > case I am interested in asking to bookies their status and the effective > contents of the bookie > [jia] The BP proposes adding a standard naming convention for adding admin endpoints. Feel free to propose the endpoints you would like to appear in the http admin portal. > > -- Enrico > > > 2017-09-13 6:09 GMT+02:00 Yiming Zang <yz...@twitter.com.invalid>: > > > Sure, I think the current HTTP endpoints in Twitter are only designed for > > Twitter specific, such as check quorum loss, check rack/region diversity. > > So the endpoints convention in Twitter are not the same as in the > proposal. > > I think it would be great to have an agreement on the API naming design, > so > > I like the API design in the proposal, I think the proposal looks good to > > me. > > > > Besides, we're currently only using GET functionalities in Twitter, but I > > notice there're a lot of POST and PUT APIs in the proposal which could > > change the bookie state or trigger some heavy workload. These APIs looks > a > > bit risky to me if we don't have any authentication enabled (in Twitter). > > > > On Tue, Sep 12, 2017 at 6:11 PM, Sijie Guo <guosi...@gmail.com> wrote: > > > > > + Yiming > > > > > > Yiming, if you have time, please take a look at this BP. let's see if > > > there are any conflicts with those you added for autorecovery. > > > > > > - Sijie > > > > > > On Tue, Sep 12, 2017 at 8:00 AM, Jia Zhai <zhaiji...@gmail.com> wrote: > > > > > >> Hi all, > > >> > > >> Based on Github #278 <https://github.com/apache/bookkeeper/pull/278>, > > I > > >> have just posted a proposal regarding define BookKeeper public http > > >> endpoints: > > >> > > >> https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP- > > >> 17%3A+Define+BookKeeper+public+http+endpoints > > >> > > >> > > >> Github #278 <https://github.com/apache/bookkeeper/pull/278> > introduces > > >> BookKeeper Http Endpoint module. However there are only two endpoints, > > >> which is “/heartbeat” and “/api/config/serverconfig”, defined in #278. > > In > > >> order to fully leverage the http modules, The goal of this BP is to > add > > >> more endpoints to this modules. > > >> > > >> Any comments are welcome and appreciated. > > >> > > >> > > >> Thanks. > > >> > > >> -Jia > > >> > > > > > > > > >
