Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-18 Thread Sijie Guo
Thank you JV. we will try to come up with a BP covering the overall picture on the metadata enhancements we are thinking. we can then discuss and collaborate. On Sat, Sep 16, 2017 at 3:52 PM, Venkateswara Rao Jujjuri wrote: > If the real use of this work item is to use: > - Only one bookie > -

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-16 Thread Venkateswara Rao Jujjuri
If the real use of this work item is to use: - Only one bookie - Bookie is the metadata server too Maybe it is ok. But I am worried about the *magnitude* of the code refactor to accommodate this. I really like the idea of finding an alternate to ZK, but I think we need to have that direction iron

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-16 Thread Enrico Olivelli
Yep Thank you Jia Enrico On sab 16 set 2017, 12:10 Jia Zhai wrote: > Since there is not objective. would like to make this BP approved. > > On Wed, Sep 13, 2017 at 4:24 PM, Sijie Guo wrote: > > > On Wed, Sep 13, 2017 at 1:18 AM, Enrico Olivelli > > wrote: > > > > > 2017-09-13 10:10 GMT+02:00

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-16 Thread Jia Zhai
Since there is not objective. would like to make this BP approved. On Wed, Sep 13, 2017 at 4:24 PM, Sijie Guo wrote: > On Wed, Sep 13, 2017 at 1:18 AM, Enrico Olivelli > wrote: > > > 2017-09-13 10:10 GMT+02:00 Sijie Guo : > > > > > On Wed, Sep 13, 2017 at 12:16 AM, Enrico Olivelli > > > > wrot

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-13 Thread Sijie Guo
On Wed, Sep 13, 2017 at 1:18 AM, Enrico Olivelli wrote: > 2017-09-13 10:10 GMT+02:00 Sijie Guo : > > > On Wed, Sep 13, 2017 at 12:16 AM, Enrico Olivelli > > wrote: > > > > > I think that this is a good direction to go. > > > > > > I believe to the reasons about ZK in huge systems even it is not

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-13 Thread Enrico Olivelli
2017-09-13 10:10 GMT+02:00 Sijie Guo : > On Wed, Sep 13, 2017 at 12:16 AM, Enrico Olivelli > wrote: > > > I think that this is a good direction to go. > > > > I believe to the reasons about ZK in huge systems even it is not my case > so > > I cannot add comments on this usecase. > > > > I am fine

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-13 Thread Sijie Guo
On Wed, Sep 13, 2017 at 12:16 AM, Enrico Olivelli wrote: > I think that this is a good direction to go. > > I believe to the reasons about ZK in huge systems even it is not my case so > I cannot add comments on this usecase. > > I am fine with direction as long as we are still going to support >

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-13 Thread Enrico Olivelli
I think that this is a good direction to go. I believe to the reasons about ZK in huge systems even it is not my case so I cannot add comments on this usecase. I am fine with direction as long as we are still going to support ZooKeeper. BookKeeper is in the Hadoop / ZooKeeper ecosystem and severa

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-12 Thread Jia Zhai
Any thoughts or comments :) Thanks a lot. -Jia On Tue, Sep 12, 2017 at 4:30 PM, Jia Zhai wrote: > This blog: https://bitworks.software/blog/en/2017-07-12-replicated- > scalable-commitlog-with-apachebookkeeper.html, which also refer a little > the limitation of zookeeper in bookkeeper > > On Thu

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-12 Thread Jia Zhai
This blog: https://bitworks.software/blog/en/2017-07-12-replicated-scalable-commitlog-with-apachebookkeeper.html, which also refer a little the limitation of zookeeper in bookkeeper On Thu, Sep 7, 2017 at 9:45 AM, Jia Zhai wrote: > 👍. Thanks a lot for the suggestions and feed back. > > On Thu, S

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Jia Zhai
👍. Thanks a lot for the suggestions and feed back. On Thu, Sep 7, 2017 at 4:24 AM, Sijie Guo wrote: > On Wed, Sep 6, 2017 at 1:07 PM, Enrico Olivelli > wrote: > > > Off topic curiosity... Jia and Sijie, do you think we are going to drop > ZK > > from DL too? > > > > Yes. That's the goal - 1) fo

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Sijie Guo
On Wed, Sep 6, 2017 at 1:07 PM, Enrico Olivelli wrote: > Off topic curiosity... Jia and Sijie, do you think we are going to drop ZK > from DL too? > Yes. That's the goal - 1) for large deployment, we are trying to overcome the limitation of zookeeper; 2) for smaller deployments, it will make dep

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Enrico Olivelli
Off topic curiosity... Jia and Sijie, do you think we are going to drop ZK from DL too? Enrico On mer 6 set 2017, 19:51 Enrico Olivelli wrote: > > > On mer 6 set 2017, 18:25 Sijie Guo wrote: > >> On Sep 6, 2017 4:57 AM, "Enrico Olivelli" wrote: >> >> Thank you Sijie and Jia for your comments a

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Enrico Olivelli
On mer 6 set 2017, 18:25 Sijie Guo wrote: > On Sep 6, 2017 4:57 AM, "Enrico Olivelli" wrote: > > Thank you Sijie and Jia for your comments and explanations, > answers inline > > 2017-09-06 2:23 GMT+02:00 Jia Zhai : > > > Thanks a lot Enrico and Sijie for your comments and information on this. >

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Sijie Guo
On Sep 6, 2017 4:57 AM, "Enrico Olivelli" wrote: Thank you Sijie and Jia for your comments and explanations, answers inline 2017-09-06 2:23 GMT+02:00 Jia Zhai : > Thanks a lot Enrico and Sijie for your comments and information on this. > > On Tue, Sep 5, 2017 at 9:31 PM, Enrico Olivelli > wrot

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-06 Thread Enrico Olivelli
Thank you Sijie and Jia for your comments and explanations, answers inline 2017-09-06 2:23 GMT+02:00 Jia Zhai : > Thanks a lot Enrico and Sijie for your comments and information on this. > > On Tue, Sep 5, 2017 at 9:31 PM, Enrico Olivelli > wrote: > > > Great to see you working on this ! > > I w

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-05 Thread Jia Zhai
Thanks a lot Enrico and Sijie for your comments and information on this. On Tue, Sep 5, 2017 at 9:31 PM, Enrico Olivelli wrote: > Great to see you working on this ! > I would be great to have such feature, as it is the first step to a > 'standalone' BookKeeper mode > > Some complementary ideas/f

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-05 Thread Enrico Olivelli
On mar 5 set 2017, 21:28 Sijie Guo wrote: > Enrico, > > Thank you for your feedback. > > Just FYI - this BP is the first part of the work that we've been working on > improving metadata management on BookKeeper. We are doing this in three > parts: > > - thin client : avoid talking to metadata sto

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-05 Thread Sijie Guo
Enrico, Thank you for your feedback. Just FYI - this BP is the first part of the work that we've been working on improving metadata management on BookKeeper. We are doing this in three parts: - thin client : avoid talking to metadata store directly in clients, moving the metadata management to t

Re: [DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-05 Thread Enrico Olivelli
Great to see you working on this ! I would be great to have such feature, as it is the first step to a 'standalone' BookKeeper mode Some complementary ideas/first look questions: - the document does not talk about security, IMHO we have at least to cover authentication and TLS, it would be great t

[DISCUSS] BP-16: remove zookeeper dependency from bookkeeper client

2017-09-05 Thread Jia Zhai
Hi all, I have just posted a proposal to remove zookeeper dependency from bookkeeper client, to make bookkeeper client a thin client: https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-16%3A+remove+zookeeper+dependency+from+bookkeeper+client BookKeeper uses zookeeper for service discover