Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-18 Thread Ismael Juma
Hi Flavio, Comments below. On Sat, Jun 18, 2016 at 4:50 PM, Flavio Junqueira wrote: > For acls, you can simply re-run the acl command to re-introduce them, > unless you assume that no record of acls is maintained once they are > introduced. If that's the case, then another way is to simply read

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-18 Thread Martin Serrano
Why are you seeking to undo it? On 06/17/2016 09:34 PM, Andrew Purtell wrote: > HBase stores replication peering configuration in ZK. We're working on > undoing that, but for now that information exists nowhere else. > > > On Thu, Jun 16, 2016 at 2:47 PM, Ismael Juma wrote: > >> Hi Jordan, >> >>

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-18 Thread Flavio Junqueira
For acls, you can simply re-run the acl command to re-introduce them, unless you assume that no record of acls is maintained once they are introduced. If that's the case, then another way is to simply read periodically the zk state and keep that information somewhere else to be extra safe. This

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-17 Thread Andrew Purtell
See HBASE-14379. The points on configuration, state management, and security apply. > On Jun 17, 2016, at 7:25 PM, Martin Serrano wrote: > > Why are you seeking to undo it? > >> On 06/17/2016 09:34 PM, Andrew Purtell wrote: >> HBase stores replication peering configuration in ZK. We're worki

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-17 Thread Andrew Purtell
HBase stores replication peering configuration in ZK. We're working on undoing that, but for now that information exists nowhere else. On Thu, Jun 16, 2016 at 2:47 PM, Ismael Juma wrote: > Hi Jordan, > > Kafka stores ACLs as well as client and topic configs in ZooKeeper so that > lends credence

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-16 Thread Ismael Juma
Hi Jordan, Kafka stores ACLs as well as client and topic configs in ZooKeeper so that lends credence to your argument, I think. Ismael On Thu, Jun 16, 2016 at 11:41 PM, Jordan Zimmerman < jor...@jordanzimmerman.com> wrote: > Contrary to recommendations everywhere, my experience is that almost >

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-16 Thread Jordan Zimmerman
Contrary to recommendations everywhere, my experience is that almost everyone is storing source of truth data in ZooKeeper. It’s just too tempting. You have a distributed file system just sitting there and it’s too easy to use. You get a lot of great features like watches, etc. People are using

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-16 Thread Chris Nauroth
Yes, thank you to Jordan for the article! Like Flavio, I personally have never come across the requirement for ZooKeeper backups. I've generally followed the pattern that data stored in ZooKeeper is truly transient, and applications are built either to tolerate loss of that data or reconstruct it

Re: FYI - Apache ZooKeeper Backup, a Treatise

2016-06-16 Thread Flavio Junqueira
Great write-up, Jordan, thanks! Whether to backup zk data or not is possibly an open topic for this community, even though we have discussed it at times. My sense has been that precisely because of the issues you mention in your post, it is typically best to have a way to recreate its data upon