Imagine a small team with very limited budget and operational capacity that wants to use Kafka, and needs coordination service for other things too (e.g. service discovery). It's limiting choices too hard to just ZooKeeper if Kafka supports ZooKeeper only. It would be better if Kafka's preferred choice was not the only choice. Maybe using abstraction, Kafka own or through 3rd party like https://github.com/spring-cloud/spring-cloud-cluster would give Kafka users more options.
Kind regards, Stevo Slavic. On Fri, Dec 4, 2015 at 3:16 AM, Todd Palino <tpal...@gmail.com> wrote: > Come on, Jay. Anyone can get up in the morning and run if they have the > willpower :) > > Granted I do have some bias here, since we have tooling in place that makes > deployments and monitoring easier. But even at that, I would not say > Zookeeper is difficult to run or monitor. I’m not denying that there are > complaints, but my experience has always been that complaints of that type > are either related more to the specific way the dependency is implemented > (that is, Kafka’s not using it correctly or is otherwise generating errors > that say “Zookeeper” in them), or it’s related to a bias against the > dependency (I don’t like Zookeeper, I have XYZ installed, etc.). > > The point that for a small installation Zookeeper can represent a large > footprint is well made. I wonder how many of these people are being > ill-served by recommendations from people like me that you should not run > Kafka and Zookeeper on the same systems. Sure, we’d never do that at > LinkedIn just because we are looking for high performance and a few more > systems isn’t a big deal. But for a lower performance environment, it’s > really not a problem to colocate the applications. > > As far as the controller goes, I’m perfectly willing to accept that my > desire to get rid of it is from my bias against it because of how many > problems we’ve run into with that code. We can probably both agree that the > controller code needs an overhaul regardless. It’s stood up well, but as > the clusters get larger it’s definitely shows cracks. > > -Todd > > > On Thu, Dec 3, 2015 at 11:37 AM, Jay Kreps <j...@confluent.io> wrote: > > > Hey Todd, > > > > I actually agree on both counts. > > > > I would summarize the first comment as "Zookeeper is not hard to > > operationalize if you are Todd Palino"--also in that category of > > things that are not hard: running 13 miles at 5:00 am. Basically I > > totally agree that ZK is now a solved problem at LinkedIn. :-) > > > > Empirically, though, it is really hard for a lot of our users. It is > > one of the largest sources of problems we see in people's clusters. We > > could perhaps get part of the way by improving our zk usage and > > documentation, and it is certainly the case that we could potentially > > make things worse in trying to make them better, but I don't think > > that is the same as saying there isn't a problem. > > > > I totally agree with your second comment. In some sense what I was > > sketching out is just replacing ZK. But part of the design of Kafka > > was because we already had ZK. So there might be a way to further > > rationalize the metadata log and the data logs if you kind of went > > back to first principles and thought about it. I don't have any idea > > how, but I share that intuition. > > > > I do think having the controller, though, is quite useful. I think > > this pattern of avoiding many rounds of consensus by just doing one > > round to pick a leader is a good one. If you think about it Paxos => > > Multi-paxos is basically optimizing by lumping together consensus > > rounds on a per message basis into a leader which then handles many > > messages, and what Kafka does is kind of like Multi-multi-paxos in > > that it lumps together many leadership elections into one central > > controller election which then picks all the leaders. In some ways > > having central decision makers seems inelegant (aren't we supposed to > > be distributed?) but it does allow you to be both very very fast in > > making lots of decisions (vs doing thousands of independent leadership > > elections) and also to do things that require global knowledge (like > > balancing leadership). > > > > Cheers, > > > > -Jay > > > > > > > > On Thu, Dec 3, 2015 at 10:05 AM, Todd Palino <tpal...@gmail.com> wrote: > > > This kind of discussion always puts me in mind of stories that start > “So > > I > > > wrote my own encryption. How hard can it be?” :) > > > > > > Joking aside, I do have a few thoughts on this. First I have to echo > > Joel’s > > > perspective on Zookeeper. Honestly, it is one of the few applications > we > > > can forget about, so I have a hard time understanding pain around > running > > > it. You set it up, and unless you have a hardware failure to deal with, > > > that’s it. Yes, there are ways to abusively use it, just like with any > > > application, but Kafka is definitely not one of those use cases. I also > > > disagree that it’s hard to share the ZK cluster safely. We do it all > the > > > time. We share most often with other Kafka clusters, but we also share > > with > > > other applications. This goes back to the “abusive use pattern”. Yes, > we > > > don’t share with them. Nobody should. Yes, it requires monitoring and > you > > > have to configure it correctly. But configuration is a one-time charge > > > (both in terms of capital expenses and knowledge acquisition) and as > > > applications goes, is minimal. > > > > > > So that said, I also don’t think this is necessarily a bad idea, but > this > > > is not the approach I would take. You’re picking off a low-level piece > > and > > > talking about reimplementing primitives. I think it is much better to > > start > > > at the top and have a discussion about where we want to end up as an > > entire > > > application at 1.0. For example, a controller-free cluster where you > just > > > add brokers with no “special node” that handles all the coordination. > > > Brokers join and leave the cluster as more of a mesh, where there is no > > > specific choke point for control. They negotiate with each other for > > > managing partitions, and new partitions are assigned out. We already > > have a > > > concept of partition leadership, which means you don’t actually need to > > > coordinate things that apply only to that partition (configs, writes, > > > deletes). > > > > > > Of course, this is just one start of a discussion on it. There are a > lot > > of > > > ways you can go with overall design, but I think that trying to swap > out > > > the consensus system is not the right place to start it. Once you get a > > > vision of where you want to be, it is much easier to identify what > pieces > > > require what levels of coordination between components. Then you can > > decide > > > on what the right way to achieve that coordination is. Maybe the answer > > > does end up being to implement internal consensus without Zookeeper or > > > another dependency. But as has been said already, consensus systems are > > > hard, and the corner cases can be a nightmare. > > > > > > While I’m generally a fan of plugins in the right places (for example, > > > filtering messages produced or consumed on the clients), I think here > it > > > would lead to more misuse and anti-patterns. I also agree with Jay that > > we > > > should have one thing that is well-managed and rigorously tested when > it > > > comes to consensus. > > > > > > -Todd > > > > > > On Thu, Dec 3, 2015 at 7:18 AM, Joel Koshy <jjkosh...@gmail.com> > wrote: > > > > > >> I’m on an extended vacation but "a native consensus implementation in > > >> Kafka" is one of my favorite topics so I’m compelled to chime in :) I > > have > > >> actually been thinking along the same lines that Jay is saying after > > >> reading the Raft paper - a couple of us at LinkedIn have had casual > > >> discussions about this (i.e., a Raft-like implementation embedded in > > Kafka) > > >> but at this point I’m not sure it is at the point of being a practical > > or > > >> necessary effort for the project per-se: I have heard from a few > people > > >> that operating ZooKeeper is difficult but haven’t quite understood why > > this > > >> is so. It does need some care in selecting hardware and configuration > > but > > >> after that it should just work right? I really think the major > concerns > > are > > >> caused by incorrect usage in the application (i.e., Kafka) which > > results in > > >> weird Kafka cluster behavior and then charging zookeeper for it. > > >> Furthermore, Raft-like implementations sound doable but I did have the > > >> concern that it will be very difficult in practice to perfect the > > >> implementation and Flavio is assuring us that it is indeed difficult > :) > > >> > > >> Although it is unclear to me if it is worthwhile doing a native > > consensus > > >> implementation in Kafka and although I’m willing to take Flavio’s word > > for > > >> it given his experience with ZK and although I'm generally a proponent > > >> of McIlroy's > > >> views in such decisions > > >> < > > >> > > > https://en.wikipedia.org/wiki/Unix_philosophy#Doug_McIlroy_on_Unix_programming > > >> >, > > >> I do think it is a super cool project to try out anyway given that the > > >> commit log is Kafka’s core abstraction and I think the network > > layer/other > > >> layers are really solid. It may just turn out to be really really > > elegant > > >> and will be a “nice-have” sort of win. i.e., I think it will be nice > to > > not > > >> have to run one more service for Kafka and neat to have a native > > consensus > > >> implementation that we understand and know really well. At this point > > I’m > > >> not sure it will be a very big win though since I don’t quite see the > > >> issues in running ZooKeeper alongside Kafka and it will mean bringing > > in a > > >> fair deal of new complexity to Kafka itself. > > >> > > >> On the topic of wrapper libraries: from a developer perspective it is > > easy > > >> to use ZooKeeper incorrectly and I’m guessing there are more than a > few > > >> anti-patterns in our code especially since we use a wrapper library. > So > > I > > >> agree with Neha’s viewpoint that using wrappers such as > zkclient/curator > > >> without being well-versed in the wrapper code itself is another cause > > for > > >> subtle bugs in our usage. Removing that wrapper is an interesting > option > > >> but would again be a fairly significant effort - this is in effect > > going to > > >> be like reimplementing our own wrapper library around ZK. Another > > option is > > >> to just have a few committers/contributors to be the “experts/owners” > of > > >> our zkclient/zookeeper usage. E.g., we have contributed a few fixes > > back to > > >> zkclient after diving into the code while debugging various issues. > > However > > >> these may take some time to get merged in and subsequently pulled into > > >> Kafka. > > >> > > >> On Tue, Dec 1, 2015 at 5:49 PM, Jay Kreps <j...@confluent.io> wrote: > > >> > > >> > Hey Flavio, > > >> > > > >> > Yeah, I think we are largely in agreement on virtually all points. > > >> > > > >> > Where I saw ZK shine was really in in-house infrastructure. LinkedIn > > had > > >> a > > >> > dozen in-house systems that all used it, and it wouldn't have made > > sense > > >> > for any of those systems to build their own. Likewise when we > started > > >> Kafka > > >> > there was really only 1-3 developers for a very long time so doing > > >> anything > > >> > more custom would have been out of reach. I guess the characteristic > > of > > >> > in-house infrastructure is that it has to be cheap to build, and it > > often > > >> > ends up having lots and lots of other system dependencies which is > > fine > > >> so > > >> > long as they are things you already run. > > >> > > > >> > For an open source product, though you are kind of optimizing with a > > >> > different objective function. You are trying to make the thing easy > to > > >> get > > >> > going with and willing to spend more time to get there. That > > >> out-of-the-box > > >> > experience of how easy it is to adopt and operationalize is the big > > issue > > >> > in how successful the system is. I think using external consensus > > systems > > >> > ends up not being quite as good here because many people won't > already > > >> have > > >> > the dependency as part of their stack and for them you effectively > > double > > >> > the operational footprint they have to master. I think this is why > > this > > >> is > > >> > such a loud and persistent complaint (Joe is right, it is the first > > >> > question asked at every talk I give)--they want to adopt one > > distributed > > >> > thingy (Kafka) which is hard enough to monitor, configure, > understand > > >> etc, > > >> > but to get that we make them learn a second one too. Even when > people > > >> > already have zk in their stack that doesn't really help because it > > turns > > >> > out that sharing the zk cluster safely is usually not easy for > people. > > >> > > > >> > Actually ZK itself is really great in this way--i think a lot of > it's > > >> > success comes from being a totally simple standalone component. If > it > > >> > required some other system to run (dunno what, but say HDFS or MySQL > > or > > >> > whatever) I think it would be far less successful. > > >> > > > >> > Obviously a lot depends on how much is reused and what the > conceptual > > >> > "footprint" of the implementation is. If you end up with something > in > > >> Kafka > > >> > that is basically all standalone code (basically a ZK-like service > but > > >> less > > >> > tested and in scala) then that would be totally stupid. I think that > > is > > >> the > > >> > outcome you are describing. > > >> > > > >> > On the other hand it might be possible to reuse much more. I have > not > > >> done > > >> > a design for something like this so all of this is totally half > baked. > > >> But > > >> > imagine you implement Raft on Kafka (Kraft!). Basically you have a > > >> magical > > >> > kafka log "metadata" and appends, recovery, and leadership for this > > log > > >> are > > >> > not handled using the normal ISR logic but go through a special code > > path > > >> > that implements the raft algorithm. A subset of 3-5 brokers would be > > full > > >> > participants in maintaining this log and the rest of the cluster > would > > >> just > > >> > passively consume it. All brokers have a callback plugin for > > processing > > >> > events appended to the log (this is what handles things like topic > > >> creates > > >> > and deletes), and all have a local rocksdb key-value store > replicating > > >> the > > >> > contents of the log which is queryable. This k/v store holds the > > various > > >> > topic configs, metadata entries, etc. You would implement a new > > >> > RequestVotes RPC for doing leadership election. The leader of the > Raft > > >> > election would also serve as the controller for the duration of it's > > >> term. > > >> > > > >> > I'm not claiming this is well thought out, it's obviously not. It > also > > >> > might not be what you ended up with if you fully rethought the > > >> relationship > > >> > between the consensus log and algorithm used for that and the data > > logs > > >> and > > >> > algorithm used for them. Buuut, pretend for a moment this is > possible > > and > > >> > the implementation was not buggy. I claim that this setup would be > > much > > >> > operationally simpler then operationalizing, securing, monitoring, > and > > >> > configuring both Kafka and a separate zk/consul/etcd cluster. The > > impact > > >> is > > >> > some additional logic for processing the metadata appends, maybe > some > > >> > changes to the log implementation, and a new RPC for leadership > > election. > > >> > > > >> > Thoughts? > > >> > > > >> > -Jay > > >> > > > >> > On Tue, Dec 1, 2015 at 2:25 PM, Flavio Junqueira <f...@apache.org> > > wrote: > > >> > > > >> > > I'm new to this community and I don't necessarily have all the > > >> background > > >> > > around the complaints on ZooKeeper, but I'd like to give my > opinion, > > >> even > > >> > > if biased. To introduce myself in the case folks here don't know > me, > > >> I'm > > >> > > one of the guys who developed ZooKeeper originally and designed > the > > Zab > > >> > > protocol. > > >> > > > > >> > > > On 01 Dec 2015, at 19:58, Jay Kreps <j...@confluent.io> wrote: > > >> > > > > > >> > > > Hey Joe, > > >> > > > > > >> > > > Thanks for raising this. People really want to get rid of the ZK > > >> > > > dependency, I agree it is among the most asked for things. Let > me > > >> give > > >> > a > > >> > > > quick critique and a more radical plan. > > >> > > > > > >> > > > > >> > > Putting aside the cases in which people already use or want to use > > >> > > something else, what's the concern here exactly? People are always > > >> trying > > >> > > to get rid of dependencies but they give up when they start > looking > > at > > >> > the > > >> > > alternatives or the difficulty of doing themselves. Also, during > > this > > >> > past > > >> > > few months that I have been interacting with this community, I > > noticed > > >> at > > >> > > least one issue that was around for a long time, was causing some > > pain, > > >> > and > > >> > > it was due to a poor understanding of the ZK semantics. I'm not > > >> convinced > > >> > > that doing your own will make it go away, unless you have a group > of > > >> > people > > >> > > dedicated to doing it, but then you're doing ZK, etcd, Consul all > > over > > >> > > again, with the difference that it is in your backyard. > > >> > > > > >> > > > I don't think making ZK pluggable is the right thing to do. I > > have a > > >> > lot > > >> > > of > > >> > > > experience with this dynamic of introducing plugins for core > > >> > > functionality > > >> > > > because I previously worked on a key-value store called > Voldemort > > in > > >> > > which > > >> > > > we made both the protocol and storage engine totally pluggable. > I > > >> > > > originally felt this was a good thing both philosophically and > > >> > > practically, > > >> > > > but in retrospect came to believe it was a huge mistake--what > > people > > >> > > really > > >> > > > wanted was one really excellent implementation with the kind of > > >> insane > > >> > > > levels of in-production usage and test coverage that > > infrastructure > > >> > > > demands. Pluggability is actually really at odds with this, and > > the > > >> > > ability > > >> > > > to actually abstract over some really meaty dependency like a > > storage > > >> > > > engine never quite works. > > >> > > > > >> > > Adding another layer of abstraction is likely to cause even more > > pain. > > >> > > Right now there is ZkUtils -> ZkClient -> ZooKeeper client. > > >> > > > > >> > > > > > >> > > > People dislike the ZK dependency because it effectively doubles > > the > > >> > > > operational load of Kafka--it doubles the amount of > configuration, > > >> > > > monitoring, and understanding needed. > > >> > > > > >> > > I agree, but it isn't necessarily a problem with the dependency, > but > > >> how > > >> > > people think about the dependency. Taking as an example the recent > > >> > security > > >> > > work, SASL was done separately for brokers and zk. It would make > > >> > > configuration easier if we just said SASL and under the hood > > configured > > >> > > what is necessary. Let me stress that I'm not complaining about > the > > >> > > security work, and it is even possible that they were separated > for > > >> > > different reasons, like a zk ensemble being shared, but I feel > this > > is > > >> an > > >> > > example in which the developers in this community tend to think of > > it > > >> > > separately. Actually, one interesting point is that some of the > > >> security > > >> > > apparently was borrowed from zk. Not a problem, it is an > open-source > > >> > > project after all, but there could be better synergy here. > > >> > > > > >> > > > Replacing ZK with a similar system > > >> > > > won't fix this problem though--all the other consensus services > > are > > >> > > equally > > >> > > > complex (and often less mature)--and it will cause two new > > problems. > > >> > > First > > >> > > > there will be a layer of indirection that will make reasoning > and > > >> > > improving > > >> > > > the ZK implementation harder. For example, note that your > plug-in > > api > > >> > > > doesn't seem to cover multi-get and multi-write, when we added > > that > > >> we > > >> > > > would end up breaking all plugins. Each new thing will be like > > that. > > >> > Ops > > >> > > > tools, config, documentation, etc will no longer be able to > > include > > >> any > > >> > > > coverage of ZK because we can't assume ZK so all that becomes > much > > >> > > harder. > > >> > > > The second problem is that this introduces a combinatorial > testing > > >> > > problem. > > >> > > > People say they want to swap out ZK but they are assuming > whatever > > >> they > > >> > > > swap in will work equally well. How will we know that is true? > The > > >> only > > >> > > way > > >> > > > to explode out the testing to run with every possible plugin. > > >> > > > > >> > > That's possibly less of a problem if you have a single instance > that > > >> the > > >> > > community supports. Others can implement their option, but the > > >> community > > >> > > doesn't necessarily have to maintain or make sure they all work > > prior > > >> to > > >> > > releases. > > >> > > > > >> > > > > > >> > > > If you want to see this in action take a look at ActiveMQ. > > ActiveMQ > > >> is > > >> > > less > > >> > > > a system than a family of co-operating plugins and a > configuration > > >> > > language > > >> > > > for assembling them. Software engineers and open source > > communities > > >> are > > >> > > > really prone to this kind of thing because "we can just make it > > >> > > pluggable" > > >> > > > ends any argument. But the actual implementation is a mess, and > > later > > >> > > > improvements in their threading, I/O, and other core models > simply > > >> > > couldn't > > >> > > > be made across all the plugins. > > >> > > > > > >> > > > This blog post on configurability in UI is a really good summary > > of a > > >> > > > similar dynamic: > > >> > > > http://ometer.com/free-software-ui.html > > >> > > > > > >> > > > Anyhow, not to go too far off on a rant. Clearly I have plugin > > PTSD > > >> :-) > > >> > > > > > >> > > > I think instead we should explore the idea of getting rid of the > > >> > > zookeeper > > >> > > > dependency and replace it with an internal facility. Let me > > explain > > >> > what > > >> > > I > > >> > > > mean. In terms of API what Kafka and ZK do is super different, > but > > >> > > > internally it is actually quite similar--they are both trying to > > >> > > maintain a > > >> > > > CP log. > > >> > > > > >> > > Yes, but interestingly your CP log depends on ZK for consensus. It > > is > > >> > > accurate that the correctness of ZK depends on a correct > > replication of > > >> > the > > >> > > txn log, though, but it is naive to think that the step to get a > > >> correct > > >> > > broadcast primitive is small. > > >> > > > > >> > > > > > >> > > > What would actually make the system significantly simpler would > > be to > > >> > > > reimplement the facilities you describe on top of Kafka's > existing > > >> > > > infrastructure--using the same log implementation, network > stack, > > >> > config, > > >> > > > monitoring, etc. If done correctly this would dramatically lower > > the > > >> > > > operational load of the system versus the current Kafka+ZK or > > >> proposed > > >> > > > Kafka+X. > > >> > > > > >> > > Perhaps the general understanding of these protocols have advanced > > >> enough > > >> > > that it doesn't make sense any longer to dependent on something > like > > >> ZK, > > >> > > etcd, or Consul. One of the reasons for doing ZK in the first > place > > was > > >> > > exactly to free developers from the burden of dealing with the > > >> complexity > > >> > > of broadcast/consensus protocols. The main problem that I've seen > > more > > >> > than > > >> > > once is that the homebred version ends up abandoned because it > isn't > > >> the > > >> > > main focus of the project, and causes more pain. One of the things > > that > > >> > > worked with ZK was to have a community and a group of people > > >> responsible > > >> > > for thinking about it and maintaining it. > > >> > > > > >> > > > > > >> > > > I don't have a proposal for how this would work and it's some > > effort > > >> to > > >> > > > scope it out. The obvious thing to do would just be to keep the > > >> > existing > > >> > > > ISR/Controller setup and rebuild the controller etc on a > > RAFT/Paxos > > >> > impl > > >> > > > using the Kafka network/log/etc and have a replicated config > > database > > >> > > > (maybe rocksdb) that was fed off the log and shared by all > nodes. > > >> > > > > > >> > > > > >> > > In principle, you need mechanisms like watches, ephemerals, and > just > > >> pure > > >> > > storage. Just implement a replicated state machine with those > > >> operations. > > >> > > Watches and ephemerals make it a bit hard to do using a REST API, > > but > > >> it > > >> > is > > >> > > fine if you don't care about doing long polls. Perhaps this is too > > much > > >> > > detail for the status of this discussion. > > >> > > > > >> > > > > >> > > > If done well this could have the advantage of potentially > > allowing us > > >> > to > > >> > > > scale the number of partitions quite significantly (the k/v > store > > >> would > > >> > > not > > >> > > > need to be all in memory), though you would likely still have > > limits > > >> on > > >> > > the > > >> > > > number of partitions per machine. This would make the minimum > > Kafka > > >> > > cluster > > >> > > > size be just your replication factor. > > >> > > > > > >> > > > > >> > > The idea of not having it in memory to scale isn't a bad one, > > specially > > >> > if > > >> > > you SSDs to cache data and such. > > >> > > > > >> > > > > >> > > > People tend to feel that implementing things like RAFT or Paxos > is > > >> too > > >> > > hard > > >> > > > for mere mortals. But I actually think it is within our > > capabilities, > > >> > and > > >> > > > our testing capabilities as well as experience with this type of > > >> thing > > >> > > have > > >> > > > improved to the point where we should not be scared off if it is > > the > > >> > > right > > >> > > > path. > > >> > > > > >> > > It is hard, but certainly not impossible. I have implemented > quite a > > >> few > > >> > > Paxos prototypes over the years for experiments, and getting off > the > > >> > ground > > >> > > isn't hard, but getting an implementation that really works and > > that it > > >> > > maintainable is difficult. I've seen more than one instance where > > >> people > > >> > > got it wrong and caused pain. But again, maybe times have changed > > and > > >> > that > > >> > > won't happen here. > > >> > > > > >> > > -Flavio > > >> > > > > >> > > > > > >> > > > This approach is likely more work then plugins (though maybe > not, > > >> once > > >> > > you > > >> > > > factor in all the docs, testing, etc) but if done correctly it > > would > > >> be > > >> > > an > > >> > > > unambiguous step forward--a simpler, more scalable > implementation > > >> with > > >> > no > > >> > > > operational dependencies. > > >> > > > > > >> > > > Thoughts? > > >> > > > > > >> > > > -Jay > > >> > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Tue, Dec 1, 2015 at 11:12 AM, Joe Stein < > joe.st...@stealth.ly> > > >> > wrote: > > >> > > > > > >> > > >> I would like to start a discussion around the work that has > > started > > >> in > > >> > > >> regards to KIP-30 > > >> > > >> > > >> > > >> > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-30+-+Allow+for+brokers+to+have+plug-able+consensus+and+meta+data+storage+sub+systems > > >> > > >> > > >> > > >> The impetus for working on this came a lot from the community. > > For > > >> the > > >> > > last > > >> > > >> year(~+) it has been the most asked question at any talk I have > > >> given > > >> > > >> (personally speaking). It has come up a bit also on the mailing > > list > > >> > > >> talking about zkclient vs currator. A lot of folks want to use > > Kafka > > >> > but > > >> > > >> introducing dependencies are hard for the enterprise so the > goals > > >> > behind > > >> > > >> this is making it so that using Kafka can be done as easy as > > >> possible > > >> > > for > > >> > > >> the operations teams to-do when they do. If they are already > > >> > supporting > > >> > > >> ZooKeeper they can keep doing that but if not they want (users) > > to > > >> use > > >> > > >> something else they are already supporting that can plug-in > to-do > > >> the > > >> > > same > > >> > > >> things. > > >> > > >> > > >> > > >> For the core project I think we should leave in upstream what > we > > >> have. > > >> > > This > > >> > > >> gives a great baseline regression for folks and makes the work > > for > > >> > > "making > > >> > > >> what we have plug-able work" a good defined task (carve out, > > layer > > >> in > > >> > > API > > >> > > >> impl, push back tests pass). From there then when folks want > > their > > >> > > >> implementation to be something besides ZooKeeper they can > > develop, > > >> > test > > >> > > and > > >> > > >> support that if they choose. > > >> > > >> > > >> > > >> We would like to suggest that we have the plugin interface be > > Java > > >> > based > > >> > > >> for minimizing depends for JVM impl. This could be in another > > >> > directory > > >> > > >> something TBD /<name>. > > >> > > >> > > >> > > >> If you have a server you want to try to get it working but you > > >> aren't > > >> > on > > >> > > >> the JVM don't be afraid just think about a REST impl and if you > > can > > >> > work > > >> > > >> inside of that you have some light RPC layers (this was the > first > > >> pass > > >> > > >> prototype we did to flush-out the public api presented on the > > KIP). > > >> > > >> > > >> > > >> There are a lot of parts to working on this and the more > > >> > > implementations we > > >> > > >> have the better we can flush out the public interface. I will > > leave > > >> > the > > >> > > >> technical details and design to JIRA tickets that are linked > > through > > >> > the > > >> > > >> confluence page as these decisions come about and code starts > for > > >> > > reviews > > >> > > >> and we can target the specific modules having the context > > separate > > >> is > > >> > > >> helpful especially if multiple folks are working on it. > > >> > > >> https://issues.apache.org/jira/browse/KAFKA-2916 > > >> > > >> > > >> > > >> Do other folks want to build implementations? Maybe we should > > start > > >> a > > >> > > >> confluence page for those or use an existing one and add to it > > so we > > >> > can > > >> > > >> coordinate some there to. > > >> > > >> > > >> > > >> Thanks! > > >> > > >> > > >> > > >> ~ Joe Stein > > >> > > >> - - - - - - - - - - - - - - - - - - - > > >> > > >> [image: Logo-Black.jpg] > > >> > > >> http://www.elodina.net > > >> > > >> http://www.stealth.ly > > >> > > >> - - - - - - - - - - - - - - - - - - - > > >> > > >> > > >> > > > > >> > > > > >> > > > >> > > > > > > > > > > > > -- > > > *—-* > > > *Todd Palino* > > > Staff Site Reliability Engineer > > > Data Infrastructure Streaming > > > > > > > > > > > > linkedin.com/in/toddpalino > > > > > > -- > *—-* > *Todd Palino* > Staff Site Reliability Engineer > Data Infrastructure Streaming > > > > linkedin.com/in/toddpalino >
