Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Gerrit Jansen van Vuuren
Hi Gwen, For other reasons :), I've used zookeeper (from Java) in the past to store similar information and have found it doesnt work good enough for this use case in particular. Zk is great for config and locks but not for data that changes fast and can grow beyond a certain limit. In particular

Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Gwen Shapira
Out of curiosity: did you choose Redis because ZooKeeper is not well supported in Clojure? Or were there other reasons? On Mon, Oct 13, 2014 at 2:04 PM, Gerrit Jansen van Vuuren wrote: > Hi Steven, > > Redis: > > I've had a discussion on redis today, and one architecture that does come > up is

Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Gerrit Jansen van Vuuren
Hi Steven, Redis: I've had a discussion on redis today, and one architecture that does come up is using a master slave, then if the master fails the have the application start writing to the slave. Writing to a slave is possible in redis, albeit you cannot fail back to the master because writes

Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Steven Schlansker
Couple of mostly-uninformed comments inline, On Oct 13, 2014, at 2:00 AM, Gerrit Jansen van Vuuren wrote: > Hi Daniel, > > At the moment redis is a spof in the architecture, but you can setup > replication and I'm seriously looking into using redis cluster to eliminate > this. > Some docs t

Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Gerrit Jansen van Vuuren
Hi Daniel, At the moment redis is a spof in the architecture, but you can setup replication and I'm seriously looking into using redis cluster to eliminate this. Some docs that point to this are: http://redis.io/topics/cluster-tutorial http://redis.io/topics/sentinel Consumer: Consumpt

Re: kafka java api (written in 100% clojure)

2014-10-13 Thread Daniel Compton
Hi Gerrit Thanks for your contribution, I'm sure everyone here appreciates it, especially Clojure developers like myself. I do have one question: what are the guarantees you offer to users of your library under failures, particularly when Redis fails? -- Daniel > On 13/10/2014, at 10:22 am, G

kafka java api (written in 100% clojure)

2014-10-12 Thread Gerrit Jansen van Vuuren
Hi, Just thought I'll put this out for the kafka community to see (if anyone finds it useful great!!). Kafka-fast is 100% pure clojure implementation for kafka, but not just meant for clojure because it has a Java API wrapper that can be used from Java, Groovy, JRuby or Scala. This library does