restoring broker node without Zookeeper data?

Running isolated primary, secondary, etc, Kafka brokers without replication (clients failover to other Kafka brokers if needed). Each broker instance has a single local Zookeeper instance. Using periodic backups/snapshots of topic data across locations for disaster recovery. It appears without t

Re: compile error sbt / log4j , cannot exclude log4j

er: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********/ > > > On Thu, Jan 16, 2014 at 2:04 AM, Ran RanUser wrote: > > > Using : > > > > SBT 13 > > Scala 2.10.3 > > Kafka 0.8.0 > > >

compile error sbt / log4j , cannot exclude log4j

Using : SBT 13 Scala 2.10.3 Kafka 0.8.0 Getting a long cryptic compile error ending if I exclude log4j: [error] uncaught exception during compilation: scala.reflect.internal.Types$TypeError [trace] Stack trace suppressed: run last shops-integrationlayer/compile:compile for the full output. [erro

restoring broker node / zookeeper question

I have two questions: 1. We plan to initially run Kafka without partitioning or replication, simply having two single node Kafka (broker) servers in two geographically diverse locations. Using periodic backups/snapshots across locations for disaster recovery. Our producer / consumer application

SimpleConsumer - Should FetchRequests span log segments?

Using Kafka 0.8, when specifying a starting offset and appropriate fetchSize, SimpleConsumer will only return up to the highest offset of the log segment containing the starting offset. For example, log segment #1 contains offsets 1 - 10 log segment #2 contiains offsets 11 - 100 A fetch request

Re: 0.8 backup strategy anyone?

sync mode) utilizing request.required.acks = 0 (I assume connection failures still throw exceptions) and send(messages: KeyedMessage[K,V]*) in hopes of achieving async-like performance Thanks! On Sun, Jun 23, 2013 at 7:11 AM, Ran RanUser wrote: > Offset preserving mirroring would be a great addition,

Re: kafka.common.FailedToSendMessageException - 0.8

why the send fails in WARN. Could you > post that? Also, any error/exception in the broker log (including > state-change log)? > > Thanks, > > Jun > > > On Thu, Jun 20, 2013 at 12:08 AM, Ran RanUser wrote: > > > I'm stumped: > > > > My sample P

Re: 0.8 backup strategy anyone?

n > with this approach, the offsets in the target cluster will be slightly > behind those in the source cluster since the mirroring will be async. Since > not all offsets will be preserved. > > Thanks, > > Jun > > > On Sun, Jun 16, 2013 at 3:02 PM, Ran RanUser wrote

kafka.common.FailedToSendMessageException - 0.8

I'm stumped: My sample Producer code does not work in Windows, but does work in Linux (Ubunuty). My sample Consumer code is fine. - Kaka 0.8 (June 19), compiled using Scala 2.9.2 - Java version is JDK 1.7x (Oracle version in Windows, OpenJDK on Ubuntu) - 1 broker, 1 partition,1 replica, no compr

Upgrade to Zookeeper 3.4.5 stable ?

I'm running Zookeeper 3.4.5 server and using 3.4.5 clients in other code. Will Kafka switch to Zookeeper 3.4.5 soon? Should I expect any potential issues if Kafka's Zookeeper client is 3.3.4 while our server is 3.4.5? Thanks!

Re: 0.8 backup strategy anyone?

I've been researching Kafka for our requirements and am trying to figure out the best way to implement multi-region failover (lowest complexity). One requirement we have is that the offsets of the backup must match the primary. As I understand it, MirrorMaker does not (currently) guarantee that t