Hello, We use docker for kafka on vm's with both nas and local disk. We mount the volumes externally. We havent had many problems at all, and a restart has cleared any issue. We are on .8.1
We are also started to deploy to aws. -- Colin +1 612 859 6129 Skype colin.p.clark > On Mar 4, 2015, at 10:46 PM, Otis Gospodnetic <otis.gospodne...@gmail.com> > wrote: > > Hi, > >> On Fri, Feb 27, 2015 at 1:36 AM, James Cheng <jch...@tivo.com> wrote: >> >> Hi, >> >> I know that Netflix might be talking about "Kafka on AWS" at the March >> meetup, but I wanted to bring up the topic anyway. >> >> I'm sure that some people are running Kafka in AWS. > > > I'd say most, not some :) > > >> Is anyone running Kafka within docker in production? How does that work? > > Not us. When I was at DevOps Days in NYC last year, everyone was talking > about Docker, but only about 2.5 people in the room actually really used it. > > For both of these, how do you persist data? If on AWS, do you use EBS? Do >> you use ephemeral storage and then rely on replication? And if using >> docker, do you persist data outside the docker container and on the host >> machine? > > We've used both EBD and local disks in AWS. We don't have Kafka > replication, as far as I know. > > And related, how do you deal with broker failure? Do you simply replace it, >> and repopulate a new broker via replication? Or do you bring back up the >> broker with the persisted files? > > We monitor all Kafka pieces - producers, consumer, and brokers with SPM. > We have alerts and anomaly detection enabled for various Kafka metrics > (yeah, consumer lag being one of them). > Broker failures have been very rare (we've used 0.7.2, 0.8.1.x, and are now > on 0.8.2). When they happened a restart was typically enough. I can recall > one instance where segments recovery tool a long time (minutes, maybe more > than an hour), but this was >6 months ago. > > >> Trying to learn about what people are doing, beyond "on premises and >> dedicated hardware". > > In my world almost everyone I talk to is in AWS. > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/
