Should not matter. We're running 12.04. Wes On Jun 16, 2015 12:18 PM, "Henry Cai" <h...@pinterest.com.invalid> wrote:
> Does it still matter whether we are using Ubuntu 14 or 12? > > On Tue, Jun 16, 2015 at 8:44 AM, Wesley Chow <w...@chartbeat.com> wrote: > > > > > A call with Amazon confirmed instability for d2 and c4 instances > triggered > > by lots of network activity. They fixed the problem and have since rolled > > it out. We've been running Kafka with d2's for a little while now and so > > far so good. > > > > Wes > > > > > > On Tue, Jun 2, 2015 at 1:39 PM, Wes Chow <w...@chartbeat.com> wrote: > > > >> > >> We have run d2 instances with Kafka. They're currently unstable -- > Amazon > >> confirmed a host issue with d2 instances that gets tickled by a Kafka > >> workload yesterday. Otherwise, it seems the d2 instance type is ideal > as it > >> gets an enormous amount of disk throughput and you'll likely be network > >> bottlenecked. > >> > >> Wes > >> > >> > >> Steven Wu <stevenz...@gmail.com> > >> June 2, 2015 at 1:07 PM > >> EBS (network attached storage) has got a lot better over the last a few > >> years. we don't quite trust it for kafka workload. > >> > >> At Netflix, we were going with the new d2 instance type (HDD). our > >> perf/load testing shows it satisfy our workload. SSD is better in > latency > >> curve but pretty comparable in terms of throughput. we can use the extra > >> space from HDD for longer retention period. > >> > >> On Tue, Jun 2, 2015 at 9:37 AM, Henry Cai <h...@pinterest.com.invalid> > >> <h...@pinterest.com.invalid> > >> > >> Henry Cai <h...@pinterest.com.INVALID> > >> June 2, 2015 at 12:37 PM > >> We have been hosting kafka brokers in Amazon EC2 and we are using EBS > >> disk. But periodically we were hit by long I/O wait time on EBS in some > >> Availability Zones. > >> > >> We are thinking to change the instance types to a local HDD or local > SSD. > >> HDD is cheaper and bigger and seems quite fit for the Kafka use case > which > >> is mostly sequential read/write, but some early experiments show the HDD > >> cannot catch up with the message producing speed since there are many > >> topic/partitions on the broker which actually makes the disk I/O more > >> randomly accessed. > >> > >> How are people's experience of choosing disk types on Amazon? > >> > >> > > >
