GC Woes

2015-11-18 Thread Cory Kolbeck
Hi folks, I've been chasing an issue for a bit now without much luck. We're seeing occasional (1-2 times a day) pause times of 10+ seconds in a 0.8.2.0 broker only handling ~3k messages/s. We're only seeing it on one node at a time in a three node cluster, though which node is affected can change

Re: G1 tuning

2015-10-14 Thread Cory Kolbeck
> > -Todd > > > On Wed, Oct 14, 2015 at 2:18 PM, Scott Clasen wrote: > > > You can also use -Xmn with that gc to size the new gen such that those > > buffers don't get tenured > > > > I don't think that's an option with G1 > > >

Re: G1 tuning

2015-10-14 Thread Cory Kolbeck
seen pauses using G1 in other applications and have found that > -XX:+UseParallelGC > -XX:+UseParallelOldGC works best if you're having GC issues in general on > the JVM. > > > Regards, > Gerrit > > On Wed, Oct 14, 2015 at 4:28 PM, Cory Kolbeck wrote: > > > Hi

G1 tuning

2015-10-14 Thread Cory Kolbeck
Hi folks, I'm a bit new to the operational side of G1, but pretty familiar with its basic concept. We recently set up a Kafka cluster to support a new product, and are seeing some suboptimal GC performance. We're using the parameters suggested in the docs, except for having switched to java 1.8_40

Re: Urban Airship Connect

2015-10-09 Thread Cory Kolbeck
h load does the first and second level Kafka clusters > intake? > > What is your write throughput, and end-to-end latency, for how long do you > hold data in Kafka and how many minimum in-sync replicas you've chosen? > > Thanks, > Prabhjot > On Oct 10, 2015 1:41 AM, &q

Urban Airship Connect

2015-10-09 Thread Cory Kolbeck
://www.urbanairship.com/blog/how-we-built-urban-airship-connect respectively. The posts aren't hugely in depth engineering wise, but I hope folks find them interesting. Cory Kolbeck Senior Data Engineer Urban Airship