Re: Using Kafka 0.8 from Scala and Akka

2013-12-03 Thread Richard Rodseth
Any update on this, Chetan? Thanks. On Thu, Oct 31, 2013 at 4:11 PM, chetan conikee wrote: > I am in the process of releasing out Scala and RxJava consumer(s) on > github. Will be releasing it soon. Keep an eye out. > > > On Thu, Oct 31, 2013 at 3:49 PM, Richard Rodseth > wrote: > > > So I hav

[ANNOUNCEMENT] Apache Kafka 0.8.0 Released

2013-12-03 Thread Joe Stein
The Apache Kafka community is pleased to announce the general availability release of Apache Kafka 0.8.0. Apache Kafka is publish-subscribe messaging system rethought as a distributed commit log. ** Fast => A single Kafka broker can handle hundreds of megabytes of reads and writes per second f

Re: snappy 1.0.4.1, java7, macos issue...

2013-12-03 Thread Joe Stein
too late for 0.8.0 , yup don't see any reason for it to not be in 0.8.1 and get on trunk once there is a patch available for it ... happy to review once if a patch gets up there otherwise I can do the patch , np i update the ticket to label it for the release so we don't miss it, always good to k

snappy 1.0.4.1, java7, macos issue...

2013-12-03 Thread Jason Rosenberg
So, It looks like there is an issue with using snappy version 1.0.4.1, with java 7, on MacOSX, which is fixed in version 1.0.5. We've verified this. https://github.com/ptaoussanis/carmine/issues/5 Perhaps 1.0.5 should be the default for kafka 0.8 (or maybe too late for that?). I've filed: htt

Re: set new retention size on the fly

2013-12-03 Thread Jun Rao
Just to clarify. Even though # partition is a config property of a topic, it has to be done through the add partition admin command. Other topic level config changes can be made through "kafka-topics.sh --alter" in 0.8.1 (trunk). Thanks, Jun On Tue, Dec 3, 2013 at 3:56 PM, Jay Kreps wrote: >

Re: set new retention size on the fly

2013-12-03 Thread Jay Kreps
0.8 includes the partition increasing tool. 0.8.1 makes all per-topic configuration dynamic and updatable via a command line tool. -Jay On Tue, Dec 3, 2013 at 1:23 PM, Yu, Libo wrote: > Hi Neha, > > "0.8.1 includes the ability to dynamically change per topic configs." > Do you mean number of p

RE: set new retention size on the fly

2013-12-03 Thread Yu, Libo
Hi Neha, "0.8.1 includes the ability to dynamically change per topic configs." Do you mean number of partitions or retention size? Regards, Libo -Original Message- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Tuesday, December 03, 2013 1:25 PM To: users@kafka.apache.org

Re: set new retention size on the fly

2013-12-03 Thread Neha Narkhede
For adding partitions, look at https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-5.AddPartitionTool 0.8.1 includes the ability to dynamically change per topic configs. Thanks, Neha On Tue, Dec 3, 2013 at 10:21 AM, Yu, Libo wrote: > Hi folks, > > For 0.8, it i

set new retention size on the fly

2013-12-03 Thread Yu, Libo
Hi folks, For 0.8, it is possible to add a partition dynamically. Is it possible to increase the retention size on the fly? This feature will be very useful for operation. I know rolling start can pick up the change but it takes too much effort. Thanks. Libo

Re: Loggly's use of Kafka on AWS

2013-12-03 Thread Philip O'Toole
Yeah, we use provisioned IOPs EBS so we have good performance. We run 3 ZK nodes, one in each AZ. Quorum 2. Philip On Dec 3, 2013, at 7:31 AM, Timothy Chen wrote: > Hi Philip, > > So I wonder if you guys hit disk perf problems with EBS? It seems quite > common in the past but I haven't tri

Re: Loggly's use of Kafka on AWS

2013-12-03 Thread Timothy Chen
Hi Philip, So I wonder if you guys hit disk perf problems with EBS? It seems quite common in the past but I haven't tried recently. Also can you share how you guys deployed zookeeper in AWS so that a qurom is always available? Tim Sent from my iPhone > On Dec 2, 2013, at 5:15 PM, Steve Morin