Re: Review Request 37528: SAMZA-736 BrokerProxy will stuck in infinite loop if consumer.fetch throws OOME

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37528/#review95817 --- samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProx

Re: KIP-28 kafka processor

You can definitely discuss here, but it might be best to do it on the kafka mailing list thread. We usually link to the discussion thread from the proposal but looks like we missed that here. The discussion always has the KIP name in the title. We try to avoid wiki comments just because they quickl

Re: [Discuss/Vote] upgrade to Yarn 2.6.0

Yarn 2.4 lacks from useful rest api as well. I prefer 2.6 as well. +1 ! On Mon, Aug 17, 2015 at 8:03 PM, Chris Riccomini wrote: > +1 on my end > > On Monday, August 17, 2015, Yi Pan wrote: > > > Hi, Yan, > > > > Thanks for rolling the ball! > > > > +1 from me to upgrade the minimum supported ve

Re: Use one producer for both coordinator stream and users system?

Hi, Roger, Good point. I would agree with Yan that we do not necessarily need two producers for coordinator topic vs user topic. That would reduce the CPU overhead as well. Especially in the container, since it does not consume from coordinator topic yet, sending messages to coordinator topic is n

Re: Use one producer for both coordinator stream and users system?

Thanks Yan. I guess I am not very clear with the coordinatorStream concept before. -Tao On Tue, Aug 18, 2015 at 12:26 AM, Yan Fang wrote: > Hi Tao, > > First, one kafka producer has an i/o thread. (correct me if I am wrong). > > Second, after Samza 0.10.0, we have a coordinator stream, which sto

Re: Use one producer for both coordinator stream and users system?

Hi Yan, My (uneducated) guess is that the performance gains come from batching. I don't know if the new producer ever batches by destination broker. If not and it only batches by (broker,topic,partition) then I doubt that one vs two producers will affect performance as they send to different top

Re: Review Request 37528: SAMZA-736 BrokerProxy will stuck in infinite loop if consumer.fetch throws OOME

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37528/ --- (Updated Aug. 18, 2015, 12:58 p.m.) Review request for samza. Changes ---

Re: Use one producer for both coordinator stream and users system?

Hi Tao, First, one kafka producer has an i/o thread. (correct me if I am wrong). Second, after Samza 0.10.0, we have a coordinator stream, which stores the checkpoint, config and other locality information for auto-scaling, dynamic configuration, etc purpose. (See Samza-348

Re: Review Request 37069: SAMZA-738 Samza Timer based metrics does not have enough precision

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37069/#review95692 --- Ship it! LGTM. - Yi Pan (Data Infrastructure) On Aug. 5, 2015, 2

Re: remote kafka producer -- kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries

Thanks, Selina, for sharing this solution for the reference. :) Cheers, Fang, Yan yanfang...@gmail.com On Mon, Aug 17, 2015 at 7:17 PM, Job-Selina Wu wrote: > Hi, All: > > Finally I fixed this bug. > > 1. set advertised.host.name at config/server.properites as AWS > *private IP* address

Re: KIP-28 kafka processor

Thanks, Chris and Jay. So do we add comments in this thread? Seems I can not leave comments in the confluence. :) Thanks, Fang, Yan yanfang...@gmail.com On Mon, Aug 17, 2015 at 11:05 PM, Yi Pan wrote: > Hi, Chris and Jay, > > Thanks for the reminder. I plan to follow up this week. > > Cheers!