Re: Review Request 51142: SAMZA-967: HDFS System Consumer

> On Aug. 31, 2016, 7:25 a.m., Navina Ramesh wrote: > > gradle/dependency-versions.gradle, line 39 > > > > > > Why is this dependency introduced? Is it possible to get rid of this > > dependency ? Yes. We can remove

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

> On Aug. 26, 2016, 7:51 p.m., Chris Pettitt wrote: > > samza-api/src/main/java/org/apache/samza/system/IncomingMessageEnvelope.java, > > line 31 > > > > > > How likely are we to collide with this? Thats the problem

kafka.common.UnknownException

Hi, We are getting the error i've pasted at the bottom of the mail in a Samza task, it never recovers and keeps printing messages like this in the log: 2016-07-27 17:46:20 BrokerProxy [DEBUG] Removed [event,1] 2016-07-27 17:46:20 KafkaSystemConsumer [INFO] Abdicating for [event,1] 2016-07-27 17:4

Re: Periodic cleanup of unused local stores

Hi Santhosh, Thanks for picking SAMZA-656. This is long overdue and will help make our host-affinity based solution more robust. I have a couple of thoughts on your design proposal. 1. It is always very useful to provide more context to the reader, esp. in explaining what the different terms mean

Re: Samza container hang on exception

Hi, Sining, I took a look at your log and stack traces and want to clarify two points: 1) It seems that your container actually exited, instead of hanging, based on the log, which is the expected behavior from 0.10.1 (retry X times and error-out in SamzaContainer RunLoop). 2) The Kafka producer c

Re: checkpoint on flush of system producer

Hi, Jarrad, Yes! You have found your answer! Looking forward to your implementation of SystemProducer. Just curious, what's the target output system that you are writing to? -Yi On Tue, Sep 6, 2016 at 9:01 AM, Jarrad, Ken wrote: > I think I have discovered the answer to my question. Hopefully

RE: checkpoint on flush of system producer

I think I have discovered the answer to my question. Hopefully someone can confirm my understanding. The 'flush' method of my SystemProducer will be invoked prior to the checkpoint. Thus the checkpoint cannot get ahead of the SystemProducer. Thus the guarantee is 'at least once'. I will do my

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51346/ --- (Updated Sept. 6, 2016, 4:02 p.m.) Review request for samza, Boris Shkolnik, Ch

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

> On Aug. 29, 2016, 11:54 p.m., Xinyu Liu wrote: > > samza-core/src/main/java/org/apache/samza/task/AsyncRunLoop.java, line 375 > > > > > > No need to resume. This is running on the main thread. Not calling resume()

checkpoint on flush of system producer

I am writing a SystemProducer and I would be grateful for any comments or documentation. I want my SystemProducer to collect messages until flush is invoked. That is, I don't want to transmit each message 'one by one'. I want to wait until flush is invoked and then transmit 'in bulk'. I don't kn

Re: Review Request 51630: SAMZA-1007: Pass job config from factory to grouper

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51630/ --- (Updated Sept. 6, 2016, 8:41 a.m.) Review request for samza. Bugs: SAMZA-1007

Re: Review Request 51630: SAMZA-1007: Pass job config from factory to grouper

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51630/ --- (Updated Sept. 6, 2016, 8:39 a.m.) Review request for samza. Bugs: SAMZA-1007