kafka.common.UnknownException

2016-09-06 Thread Michael Ravits
Hi, We are getting the error i've pasted at the bottom of the mail in a Samza task, it never recovers and keeps printing messages like this in the log: 2016-07-27 17:46:20 BrokerProxy [DEBUG] Removed [event,1] 2016-07-27 17:46:20 KafkaSystemConsumer [INFO] Abdicating for [event,1] 2016-07-27 17:4

Samza 0.10.0 with Kafka 0.9.0.0

2016-02-19 Thread Michael Ravits
Hi Everyone, Does anyone run Samza 0.10 with Kafka 0.9.0.0? Is this a stable configuration? Thanks, Michael

Re: Monitoring consumer lag

2015-11-16 Thread Michael Ravits
inspecting on the consumer end. > -Jordan > > On Mon, Nov 16, 2015 at 9:16 AM, Michael Ravits > wrote: > > > Hi, > > > > I'd like to monitor consumer's lag. > > Found this tool https://github.com/linkedin/Burrow. > > But now realized that Samza

Monitoring consumer lag

2015-11-16 Thread Michael Ravits
Hi, I'd like to monitor consumer's lag. Found this tool https://github.com/linkedin/Burrow. But now realized that Samza is using it's own checkpointing mechanism. So question is what's the best way to monitor whether and how much the consumer is lagging? On a related subject, I'd also like to mo

Re: ProcessJobFactory parent process

2015-06-01 Thread Michael Ravits
less memory is needed) with ThreadJobFactory. > > One downside was that the logger wouldn't respect the lib/log4j.xml > configuration file anymore, but that's probably some error on my part. > > Lukas > > On Sat, May 30, 2015 at 2:41 PM, Michael Ravits > wrote: &g

Re: ProcessJobFactory parent process

2015-05-30 Thread Michael Ravits
Hi Lukas, We use similar docker based deployment. Would be glad if you could post here if ThreadJobFactory worked well for you. Thanks, Michael On Sat, May 30, 2015 at 12:22 AM, Lukas Steiblys wrote: > Yes, I think switching to ThreadJobFactory is a good solution. I think the > reasons why I s

Re: Number of partitions

2015-05-21 Thread Michael Ravits
t 1 partition. So there will be 500 processes working > through the log. > > I'd try to figure out what your scaling needs are for the next 2-3 years > and then calculate your resource requirements accordingly (how many > parallel executing tasks you would need). If yo

Re: Number of partitions

2015-05-21 Thread Michael Ravits
ur system processing 100 messages/s and there's plenty of > room to grow. > > Lukas > > On Thursday, May 21, 2015, Michael Ravits wrote: > > > Hi, > > > > I wonder what are the considerations I need to account for in regard to > the > > number of pa

Number of partitions

2015-05-21 Thread Michael Ravits
Hi, I wonder what are the considerations I need to account for in regard to the number of partitions in input topics for Samza. When testing with a 500 partitions topic with one Samza job I noticed the start up time to be very long. Are there any problems that might occur when dealing with this nu

Re: Clojure Samza Task

2015-05-18 Thread Michael Ravits
which means void actually returns something else? Sincerely, Michael On Mon, May 18, 2015 at 4:21 PM, Gustavo Anatoly wrote: > Hi, Michael Ravits. > > The error happens when the Log class try to read the > jetty-logging.properties file. Please take a

Re: Clojure Samza Task

2015-05-18 Thread Michael Ravits
which means void actually returns something else? Sincerely, Michael On Mon, May 18, 2015 at 4:21 PM, Gustavo Anatoly wrote: > Hi, Michael Ravits. > > The error happens when the Log class try to read the > jetty-logging.properties file. Please take a

Clojure Samza Task

2015-05-17 Thread Michael Ravits
Hi Everybody, Trying to get up and running with a Clojure Samza Task. I think that my question is Clojure specific but I guess that some of the folks here are Clojure developers which might also know the answers. I'm trying to run a job, using the same code as here: https://github.com/apache/samz