Re: Review Request 43350: SAMZA-867 Fix job restart/shutdown in the event of a node outage.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43350/#review118610 --- LGTM. Just one minor comment. samza-yarn/src/main/java/org/apach

Re: Allow user-specified class-loader

No Problem. I'm new to samza so it is entirely possible there's already a way to do this that I'm missing. What I'm aiming for, is to implement the features described in the README of my project https://github.com/cddr/samza-config. For discussion purposes, I'll copy the example here... (ns examp

Re: Review Request 43350: SAMZA-867 Fix job restart/shutdown in the event of a node outage.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43350/#review118604 --- First question: is this an independent patch or depend on RB 43074

Re: Review Request 43074: SAMZA-866 Refactor and fix Container allocation logic.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43074/#review118603 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On Feb. 3,

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43053/#review118602 --- samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala

Re: Review Request 43053: allow warning instead of fail in case of invalid num of partitions in the checkpoint partition

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43053/#review118598 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On Feb. 5,

Re: Allow user-specified class-loader

Hi, Andy, Forgive me for my ignorance on the topic of Clojure. Could you give some simple example that Config and Serde Factories are in "Clojure-land" and how would a customized ClassLoader help in this case? Thanks a lot! -Yi On Tue, Feb 9, 2016 at 10:36 PM, Andy Chambers < andy.chamb...@fund

Re: Allow user-specified class-loader

That would help but if I understand it correctly, it would still be necessary to implement things like Config and Serde Factories in Java. It would be nice to stay in "Clojure-land" to implement those too. *Andy Chambers* | Full-Stack Software Developer *andy.chamb...@fundingcircle.com * | 707-20

Re: Allow user-specified class-loader

Hi, Andy, I think that you are looking for the feature in SAMZA-697. Or are you looking for something even more specific? On Tue, Feb 9, 2016 at 10:22 PM, Andy Chambers < andy.chamb...@fundingcircle.com> wrote: > Hey Folks, > I'm trying to build some tooling to make writing jobs in Clojure a lit

Allow user-specified class-loader

Hey Folks, I'm trying to build some tooling to make writing jobs in Clojure a little bit more interactive. One feature that I think would help a lot is to allow specification of a ClassLoader that is able to find classes defined dynamically in Clojure. Would you consider a feature/patch that adds

Re: HTTP-based Elasticsearch system producer and reusable task

Hi, Roger, Got it! I would like to understand more on the SystemProducer API changes required by #1 and #2. Could you elaborate a bit more? Regarding to JDK8 required in the new HTTP-based Elasticsearch producer, I want to ask how you are motivated to go w/ JDK8. It does bring a lot more nice fea

Review Request 43404: SAMZA-851: update Hello-Samza w/ CDH tutorial documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43404/ --- Review request for samza. Bugs: SAMZA-851 https://issues.apache.org/jira/br

Re: Review Request 43350: SAMZA-867 Fix job restart/shutdown in the event of a node outage.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43350/#review118554 --- Ship it! Thanks for addressing the nits :-) Looks Great to me!

Re: Review Request 43350: SAMZA-867 Fix job restart/shutdown in the event of a node outage.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43350/ --- (Updated Feb. 10, 2016, 2:40 a.m.) Review request for samza, Navina Ramesh, Jag

Re: Review Request 43350: SAMZA-867 Fix job restart/shutdown in the event of a node outage.

> On Feb. 9, 2016, 2:18 a.m., Jagadish Venkatraman wrote: > > samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java, line > > 261 > > > > > > Great work on separating out various methods :) > >

Re: HTTP-based Elasticsearch system producer and reusable task

Hi Yi, It could be merged into the Samza project if there's enough interest but may need some re-working depending on which dependencies are ok to bring in. I did it outside of the Samza project first because I had to get it done quickly so it relies on Java 8 features, dropwizard metrics for his

Re: Compression type Snappy is not linked with the binary

Thanks, Edi. Not sure why we are running into this problem yet again. Looks like it has to be a combination of OS and java options to reproduce this, related to SAMZA-836. Can you please attach the patch that you were trying with the latest repo? it could be a different code path in the library tha

Impersonation in samza

Hello, within my job I'm attempting to connect to a phoenix DB. Unfortunately, samza is trying to connect as the yarn user. Is there anyway to enable impersonation? Or is there any workaround for this issue? -David

Re: Compression type Snappy is not linked with the binary

Hi Navina, I did indeed came across the long thread dealing with this. And yes, things were working fine after the rocksdb upgrade to 3.13.1 which I'm still using. Not sure what I changed that's causing this. I did merge latest from upstream into my repo as well as exposed some RocksDb config thou

Re: Review Request 39032: SAMZA-787: task.log4j.system should not be guessed if not configured

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39032/#review118471 --- Ship it! - Boris Shkolnik On Oct. 6, 2015, 12:44 a.m., Navina

Re: ChangeLog Question for TTL rocksDB stores

That's the reason for the warning! In my understanding it may happen. Events deleted by RocksDB TTL may be added back if restored from the changelog. Boris. On Wed, Feb 3, 2016 at 5:46 PM, David Garcia wrote: > Boris, thank you for the clarification. But just to make sure I > understand, is it

Re: Review Request 42484: hello-samza 0.10.0 changes for CDH 5.4 distribution

> On Jan. 19, 2016, 4:38 a.m., Yi Pan (Data Infrastructure) wrote: > > pom.xml, line 289 > > > > > > We have observed an issue with YARN 2.6.0 AM client that would not > > refresh the token (YARN-3103). Hence, we ha

Re: HTTP-based Elasticsearch system producer and reusable task

Hi, Roger, That's awesome! Are you planning to submit the HTTP-based system producer in Samza open-source samza-elasticsearch module? If ElasticSearch community suggest that HTTP-based clients be the recommended way, we should use it in samza-elasticsearch as well. And what's your opinion on the e

HTTP-based Elasticsearch system producer and reusable task

Hi Samza folks, For people who want to use HTTP to integrate with Elasticsearch, I wrote an HTTP-based system producer and a reusable task, including latency stats from event origin time, task processing time, and time spent talking to Elasticsearch API. https://github.com/quantiply/rico/blob/mas

Re: java.lang.ClassNotFoundException: org.codehaus.jackson.annotate.JsonClass

On Tue, Feb 9, 2016 at 12:01 PM, Yi Pan wrote: > Could you try to fix the jackson version to 1.9.13 in your build first? That fixed it! Thank you! > It seems like a incompatible jackson version issue, as you have noticed in > your classpath. It would be good if you can share your maven pom.xml a

Re: Download page seems out of date

On Mon, Feb 8, 2016 at 6:59 PM, Yi Pan wrote: > Thanks for point out the online doc problem! I messed the latest online doc > up when fixing SAMZA-782. Now it should be fixed! Please check the latest > online doc again and let us know if you see any further problems w/ it. Looks great! My pleasur

Re: java.lang.ClassNotFoundException: org.codehaus.jackson.annotate.JsonClass

Hi, Avi, Could you try to fix the jackson version to 1.9.13 in your build first? It seems like a incompatible jackson version issue, as you have noticed in your classpath. It would be good if you can share your maven pom.xml as well. Thanks! -Yi On Mon, Feb 8, 2016 at 1:12 PM, Avi Flax wrote: