Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Tom Davis
ssage- From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com] Sent: Tuesday, March 20, 2018 11:36 To: Tom Davis Cc: Prateek Maheshwari ; dev@samza.apache.org; yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alternative deployment model(s) Hi Tom, H

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Thunder Stumpges
36 To: Tom Davis Cc: Prateek Maheshwari ; dev@samza.apache.org; yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alternative deployment model(s) Hi Tom, >> Happy to put something together this weekend as well. Great, can't wait!! >> What form

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Thunder Stumpges
shwari Cc: dev@samza.apache.org; t...@recursivedream.com; yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alternative deployment model(s) Hi Thunder, Thank you for the PR. Really nice work! Since, you have a working implementation on K8s, would you be willing to co

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Jagadish Venkatraman
;>> "partition" : 0, >>>> "stream" : "test_topic1" >>>> }, { >>>> "system" : "kafka", >>>> "partition" : 0, >>>> &q

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Tom Davis
k Queue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolEx ecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool Executor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Jagadish Venkatraman
;> >> >> -Original Message- >> From: Thunder Stumpges >> Sent: Monday, March 19, 2018 20:58 >> To: dev@samza.apache.org >> Cc: Jagadish Venkatraman ; t...@recursivedream.com; >> yi...@linkedin.com; Yi Pan >> Subject: RE: Old style "low le

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-20 Thread Prateek Maheshwari
> Thunder > > > -Original Message- > From: Thunder Stumpges > Sent: Monday, March 19, 2018 20:58 > To: dev@samza.apache.org > Cc: Jagadish Venkatraman ; t...@recursivedream.com; > yi...@linkedin.com; Yi Pan > Subject: RE: Old style "low level" Tasks

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-19 Thread Thunder Stumpges
; yi...@linkedin.com; Yi Pan Subject: RE: Old style "low level" Tasks with alternative deployment model(s) Well I figured it out. My specific issue was due to a simple dependency problem where I had gotten an older version of the Jackson-mapper library. However the code wa

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-19 Thread Thunder Stumpges
edWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadP

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-16 Thread Thunder Stumpges
nal Message- From: Prateek Maheshwari [mailto:prateek...@gmail.com] Sent: Friday, March 16, 2018 17:02 To: dev@samza.apache.org Cc: Jagadish Venkatraman ; t...@recursivedream.com; yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alternative deployment model(s) Hi Thunder,

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-16 Thread Prateek Maheshwari
+--- org.codehaus.jackson:jackson-mapper-asl:1.9.13 > ||||\--- org.codehaus.jackson:jackson-core-asl:1.9.13 > > Thanks! > Thunder > > -Original Message- > From: Thunder Stumpges [mailto:tstump...@ntent.com] > Sent: Friday, March 16, 2018 15:29 > To:

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-16 Thread Thunder Stumpges
l:1.9.13 Thanks! Thunder -Original Message- From: Thunder Stumpges [mailto:tstump...@ntent.com] Sent: Friday, March 16, 2018 15:29 To: dev@samza.apache.org; Jagadish Venkatraman Cc: t...@recursivedream.com; yi...@linkedin.com; Yi Pan Subject: RE: Old style "low level"

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-16 Thread Thunder Stumpges
e more debug logging and try again. Thanks for any guidance you all might have. -Thunder -Original Message- From: Thunder Stumpges [mailto:tstump...@ntent.com] Sent: Friday, March 16, 2018 14:43 To: dev@samza.apache.org; Jagadish Venkatraman Cc: t...@recursivedream.com; yi...@linkedin.com; Yi

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-16 Thread Thunder Stumpges
stump...@ntent.com] Sent: Thursday, March 15, 2018 16:35 To: Jagadish Venkatraman Cc: dev@samza.apache.org; t...@recursivedream.com; yi...@linkedin.com; Yi Pan Subject: RE: Old style "low level" Tasks with alternative deployment model(s) Thanks a lot for the info. I have something basic

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-15 Thread Thunder Stumpges
e.org; t...@recursivedream.com; yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alternative deployment model(s) >> Thanks for the info on the tradeoffs. That makes a lot of sense. I am >> on-board with using ZkJobCoordinator, sounds like some good

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-15 Thread Jagadish Venkatraman
wn answers to my questions, but figured I should go > ahead and ask now anyway. Thanks! > > > > -Thunder > > > > > > *From:* Jagadish Venkatraman [mailto:jagadish1...@gmail.com] > *Sent:* Thursday, March 15, 2018 1:41 > *To:* dev@samza.apache.org; Thunder St

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-15 Thread Thunder Stumpges
ad and ask now anyway. Thanks! -Thunder From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com] Sent: Thursday, March 15, 2018 1:41 To: dev@samza.apache.org; Thunder Stumpges ; t...@recursivedream.com Cc: yi...@linkedin.com; Yi Pan Subject: Re: Old style "low level" Tasks with alter

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-15 Thread Jagadish Venkatraman
r comes with other nice tools for monitoring offsets, > lag, etc > > Thanks guys! > -Thunder > > -Original Message- > From: Tom Davis [mailto:t...@recursivedream.com] > Sent: Wednesday, March 14, 2018 17:50 > To: dev@samza.apache.org > Subject: Re: Old sty

RE: Old style "low level" Tasks with alternative deployment model(s)

2018-03-14 Thread Thunder Stumpges
tools for monitoring offsets, lag, etc Thanks guys! -Thunder -Original Message- From: Tom Davis [mailto:t...@recursivedream.com] Sent: Wednesday, March 14, 2018 17:50 To: dev@samza.apache.org Subject: Re: Old style "low level" Tasks with alternative deployment model(s)

Re: Old style "low level" Tasks with alternative deployment model(s)

2018-03-14 Thread Tom Davis
Hey there! You are correct that this is focused on the higher-level API but doesn't preclude using the lower-level API. I was at the same point you were not long ago, in fact, and had a very productive conversation on the list: you should look for "Question about custom StreamJob/Factory" in the

Old style "low level" Tasks with alternative deployment model(s)

2018-03-14 Thread Thunder Stumpges
Hi all, We are using Samza (0.12.0) in about 2 dozen jobs implementing several processing pipelines. We have also begun a significant move of other services within our company to Docker/Kubernetes. Right now our Hadoop/Yarn cluster has a mix of stream and batch "Map Reduce" jobs (many reporting