Re: Regarding my use case to explore with Samza-

2016-03-01 Thread Manohar Reddy
Hi Ramesh&Jagadesh, You are very clear about my use case .Please find my inline comments for your query on my use case. We are very thanks about your inputs on my use case and definitely we will consider your adapter concept to do a small batch jdbc calls instead of each and every event. Pl

Re: Regarding my use case to explore with Samza-

2016-03-01 Thread Navina Ramesh
Hi Manohar, On a side note regarding your use-case, I have a question. After consuming the DML changes from the kafka topic, why do you have to query back? Are you trying to decorate the event or perform some kind of join? The point I am trying to make is that if you perform a remote lookup with e

Re: Regarding my use case to explore with Samza-

2016-03-01 Thread Jagadish Venkatraman
Please take a look at the hello-world example. You can implement your business logic in the process() callback. What kind of transformation are you doing? Are you doing a group by/count style aggregation to generate the report? If so, you could use the embedded rocksdb store in Samza and potential

Re: Review Request 43732: Implemented AvroDataFileHdfsWriter

2016-03-01 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43732/#review121594 --- Overall lgtm. There is some problem in the last file in the diff u

Regarding my use case to explore with Samza-

2016-03-01 Thread Manohar Reddy
Hello Team, we are part of some service based company and trying to explore the available real time streaming technologies.so one of the first option we are trying is Samza. let me explain brief about my use case here: we are trying to build real time reporting dashboard for e-learning domain.

Re: Review Request 44241: SAMZA-883 Improve logging for container handling and kafka refresh

2016-03-01 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44241/ --- (Updated March 2, 2016, 2:24 a.m.) Review request for samza, Jagadish Venkatram

Re: Review Request 44241: SAMZA-883 Improve logging for container handling and kafka refresh

2016-03-01 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44241/ --- (Updated March 2, 2016, 2:16 a.m.) Review request for samza, Jagadish Venkatram

Re: checkpoint example?

2016-03-01 Thread Boris Shkolnik
To add to Jacob's and Jagadish's answers. If you want to to read from 24 hours before (not from the beginning or the end of the stream) you can set the checkpoint interval (see Jagadish's comment) to 24 hours. It is kind of unusual, but should work :). On Tue, Mar 1, 2016 at 4:32 PM, Jacob Maes w

CISCO AND AVAYA IP Phones

2016-03-01 Thread Laison Computech Inc
Hi, Clean tested working pulls CPUs and QTYs in stock. 115 X X5650 65 X E5410 75 X X5660 145 X E5530 100 X E5645 40 X X5680 75 X X5690 Brand new sealed IP phones and QTYs in stock. 55 x CP-7937G 77 x CP-7942G 54 x CP-7945G 75 x CP-7962G .. 45 x Avaya 9630 65 x Avaya 9641 55 x Avaya 9640 U

Review Request 44241: SAMZA-883 Improve logging for container handling and kafka refresh

2016-03-01 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44241/ --- Review request for samza, Jagadish Venkatraman and Yi Pan (Data Infrastructure).

Re: checkpoint example?

2016-03-01 Thread Jacob Maes
A couple notes that may be helpful: 1. When you have a stateful processor that dies, the changelog is the default means by which the state is restored. Change logging is enabled with this config: stores.store-name.changelog 2. If, when the job comes back up, it needs to reprocess historical messa

Re: checkpoint example?

2016-03-01 Thread Jagadish Venkatraman
Users need not worry about checkpointing. Samza will automatically commit offsets every 60s. You can choose to commit more often by either 1. Setting task.commit.ms to a smaller value (or) 2. Doing manual commit yourself by setting task.commit.ms = -1. and calling taskCoordinator.commit(); I'm cur

Re: checkpoint example?

2016-03-01 Thread Jeff Ramin
On 03/01/2016 03:10 PM, Jagadish Venkatraman wrote: You don't have to implement any state checkpoint. Samza automatically checkpoints state for you. When you recover from a failure/restart you will resume processing from the previous checkpoint. So, it's merely a configuration issue? What'

Re: checkpoint example?

2016-03-01 Thread Jagadish Venkatraman
You don't have to implement any state checkpoint. Samza automatically checkpoints state for you. When you recover from a failure/restart you will resume processing from the previous checkpoint. What's your usecase? On Tue, Mar 1, 2016 at 12:57 PM, Jeff Ramin wrote: > > Hi folks. > > Is there an

checkpoint example?

2016-03-01 Thread Jeff Ramin
Hi folks. Is there an example or tutorial for implementing a state checkpoint?: http://samza.apache.org/learn/documentation/0.10/container/checkpointing.html Or should I just start reading the documentation? Thanks. -- Jeff Ramin Software Engineer Singlewire Software 2601 W Beltline Hwy #510

[DISCUSS] Re-thinking the Samza Job Coordinator

2016-03-01 Thread Jagadish Venkatraman
Hi all, Currently, the only way to run distributed Samza containers is using Yarn. There have been requests to run Samza with other resource managers like Mesos, to run Samza as an embedded library, and to run in Docker containers managed by ECS/Kubernetes. I created SAMZA-881 for this proposing

Re: problem w/ hello samza

2016-03-01 Thread Jagadish Venkatraman
Perfect. Glad to see the problem solved! :-) On Tue, Mar 1, 2016 at 7:19 AM, Jeff Ramin wrote: > > I'm running the most recent version of linux mint, and maven 3.0.5. > > I opted for option 3 below, which corrected the problem. Thank you! > > > On 02/29/2016 04:53 PM, Jagadish Venkatraman wrote:

Re: problem w/ hello samza

2016-03-01 Thread Jeff Ramin
I'm running the most recent version of linux mint, and maven 3.0.5. I opted for option 3 below, which corrected the problem. Thank you! On 02/29/2016 04:53 PM, Jagadish Venkatraman wrote: The problem seems to be a ClassNotFound error for a class in Apache commons. There appears to be somethin