Re: Number of Kafka producers

Hi David, Have you tried increasing the number of containers? Our benchmarks show about 1M messages per sec processed on one machine. You should be able to scale linearly by increasing number of containers. All relevant Kafka producer properties like batch size, compression and linger can be passe

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

> On July 27, 2016, 9:36 p.m., Navina Ramesh wrote: > > I think it will be simpler if we can just provide architectural overview in > > the website and add a separate tutorial to run it with samza-hello-samza. > > Thanks! So, do you suggest moving the deployment steps to the tutorial or duplic

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

> On July 27, 2016, 9:37 p.m., Navina Ramesh wrote: > > docs/learn/documentation/versioned/rest/overview.md, line 104 > > > > > > Is there is no default log4j in the generate tar ball, it will good to > > move this

Re: HdfsWriter opens a Bucket for every new file

@Thees, Could you open a JIRA to track this issue? And could you also describe the issue in more specific details in the JIRA? e.g. when you mentioned that "HdfsWriter opens a Bucket for every new file", do you mean that HDFSWriter will open a new file everytime a new event is sent via HdfsSystemP

Re: Review Request 50174: SAMZA-977: User doc for samza multithreading

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50174/#review143847 --- docs/learn/documentation/versioned/api/overview.md (line 22)

Number of Kafka producers

Is there a way to control the number of producers? Our Samza job writes a lot of data to the downstream Kafka topic. I was wondering if there is a way to optimize concurrency by creating more async producers. Thanks, David

Re: Review Request 50174: SAMZA-977: User doc for samza multithreading

> On July 20, 2016, 7:06 p.m., Chris Pettitt wrote: > > docs/learn/documentation/versioned/container/event-loop.md, line 43 > > > > > > s/in the a single thread/in a single thread/. > > > > A few other minor

Re: Review Request 50174: SAMZA-977: User doc for samza multithreading

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50174/ --- (Updated July 27, 2016, 11:05 p.m.) Review request for samza, Chris Pettitt, Na

Re: Review Request 50527: SAMZA-970: fix integration tests

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50527/#review143839 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On July 27,

Re: SamzaSQL document required

Thanks Yi, that was a very helpful overview! > On Jul 27, 2016, at 12:27 PM, Yi Pan wrote: > > Hi, Ankita, > > There is no official release documentation for SamzaSQL yet. If you are > referring to the paper in HPBDC this year by Milinda, it is based on > several patches under development. I wi

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50154/#review143826 --- docs/learn/documentation/versioned/rest/overview.md (line 104)

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50154/#review143777 --- I think it will be simpler if we can just provide architectural ov

HdfsWriter opens a Bucket for every new file

Hello, using the HdfsWriter provided by samza package samza-hdfs we try to write events to HDFS. After the newest patches in version 0.10.1 the Bug regarding closing files was fixed. But the Bucketer does not seem to work with any of the given HdfsWriter implementations. Every new event sent to th

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50154/ --- (Updated July 27, 2016, 8:47 p.m.) Review request for samza, Boris Shkolnik, Ch

Re: SamzaSQL document required

Hi, Ankita, There is no official release documentation for SamzaSQL yet. If you are referring to the paper in HPBDC this year by Milinda, it is based on several patches under development. I will start by listing the relevant JIRAs: - SAMZA-390: the over-arching ticket describing the view of SQL on

Review Request 50527: SAMZA-970: fix integration tests

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50527/ --- Review request for samza and Navina Ramesh. Repository: samza Description ---

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

> On July 27, 2016, 12:52 a.m., Navina Ramesh wrote: > > bin/generate-javadocs.sh, line 22 > > > > > > Ok. You are almost there :) > > > > This will overwrite the index files that you copied over for samza-a

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50154/ --- (Updated July 27, 2016, 6:12 p.m.) Review request for samza, Boris Shkolnik, Ch

Re: Review Request 50154: SAMZA-976 Samza REST Documentation

> On July 27, 2016, 12:52 a.m., Navina Ramesh wrote: > > bin/generate-javadocs.sh, line 22 > > > > > > Ok. You are almost there :) > > > > This will overwrite the index files that you copied over for samza-a

Re: 0.10.1 Release

Yes, it is undergoing. We were blocked by the integration tests ( https://issues.apache.org/jira/browse/SAMZA-970) and other issues for a while. Now we got a fix for the tests and I will submit a patch today. The rest of release should be quick. I will sync with Navina with an ETA. Sorry about the

Re: 0.10.1 Release

We're going on 3 months here, is a 0.10.1 still planned? On 05/12/2016 01:53 PM, Yi Pan wrote: Hi, Andy, We are doing some pre-release work at this moment. My rough estimation on 0.10.1 timeline would be about 1 month away. Thanks a lot! -Yi On Thu, May 12, 2016 at 9:56 AM, Andy Throgmorton