Running Samza programmatically

2016-03-09 Thread Robert Zuljevic
Hi all, We would like to write and run integration tests for our own Samza Tasks/Jobs, but without using external tools like Zoopkio, but rather as a part of the build process. What we would like to do is start Yarn/Zookeeper/Kafka locally (or in a docker container) and then submit jobs program

Re: Review Request 36767: SAMZA-423: Add document store to Samza

2015-09-18 Thread Robert Zuljevic
/store/TestMetricDocumentStore.java PRE-CREATION settings.gradle 4c1aa10 Diff: https://reviews.apache.org/r/36767/diff/ Testing --- Thanks, Robert Zuljevic

[SAMZA-423] Integrate Lucene into Samza

2015-08-13 Thread Robert Zuljevic
Hello all, I've added a design document to task SAMZA-423. If anyone is interested please take a look : ) I have some outstanding questions that I would like to resolve so I could continue working on this as soon as possible: 1. Should the API support bulk indexing/removing/matching? (n

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-10 Thread Robert Zuljevic
ld be greatly appreaciated! I tested the change with the unit/integration tests already available. Thanks, Robert Zuljevic

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-10 Thread Robert Zuljevic
nters would be greatly appreaciated! I tested the change with the unit/integration tests already available. Thanks, Robert Zuljevic

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-05 Thread Robert Zuljevic
h the unit/integration tests already available. Thanks, Robert Zuljevic

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-05 Thread Robert Zuljevic
- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/#review93093 ------- On July 9, 2015, 2:39 p.m., Robert Zuljevic wrote: > > ---

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-08-03 Thread Robert Zuljevic
n JobCoordinator, right after cration? - Robert --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/#review93019 ---

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-29 Thread Robert Zuljevic
rt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/#review93019 --- On July 9, 2015, 2:39 p.m., Robert Zuljevi

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-29 Thread Robert Zuljevic
temStream.getStream, changeLogPartitions) } This is the only way I could thought of for simplifing this. I don't think what you posted would work, because you're using String's map function, but it did steer me in the right direction. Do you agree? - Robert Zuljevic

Review Request 36767: SAMZA-423: Integrate Lucene into Samza

2015-07-24 Thread Robert Zuljevic
19bff971ad221084dac10d3f7f3facfa42b829a7 Diff: https://reviews.apache.org/r/36767/diff/ Testing --- Thanks, Robert Zuljevic

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-09 Thread Robert Zuljevic
here, so any pointers would be greatly appreaciated! I tested the change with the unit/integration tests already available. Thanks, Robert Zuljevic

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-03 Thread Robert Zuljevic
test / integration test) I should make here, so any pointers would be greatly appreaciated! I tested the change with the unit/integration tests already available. Thanks, Robert Zuljevic

Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-03 Thread Robert Zuljevic
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterTaskManager.scala 1e936b42a5b9a4bfb43766c17847b2947ebdb21d Diff: https://reviews.apache.org/r/36163/diff/ Testing --- Thanks, Robert Zuljevic

RE: [SAMZA-690] Changelog topic creation should not be in the container code

2015-06-26 Thread Robert Zuljevic
:40 AM, Robert Zuljevic wrote: > 1. Is checkpoint topic referred to in the description coordinator > stream/topic? > In the master branch, checkpoint topic is deprecated (except for migration purpose). The checkpoints will be sent to coordinator stream. Hence, there is no need

[SAMZA-690] Changelog topic creation should not be in the container code

2015-06-25 Thread Robert Zuljevic
Hello all, Description of SAMZA-690 (https://issues.apache.org/jira/browse/SAMZA-690) ticket says that creation of changelog and checkpoint topics should be moved from SamzaContainer class to JobRunner/JobCoordinator class. I investigated for a while, but seem to have hit a wall and require som

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-16 Thread Robert Zuljevic
95f76b61f70058bb567fe4eff3a3bb2298a0e197 samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e Diff: https://reviews.apache.org/r/35067/diff/ Testing --- Thanks, Robert Zuljevic

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-15 Thread Robert Zuljevic
82006960adc45a92a79e28f078ebd5eb01d3205e Diff: https://reviews.apache.org/r/35067/diff/ Testing --- Thanks, Robert Zuljevic

Re: Review Request 35067: SAMZA-546: Add config option max.in.flight.requests.per.connection

2015-06-15 Thread Robert Zuljevic
throw an NumberFormatException. Is this ok? - Robert --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35067/#review86980 --- On June 4, 2015, 1:50 p.m., Robert Zuljevic wrote: > >