Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36006/#review94005 --- samza-autoscaling/src/main/java/org/apache/samza/autoScaling/utils/

RE: testThreadInterruptInOperationSleep on clean installation

Hi, There was no gradle process and I have tried to run it increasing the log level. It seems that the test that is failing is the one related to Rocksdb. How can I run the test on eclipse? This is what I get: Executing task ':samza-kv-rocksdb_2.10:sourcesJar' (up-to-date check took 0.005 sec

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

> On July 25, 2015, 1:33 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala, > > line 126 > > > > > > add logs for the case where the topic is already existied. Log

Re: samza environment variable on containers

Maybe @Eli Reisman can give you some insights, since he is writing the HDFS producer. Because the exception looks like related to the hdfs consumer. Thanks, Fang, Yan yanfang...@gmail.com On Sun, Aug 2, 2015 at 12:41 PM, Chen Song wrote: > Thanks Yan. > > I ran into issues when testing jobs on

Re: testThreadInterruptInOperationSleep on clean installation

Hi Jordi, Those two exceptions seems like caused by the race condition. Since I can not reproduce it, can you try 1) kill all the GradleDaemon and GradleWrapperMain processes when you rerun the build ? 2) can you try to run those two tests in the eclipse (or some other ways) without gradle ? I dou

Re: Keep samza container logs from being deleted

Hi Chen, You can set the yarn.nodemanage.delete.debug-delay-sec config in yarn-site.xml, in order to retain the logs after the job crashes. Refer https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml (This was intended for debug purpose). If you need to retain lo

Re: Review Request 37039: SAMZA-748 Coordinator URL always 127.0.0.1

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37039/#review93971 --- samza-core/src/main/scala/org/apache/samza/util/Util.scala (line 31

Keep samza container logs from being deleted

Dump question. When running samza job on YARN, I found that container logs gone after days. Is there a YARN config to keep the logs from being deleted for long-lived streaming job? -- Chen Song

Re: Dinamically asigning output topic

I believe you only need to declare the system stream you are writing to before hand . ie your Kafka broker list if you are using Kafka. If auto create topics is enabled, you can dynamically write to any topic within a predefined system stream. I'd suggest simply use a hash map of topic to OutputStr

Review Request 37039: SAMZA-748 Coordinator URL always 127.0.0.1

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37039/ --- Review request for samza. Repository: samza Description --- We are using

Dinamically asigning output topic

Hi, Is there a way to assign dynamically in the job code the topic where the output messages should be sent to? I am testing an idea that needs to decide programmatically (and not in from limited range) the destination topic. As I am understanding it, I need to declare the output format in the

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

> On July 25, 2015, 1:33 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala, > > line 126 > > > > > > add logs for the case where the topic is already existied. Log