Re: Check website

Yes, verified. It's working. Thanks! Cheers, Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Sat, Jan 24, 2015 at 3:58 PM, Chris Riccomini wrote: > Hey all, > > I've published Yi and Navina's changes to update the website as a TLP from > SAMZA-520 and SAMZA-521. Please have a look and veri

Re: Review Request 29993: SAMZA-503: update messages-behind-high-watermark even BrokerProxy does not fetch any messages

/BrokerProxy.scala 9daf824 samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumerMetrics.scala cf0dd22 samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala 6d01071 Diff: https://reviews.apache.org/r/29993/diff/ Testing --- Thanks, Yan

Re: Review Request 29993: SAMZA-503: update messages-behind-high-watermark even BrokerProxy does not fetch any messages

-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumerMetrics.scala cf0dd22 samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala 6d01071 Diff: https://reviews.apache.org/r/29993/diff/ Testing --- Thanks, Yan Fang

Re: [DISCUSS] SQL workflow

+1 on this. Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini wrote: > Hey all, > > Are we +1 on this? I think Jakob was the only one who was curious about it. > > Cheers, > Chris > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan wrote: > > > Hi, Jakob,

Re: Review Request 30781: SAMZA-548

st/src/main/python/samza_job_yarn_deployer.py <https://reviews.apache.org/r/30781/#comment117585> space - Yan Fang On Feb. 9, 2015, 4:08 a.m., Chris Riccomini wrote: > > --- > This is an automatically generated

Review Request 30920: SAMZA-554: Simplify serde configuration by providing default serde names

/diff/ Testing --- Thanks, Yan Fang

Re: Review Request 30920: SAMZA-554: Simplify serde configuration by providing default serde names

) - samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala ac6e24f samza-core/src/test/scala/org/apache/samza/container/TestSamzaContainer.scala 19ceeaa Diff: https://reviews.apache.org/r/30920/diff/ Testing --- Thanks, Yan Fang

Review Request 31034: SAMZA-479: Make StreamAppender pluggable for different log formats

/src/test/java/org/apache/samza/logging/log4j/TestStreamAppender.java 46e4b8c samza-log4j/src/test/java/org/apache/samza/logging/log4j/serializers/TestLoggingEventStringSerde.java PRE-CREATION Diff: https://reviews.apache.org/r/31034/diff/ Testing --- Thanks, Yan Fang

Re: Samza Install

Is this helpful? http://samza.apache.org/learn/tutorials/latest/deploy-samza-to-CDH.html Thanks, Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Tue, Feb 17, 2015 at 11:46 AM, Naveen Madhire wrote: > Hi, > > Is there a process document which shows how to install Samza on an existing > had

Re: Modeling charts

Hi Uwe, Your use case seems to me is more like a state-management case. What comes to my mind is that, 1) every time a song is played, you updates the count of this song. You do not put the map in memory, as you said, the memory could be quite large. Instead, you use Samza's build-in key-value sto

Re: Review Request 30920: SAMZA-554: Simplify serde configuration by providing default serde names

ac6e24f samza-core/src/test/scala/org/apache/samza/container/TestSamzaContainer.scala 19ceeaa Diff: https://reviews.apache.org/r/30920/diff/ Testing --- Thanks, Yan Fang

Re: Review Request 31034: SAMZA-479: Make StreamAppender pluggable for different log formats

/java/org/apache/samza/logging/log4j/TestStreamAppender.java 46e4b8c samza-log4j/src/test/java/org/apache/samza/logging/log4j/serializers/TestLoggingEventStringSerde.java PRE-CREATION Diff: https://reviews.apache.org/r/31034/diff/ Testing --- Thanks, Yan Fang

Re: Review Request 31034: SAMZA-479: Make StreamAppender pluggable for different log formats

/TestLoggingEventStringSerde.java PRE-CREATION Diff: https://reviews.apache.org/r/31034/diff/ Testing --- Thanks, Yan Fang

Re: Review Request 31034: SAMZA-479: Make StreamAppender pluggable for different log formats

/TestLoggingEventStringSerde.java PRE-CREATION Diff: https://reviews.apache.org/r/31034/diff/ Testing --- Thanks, Yan Fang

Re: Review Request 31392: SAMZA-555

oneZkCoordinatorController.java <https://reviews.apache.org/r/31392/#comment120763> this is just for draft, right? :) - Yan Fang On Feb. 25, 2015, 12:54 a.m., Chris Riccomini wrote: > > --- > This is an automatic

Re: Review Request 31520: SAMZA-579

by thread 2. * thread 2 runs, synchronized, creates the broker for topic "t1", and remove it from "droopedTopicAndPartitions". release the lock. * thread 1 goes into the "synchronized" part, it will still create broker for topic "

Re: Review Request 31520: SAMZA-579

> On Feb. 27, 2015, 7:40 p.m., Yan Fang wrote: > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala, > > line 151 > > <https://reviews.apache.org/r/31520/diff/3/?file=880457#file880457line151> > > > > 1. besides "

Re: [DISCUSS] Samza 0.9.0 release

What about SAMZA-448 ? Agreed on the release vote date. I think it is a good idea to have a release before the ApacheCon (April 16-3-17), which can help spread the words. Cheers, Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Tue, Mar 3, 2015 at 8:01 AM, Chris Riccomini wrote: > Hey all,

Re: [DISCUSS] Samza 0.9.0 release

gt; > Cheers, > Chris > > On Tue, Mar 3, 2015 at 11:26 AM, Yan Fang wrote: > > > What about SAMZA-448 ? > > > > Agreed on the release vote date. I think it is a good idea to have a > > release before the ApacheCon (April 16-3-17), which can help spread the >

Re: Review Request 31910: SAMZA-505

/CachedStore.scala <https://reviews.apache.org/r/31910/#comment123394> Is it duplicated? in "put" method, it's checked anyway. samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala <https://reviews.apache.org/r/31910/#comment123396> space - Yan Fang

Re: Review Request 32006: SAMZA-597

/Log4jSystemConfig.java <https://reviews.apache.org/r/32006/#comment123943> what is the reason of getting rid of the system serde? - Yan Fang On March 13, 2015, 12:57 a.m., Chris Riccomini wrote: > > --- > This is an automatically gener

Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

/samza/system/kafka/TestKafkaSystemProducer.scala ca10ea5 Diff: https://reviews.apache.org/r/32155/diff/ Testing --- ran unit tests and integration test Thanks, Yan Fang

Re: Review Request 32127: SAMZA-586

/StorageConfig.scala <https://reviews.apache.org/r/32127/#comment124460> is it necessary to convert to Set ? Seq has the contain method. - Yan Fang On March 17, 2015, 7:01 a.m., Chris Riccomini wrote: > > --- > This is an automatica

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

ource2, the sendFailed.get() is not run. That's why the stop is outside of the conditions. - Yan --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32155/#review76944 --

Re: Review Request 32155: SAMZA-458: Close in KafkaSystemProducer should flush all source buffers

83668dd samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestKafkaSystemProducer.scala ca10ea5 Diff: https://reviews.apache.org/r/32155/diff/ Testing --- ran unit tests and integration test Thanks, Yan Fang

Review Request 32332: SAMZA-367

/browse/SAMZA-367 Repository: samza Description --- added Timer in the metric doc Diffs - docs/learn/documentation/versioned/container/metrics.md 8ec7740 Diff: https://reviews.apache.org/r/32332/diff/ Testing --- Thanks, Yan Fang

[VOTE] Apache Samza 0.9.0 RC0

Hey all, This is a call for a vote on a release of Apache Samza 0.9.0. This is our first release as the Apache top-level project. Thanks to everyone who has contributed to this release. We are very glad to see some new contributors in this release. The release candidate can be downloaded from her

Re: [VOTE] Apache Samza 0.9.0 RC0

Yan, > > Were you able to validate the source tarball? I ran: > > $ gpg --keyserver pgpkeys.mit.edu --recv-key CAC06239EA00BA80 > gpg: requesting key EA00BA80 from hkp server pgpkeys.mit.edu > gpg: key EA00BA80: public key "Yan Fang (CODE SIGNING KEY) < > yanf...@apache.or

Re: Error running integration tests

Yes, you need to turn it on to run the integration test. Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Wed, Mar 25, 2015 at 5:40 PM, Roger Hoover wrote: > That's what I was wondering. I can turn it on if necessary. > > Sent from my iPhone > > > On Mar 25, 2015, at 5:15 PM, Chris Riccomin

Re: Review Request 32528: SAMZA-540 modified

locally. Checked the metrics. Thanks, Yan Fang

Review Request 32528: SAMZA-540 modified

/SamzaContainer.scala 5416dd6 samza-core/src/test/scala/org/apache/samza/checkpoint/TestOffsetManager.scala d18d4c4 Diff: https://reviews.apache.org/r/32528/diff/ Testing --- tested locally. Checked the metrics. Thanks, Yan Fang

Re: Samza questions

I guess you mean "Martin", not "Matrin", here is the link for Ori's question. To give everyone a background. https://thestrangeloop.com/sessions/turning-the-database-inside-out-with-apache-samza Fang, Yan yanfang...@gmail.com +1 (206) 849-4108 On Thu, Mar 26, 2015 at 3:15 AM, Ori Cohen wrote:

Re: [VOTE] Apache Samza 0.9.0 RC0

> > > > > I'm running validations and some burn-in. I'll post my vote tomorrow. > > > > > > It's been pretty quiet. It'd be good to get other > > committers/non-committers > > > to do validation as well. > > > > > > Cheers, &g

Re: [VOTE] Apache Samza 0.9.0 RC0

I can definitely look at the release tomorrow. > > -jg > > > > > > On 27 March 2015 at 16:08, Yan Fang wrote: > > > Hi guys, > > > > > > It has been 72 hours. We got +1 from Yi Pan. Do we extend the voting to > > > this weekend ? > &g

Re: Cannot connect simple Samza task to Kafka or diagnose the problem

Hi Andrew, The first thought I have is that, the container keeps failing due to some exceptions. Could you check all the AM and containers run successfully? You can see the logs in $HADOOP_Home/logs/userlogs Thanks, Fang, Yan yanfang...@gmail.com On Mon, Mar 30, 2015 at 4:05 PM, Andrew Sannier

Re: [VOTE] Apache Samza 0.9.0 RC0

> >> > > > >> > Good work, Yan. > > >> > -jg > > >> > > > >> > > > >> > On 29 March 2015 at 13:08, Chinmay Soman > > > >> > wrote: > > >> > > +1 > > >> > > &g

Re: [VOTE] Apache Samza 0.9.0 RC0

gt; wrote: > > > Hey Yan, > > > > > > Let's confirm with Jakob. I *think* we don't need any intervention from > > > Apache. We should be able to move forward with the release. @Jakob, can > > you > > > confirm this? > > > > &

Re: [VOTE] Apache Samza 0.9.0 RC0

Hi guys, Have updated website for 0.9 release. Feel free to check it. Only one thing remaining - waiting for the blog account. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Mar 31, 2015 at 8:44 PM, Roger Hoover wrote: > Nice. Thanks Yan! > > On Tue, Mar 31, 2015 at 3:24 PM, Yan Fa

0.9.0 Release

Hi guys, We have released the 0.9.0 version. Websites, hello-samza and documents have been upgraded accordingly. Feel free to try/upgrade. The blog that summaries this release is posted here https://blogs.apache.org/samza/ Thank everyone for your support and hard work. Cheers, Fang, Yan yanfang.

Re: consistency between input, output and changelog streams

Hi Bart, In terms of your assumption, * Ts <= To , this is correction. The code backups this assumption is here: in RunLoop , the commit is called after each process and window methods

Re: consistency between input, output and changelog streams

uld be solved by storing the offset of the first message > in the state (not just the fact that we saw it), so that on replay of the > first message you can determine that, while you have seen an input from > this group, this is the replay of that first input and it should be > re-emitted t

Re: Review Request 32528: SAMZA-540 modified

--- tested locally. Checked the metrics. Thanks, Yan Fang

Re: [Google Summer of Code] Wider spectrum of data consumers/producers for Apache Samza

.apache.org/repos/private/committers/GsocLinkId.txt > > > > > > Thanks and great news to hear that you may be working together. > > Lewis > > > > On Fri, Apr 10, 2015 at 11:15 AM, Yan Fang wrote: > > > >> Hi Lewis, > >> > >> Than

Fwd: Google Summer of Code 2015 Mentor Registration

For Samza Committers who want to be the mentors. Fang, Yan yanfang...@gmail.com -- Forwarded message -- From: Ulrich Stärk Date: Fri, Mar 6, 2015 at 11:32 AM Subject: Google Summer of Code 2015 Mentor Registration To: ment...@community.apache.org Dear PMCs, I'm happy to announ

Re: How to deal with bootstrapping

Hi Jeremy, If my understanding is correct, whenever you add a new rule, you want to apply this rule to the historical data. Right? If you do not care about duplication, you can create a new task that contains existing rules and new rules. Configure bootstrap. This will apply all the rules from th

Re: Samza Unit Test Instrucations

Hi Yuanchi, There is no out-of-box unit tests provided by Samza. But there are some ways: 1) If you only want to test the logic in the Task class, normal unit tests will work. You can create a unit test that tests init(), process(), etc. 2) Create mock systems by implementing SystemAdmin, System

Re: How to deal with bootstrapping

bootstrap the new rules while > we're still processing the old rules. Do you think there is a way to do > that? > > On Wed, Apr 15, 2015 at 2:56 PM, Yan Fang wrote: > > > Hi Jeremy, > > > > If my understanding is correct, whenever you add a new rule, you want to &

Re: How to deal with bootstrapping

supported. > > Guozhang > > On Wed, Apr 15, 2015 at 5:43 PM, Yan Fang wrote: > > > Hi Jeremy, > > > > In order to reach this goal, we have to assume that the job with new > rules > > can always catch up with the one with old rules. Otherwise, I think we do

Re: How to deal with bootstrapping

w and old rules can catch up, as Yan pointed out. If answers > to > > > both > > > > questions above are no (which I doubt since you need to build-up > > certain > > > > "history" for the new rule before you can apply it to later > message

Re: How to deal with bootstrapping

; > > > > > > > I want to ask a few more questions regarding to your > > requirements: > > > > > > > > > > > > > > Q.1) Is it required that for a new rule, the bootstrap > processing > > > of > > > > > >

Re: How to pass arguments to a Samza job

Hi Jeremy, Currently Samza does not support accepting from command. After SAMZA-655 and SAMZA-348 , you will have other two options - environment variable and stream. *Then How to pass argument fro

Re: How to pass arguments to a Samza job

Hi Jeremy, Naveen is correct. You can also pass the argument through "--config". I overlooked it. Sorry for the confusion. Cheers, Fang, Yan yanfang...@gmail.com On Tue, Apr 21, 2015 at 2:10 PM, Naveen S wrote: > Hey Jeremy, >Sorry, it's probably because it's not documente

Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

/java/org/apache/samza/config/TestLog4jSystemConfig.java 6314a3e samza-shell/src/main/bash/state-storage-tool.sh PRE-CREATION Diff: https://reviews.apache.org/r/33419/diff/ Testing --- tested with multiple partitions and multiple stores recovery. Thanks, Yan Fang

Re: pleas add me to the email list

Hi Naveen, Send email to dev-subscr...@samza.apache.org for subscribing. Thanks, Fang, Yan yanfang...@gmail.com On Thu, Apr 23, 2015 at 5:42 AM, Naveen k wrote: > Hi All, > > Am very much interested to be part of Samza project. Please add me to the > mail list. > > Regards, > Naveen K >

Re: pleas add me to the email list

r < asifsaeedakh...@googlemail.com> wrote: > Hi guys, > > I have some time to contribute to the project. Except am still relatively > new to Scala! Any Java APIs i can write ? Most JIRAs seem Scala centric... > > Thanks > Asif > On 23 Apr 2015 19:44, "Yan Fang"

Re: Review Request 33409: SAMZA-433 - Rename "task" to "container" in AM and container

a/job/yarn/SamzaAppMasterTaskManager.scala <https://reviews.apache.org/r/33409/#comment131807> change accordingly samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMasterTaskManager.scala <https://reviews.apache.org/r/33409/#comment131808> using c

Re: Review Request 33503: SAMZA-604 - Add support for using gradle to build/run hello-samza tasks.

use startGrid here to be consistent with "stopGrid" ? build.gradle <https://reviews.apache.org/r/33503/#comment131892> name to listKafkaTopics? gradle/wrapper/gradle-wrapper.properties <https://reviews.apache.org/r/33503/#comment131895> downloading binary is s

Re: consistency between input, output and changelog streams

not solved this limitation. So I think you have to deal with this situation in the processing side. Thanks, Fang, Yan yanfang...@gmail.com On Wed, Apr 8, 2015 at 11:05 AM, Yan Fang wrote: > -- Hi Bart, > > " If KV values are immediately committed to the changelog when you do

Re: Edit Permission on Confluence Pages

Hi Guozhang, Checked the Confluence. you should have the edit permission. Thanks, Fang, Yan yanfang...@gmail.com On Mon, Apr 27, 2015 at 2:16 PM, Guozhang Wang wrote: > Hi, > > Can some committer grant me the edit permission on wiki pages? > > -- Guozhang >

Re: Review Request 33674: Fix SAMZA-660 - Default serdes do not work for changelog streams.

/SamzaContainer.scala <https://reviews.apache.org/r/33674/#comment132539> This "config" is not used anymore. remove it? - Yan Fang On April 29, 2015, 1:17 p.m., Tommy Becker wrote: > > --- > This is an automatically g

Re: Review Request 32147: SAMZA-465

using. samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala <https://reviews.apache.org/r/32147/#comment132576> Will the old config be overwritten when we produce the new config? samza-core/src/main/scala/org/apache/samza/util/Util.scala <https://reviews.apache.org/r/32147/

Re: Review Request 32147: SAMZA-465

> On April 29, 2015, 7:30 p.m., Yan Fang wrote: > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamSystemConsumer.java, > > line 149 > > <https://reviews.apache.org/r/32147/diff/7/?file=922534#file922534line149> > > > > W

Re: Review Request 32147: SAMZA-465

> On April 29, 2015, 8:38 p.m., Naveen Somasundaram wrote: > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamMessage.java, > > line 448 > > > > > > This is consistent with how it was pr

Re: Review Request 32147: SAMZA-465

> On April 29, 2015, 7:30 p.m., Yan Fang wrote: > > samza-core/src/main/java/org/apache/samza/storage/ChangelogPartitionManager.java, > > lines 60-68 > > <https://reviews.apache.org/r/32147/diff/7/?file=922539#file922539line60> > > > > actu

Re: Errors and hung job on broker shutdown

Not sure about the Kafka side. From the Samza side, from your description ( "does not exit nor does it make any progress" ), I think the code is stuck in producer.close

Re: Review Request 32147: SAMZA-465

> On April 29, 2015, 7:30 p.m., Yan Fang wrote: > > samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamSystemConsumer.java, > > line 149 > > <https://reviews.apache.org/r/32147/diff/7/?file=922534#file922534line149> > > > > W

Re: Review Request 33735: RocksDB TTL support

throw Exception when the value is not int? 2. add warning/info messages for this setting. Because if this is set too low accidently, at least we have a place to check. - Yan Fang On April 30, 2015, 11:21 p.m., Naveen Somasundaram wrote: > > --

Re: Looking for Samza Support/Consulting

Hi Percy, If you have some Samza specific questions, can post to this mailing list. We would try to help. Thanks, Fang, Yan yanfang...@gmail.com On Sat, May 2, 2015 at 9:40 AM, Percy Wegmann wrote: > Hello, > > Apologies if this isn’t the right list, but I’m looking for support and > consulti

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

the Factory classes? Then it will be eaiser to manager -- we know how many configs needed by elastic, what is set, what is read, etc. Also The ElasticConfig should only be responsible for reading the config and we create factories in other classes, such as in the Producer class, if needed. Wh

Re: Input stream partition to task mapping

Hi Ueli, This feature currently is not supported by Samza. There was some discussions in the JIRA - SAMZA-353 . But there are some workaround for this, depends on what you want to achieve. If you can specify what your requirement is, we can help th

Re: Input stream partition to task mapping

it with the data stream. > > If I understand it correctly the only viable solution at the moment is to > create a new stream for the rules messages with as many partitions as the > data stream and write each rules update message to all partitions of the > new rules stream. > > C

Re: Local state in Samza - sharing data between tasks

Hi Andreas, Not quite understand this part "Because the messages coming into the input stream are random (i.e. can hit any partition and therefore any task), each task will need its own copy of the data (i.e. the data needs to be duplicated across each task)." Messages come into the input stream

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

/TestLog4jSystemConfig.java 6314a3e samza-shell/src/main/bash/state-storage-tool.sh PRE-CREATION Diff: https://reviews.apache.org/r/33419/diff/ Testing --- tested with multiple partitions and multiple stores recovery. Thanks, Yan Fang

Re: Input stream partition to task mapping

If the rule does not change, we can get the rule when starting up StreamTasks and then localize it. Cheers, Fang, Yan yanfang...@gmail.com On Tue, May 5, 2015 at 2:41 PM, Yan Fang wrote: > "If I understand it correctly the only viable solution at the moment is to > create a new str

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

5> why do we need 2 storeDir? Is it because we want to delete the default state Dir? samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala <https://reviews.apache.org/r/33453/#comment133371> personal opinion: should this logic be another method, such as rea

Re: Review Request 33453: SAMZA-557 Reuse local state in SamzaContainer on clean shutdown

> On May 6, 2015, 12:10 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala, > > lines 481-486 > > <https://reviews.apache.org/r/33453/diff/4/?file=950515#file950515line481> > > > > 1. why does the cha

Re: Review Request 33735: RocksDB TTL support

> On May 1, 2015, 6:27 a.m., Yan Fang wrote: > > docs/learn/documentation/versioned/jobs/configuration-table.html, lines > > 1009-1010 > > <https://reviews.apache.org/r/33735/diff/1/?file=946535#file946535line1009> > > > > why do we remove the bloo

Re: Review Request 33735: RocksDB TTL support

> On May 7, 2015, 8:07 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala, > > line 94 > > > > > > It would be nicer to try catch the ex

Re: Input stream partition to task mapping

container. Is it because there is only one offset value for the > container for the Rules stream? > > Can you please expand on what you mean by "get the rule when starting > up StreamTasks > and then localize it."? Do you mean, loading messages into a changelog > stream us

Re: Review Request 34011: Add support for a Graphite Metrics Reporter

samza.apache.org/learn/documentation/0.9/jobs/configuration-table.html , that will be great. Thanks! - Yan Fang On May 11, 2015, 10:10 a.m., Luis De Pombo wrote: > > --- > This is an automatically generated e-mail. To r

Re: Access config properties inside Job

Hi Elango, Currently Samza does not support having two config files, like --classpath-config-file. But there are a few workarounds: 1. you already mentioned putting the properties in the samza properties file 2. use key=value pair. so it would look like deploy/samza/bin/run-job.sh --config-facto

Re: Yarn redundancy

Is the HA set correctly? The log looks like it's in the YARN setting side. Fang, Yan yanfang...@gmail.com On Thu, May 14, 2015 at 12:29 PM, Shekar Tippur wrote: > Other observation I forgot to mention is that if I kill the rm and nm > process, samza job seem to run properly. Only when 01 server

Re: Log rotation on Samza/yarn logs

Hi Shekar, Currently by default, the log4j is using DailyRollingFileAppender. You can change the log4j.xml to config as you want. Usually, daily base is good enough. Thanks, Fang, Yan yanfang...@gmail.com On Thu, May 14, 2015 at 10:56 AM, Shekar Tippur wrote: > Hello, > > How are the Samza/ya

Re: Yarn redundancy

adoop/mapreduce/*,/app/hadoop/hadoop-2.5.2/share/hadoop/mapreduce/lib/*,/app/hadoop/hadoop-2.5.2/share/hadoop/yarn/*,/app/hadoop/hadoop-2.5.2/share/hadoop/yarn/lib/* > > > > > > > > > 2. scp'd the config to the slave resource manager node > > 3. restart

Re: Review Request 33419: SAMZA-625: Provide tool to consume changelog and materialize a state store

://reviews.apache.org/r/33419/diff/ Testing --- tested with multiple partitions and multiple stores recovery. Thanks, Yan Fang

Re: Yarn jobs in accepted state

Hi Shekar, This website requires signed in. Could you past to another more open place ? There are a lot if you just google it. Sorry for being a little picky. Also, when you see a lot of jobs in the accepted state, what does the log say? Thanks, Fang, Yan yanfang...@gmail.com On Wed, May 20, 2

Do we want to release the 0.9.1 now?

Hi guys, Just ask, are there any other bugs that we want to back port to 0.9.1 besides SAMZA-662 ? If no, I think we can prepare the 0.9.1 release and ask for the vote. Cheers, Fang, Yan yanfang...@gmail.com

Re: Do we want to release the 0.9.1 now?

> > -Yi > > On Thu, May 21, 2015 at 2:21 PM, Yan Fang wrote: > > > Hi guys, > > > > Just ask, are there any other bugs that we want to back port to 0.9.1 > > besides SAMZA-662 <https://issues.apache.org/jira/browse/SAMZA-662> ? > > >

Re: Samza YarnJobFactory support for https

Hi José, Thank you. If you can contribute a patch for this fix (SAMZA-688 ), it would be very helpful. And here is the guide for contributing. Cheers, Fang, Yan yanfang...@

Re: Do we want to release the 0.9.1 now?

release version. bump up to > > 0.10.0-SNAPSHOT (8 weeks ago) > > > > Of these, SAMZA-658 (fix cached store iterator remove() function), > > SAMZA-608 (don't hange on serde errors in system consumers) and > > SAMZA-616 (make shutdown hook wait for container to f

Re: Review Request 34564: SAMZA-401: add utilization metrics for the event loop

apache/samza/container/SamzaContainerMetrics.scala <https://reviews.apache.org/r/34564/#comment136334> do we want to create a new Gauge every loop? I think one Gauge is sufficient, we can update its value. What do you think? - Yan Fang On May 21, 2015, 8:53 p.m., Luis De

Re: Not able to consume produced data

Hi Renato, There maybe a misunderstanding in the concept. Consumer is to feed the msgs into the Samza, while Producer is to send the msg from Samza to other systems. So if you implement the Consumer, should be able to see the msgs in the StreamTask. That's why you confuse Naveen. Cheers, Fang, Y

Re: Samza producer error

Hi Davide, This sometimes is due to Kafka broker. Are you able to send the message to the brokers from the command line? Thanks, Fang, Yan yanfang...@gmail.com On Tue, May 26, 2015 at 5:49 AM, Davide Simoncelli wrote: > Hello, > > I’m testing my Samza application on a 3 nodes clusters. When t

Re: ProcessJobFactory parent process

Hi Lukas, The parent process is used to manage the lifecycle of the actual process. I am curious how much memory the parent process takes? Thanks, Fang, Yan yanfang...@gmail.com On Thu, May 28, 2015 at 2:30 PM, Lukas Steiblys wrote: > Hello, > > I’m running Samza tasks using ProcessJobFactory

Re: ProcessJobFactory parent process

ny way > I can. > > The bigger problem is when the job crashes and the parent process stays > active preventing an auto restart by the Docker supervisor. > > Lukas > > On Thursday, May 28, 2015, Yan Fang wrote: > > > Hi Lukas, > > > > The parent process

Re: Review Request 34746: Adding new CoordinatorStreamMessage "SetContainerHostMapping" and LocalityManager (SAMZA-618)

(String host) { messageMap.put("host", host); } samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala <https://reviews.apache.org/r/34746/#comment137593> catch the UnknownHostExcepti

Re: Samza wiki

Hi Renato, Grant you the privilege to create pages. Should be able to create a page to track your GSoC progress. Thanks, Fang, Yan yanfang...@gmail.com On Sun, May 31, 2015 at 12:07 PM, Renato Marroquín Mogrovejo < renatoj.marroq...@gmail.com> wrote: > Hi all, > > I would like to create a wiki

Re: Review Request 34888: SAMZA-691: messages-chosen counter without partition id

/SystemConsumersMetrics.scala <https://reviews.apache.org/r/34888/#comment137897> change to "registerSystemStreamPartition"? - Yan Fang On June 1, 2015, 1:27 p.m., József Márton Jung wrote: > > --- > This is an automatically g

Re: About Spark Streaming overview in Samza docs

-- Hi Jay, Thanks for forwarding this. -- Hi TD, Thanks for pointing this out. That overview was written about one year ago. It is out-of-date. Good to get some critiques from your side. Will update it soon. Created SAMZA-698 to track this. Thank

Re: Nothing occurs from collector.send()

Hi Garrett, I guess you run into the serde issues as you mentioned. If you can check the Samza log and show us, we will be more helpful. Also, maybe pasting the config here (if you dont mind), we can help to see if you miss something. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 2, 2015 a

  1   2   3   >