Re: Review Request 48243: SAMZA-961: Async tasks and multithreading model

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48243/#review137369 --- samza-core/src/main/java/org/apache/samza/container/RunLoopFactor

Re: Review Request 46282: SAMZA-928 document Kerberos on YARN

> On May 26, 2016, 9:37 p.m., Yi Pan (Data Infrastructure) wrote: > > docs/learn/documentation/versioned/yarn/yarn-security.md, line 66 > > > > > > nit: trailing white space Done > On May 26, 2016, 9:37 p.m., Yi

Re: Review Request 46282: SAMZA-928 document Kerberos on YARN

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46282/ --- (Updated June 15, 2016, 2:25 a.m.) Review request for samza. Repository: samz

About planner questions！

Hello everyone: Now I am learning calcite streamSQL. Samza is a stream system and uses calcite very well. I have a question about planner. Anyone can help me. Thanks in advance. The question is following: 1、 I want to know what is the difference between HepPlanner and VolcanoPlanner? For st

Re: Review Request 48356: RFC: Samza as a library

> On June 9, 2016, 7:15 p.m., Chris Pettitt wrote: > > samza-core/src/main/java/org/apache/samza/configbuilder/ConfigBuilder.java, > > lines 42-53 > > > > > > Can these be private vs. package private? > > Navina Ram

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48459/#review137624 --- Ship it! lgtm! +1 - Navina Ramesh On June 14, 2016, 11:14 p.

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

> On June 14, 2016, 8:11 p.m., Navina Ramesh wrote: > > samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala, > > line 276 > > > > > > Documentation is not clear. It says "It does not retr

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48459/ --- (Updated June 14, 2016, 11:14 p.m.) Review request for samza, Boris Shkolnik, C

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48459/ --- (Updated June 14, 2016, 11:13 p.m.) Review request for samza, Boris Shkolnik, C

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

> On June 14, 2016, 8:11 p.m., Navina Ramesh wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > line 230 > > > > > > nit: Doc can be improved by explaining why this is more e

Re: Manually Commit Offsets?

Thanks, I see no commit method in TaskContext. Unless I am missing something it is TaskCoordinator. TaskCoordinator.commit() also does not look to give me the ability to set the value of the checkpoint, just checkpoint after unwrapping the incoming message. I need to treat the messages as if th

Re: Manually Commit Offsets?

Sorry. Correction: > 2) in your code, call TaskContext.commit() whenever you are ready to > checkpoint. > > *TaskCoordinator.commit()* > > On Tue, Jun 14, 2016 at 10:16 AM, Jeremiah Adams < > jad...@helixeducation.com> wrote: > >> We need to send messages to a remote service. I need to impleme

Re: Manually Commit Offsets?

Hi, Jeremiah, Samza does support manual checkpointing. You can following the steps below: 1) turn off auto-commit by setting task.commit.ms=-1 2) in your code, call TaskContext.commit() whenever you are ready to checkpoint. We have applications in LinkedIn using this pattern to successfully imple

Re: Review Request 48459: SAMZA-964 Improve the performance of the continuous OFFSET checkpointing for logged stores

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48459/#review137418 --- samza-api/src/main/java/org/apache/samza/system/ExtendedSystemAdm

Manually Commit Offsets?

We need to send messages to a remote service. I need to implement a circuit breaker to address the scenario in which the remote system is unavailable. I need to change the current offset to reprocess the current offset while the remote system is down. These concerns are similar to those outlined

Re: Samza stops processing

Weird, I see a response http://mail-archives.apache.org/mod_mbox/samza-dev/201606.mbox/browser but I dint get a email. Anyways, Jagadish - Thanks for responding. Yes. We were able to produce a message via console producer and consume it. - Shekar On Thu, Jun 9, 2016 at 1:42 AM, Shekar Tippur wr

Re: Samza - Invalid ContainerId: container_e09_1465882867895_0003_01_000001

Looks like there is a Jira ticket with no conclusion https://issues.apache.org/jira/browse/SAMZA-750 Is there a workaround without upgrading the code? On Mon, Jun 13, 2016 at 11:25 PM, Shekar Tippur wrote: > > > Hello, > > Running Samza 8.0 with latest hadoop, I get the below exception. What d

Re: No updates to some of the store changelog partitions

Some updates below. After changing the number of contains *from 6 to 4*, we are able to see changelog offsets moving across the board. Yay! Now the question is: why? With 20 input partitions/task instances, the previous setting will allocate a different number of task instances (4) to some cont