Re: Review for SAMZA-1012

Thanks! On 10/04/2016 11:18 AM, Jacob Maes wrote: I'm on it! On Tue, Oct 4, 2016 at 6:22 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote: Could I bother a committer for a review of this patch? https://reviews.apache.org/r/51689/ -- Tommy Becker Senior Software Engineer Di

Review for SAMZA-1012

Could I bother a committer for a review of this patch? https://reviews.apache.org/r/51689/ -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...

Review Request 52030: Fix SAMZA-1018

: https://reviews.apache.org/r/52030/diff/ Testing --- Tested manually Thanks, Tommy Becker

Re: Issue with consuming non-existent topics in 0.10.1

JIRA just reduces the number of calls we make to the broker. Another question, are you able to reproduce this issue ? Thanks! Navina On Wed, Sep 14, 2016 at 1:33 PM, Tommy Becker <mailto:tobec...@tivo.com> wrote: Thanks for the response, and done. https://issues.apache.org/jira/browse/SAMZ

Re: Issue with consuming non-existent topics in 0.10.1

Thanks for the response, and done. https://issues.apache.org/jira/browse/SAMZA-1018 On 09/14/2016 01:14 PM, Yi Pan wrote: Hi, Tommy, Could you open a JIRA for this one? Also, could you include the Kafka broker version in this test? Thanks! -Yi On Wed, Sep 14, 2016 at 6:06 AM, Tommy Becker

Re: SIGSEGV in RocksDB when killing jobs

sDB store close() explicitly? As you see that SamzaContainer#shutdownStores already calling flush() and close() automatically. Does it work for you if you remove the explicit store close() calls in your CloseableTask implementation? Thanks! -Yi On Wed, Sep 14, 2016 at 7:56 AM, Tommy Becker <mailto

SIGSEGV in RocksDB when killing jobs

if a patch is warranted for Samza also. Thoughts? -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> This email and any attachments ma

Issue with consuming non-existent topics in 0.10.1

data is not returned, resulting in an empty set being returned. The behavior of Kafka here is screwy, but this still seems like a regression. The old behavior is nice because it doesn't require that producer systems come up before the stream processors. -- Tommy Becker Senior Software Engineer Dig

Re: Review Request 51689: Fix SAMZA-1012 - Generated changelog mappings are not consistent

/coordinator/JobCoordinator.scala ba38b5c Diff: https://reviews.apache.org/r/51689/diff/ Testing --- Tested manually Thanks, Tommy Becker

How do you monitor Samza?

ve to get some information on how other folks are monitoring their Samza jobs. -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> This em

Re: Question on changelog partition mapping

Done. https://issues.apache.org/jira/browse/SAMZA-1012 On 08/26/2016 06:57 PM, Yi Pan wrote: Hi, Tommy, It is perfectly fine. Would you please open a JIRA to include this improvement? Thanks! -Yi On Fri, Aug 26, 2016 at 6:11 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote:

Review Request 51689: Fix SAMZA-1012 - Generated changelog mappings are not consistent

--- Tested manually Thanks, Tommy Becker

Review Request 51613: Fix SAMZA-842

, Tommy Becker

Re: Question on changelog partition mapping

tion that you can try is to implement a grouper that will guarantee the order of groups coming out of the group() method. Does that make sense? Thanks! -Yi On Fri, Aug 12, 2016 at 5:31 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote: Hi Yi, Thanks for the response. We are runnin

Re: Question on changelog partition mapping

come back in the map produced by the SystemStreamPartitionGrouper. This non-determinism seems to have been the cause of our issues. Obviously data loss is a problem, but it seems like Samza could have recreated the original mapping. Should I file a bug on this? -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo

Question on changelog partition mapping

eterminism seems to have been the cause of our issues. Obviously data loss is a problem, but it seems like Samza could have recreated the original mapping. Should I file a bug on this? -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digi

Re: 0.10.1 Release

ution. Thanks, Andy -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> This email and any attachments may contain confidential

Long changelog restoration times?

on a YARN cluster in AWS so some variance in performance is to be expected, but this seems pretty extreme. Is anyone else seeing this behavior? -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailt

Re: Passing Java system properties to YARN containers of Samza Job

sed therein. If you have received this e-mail by mistake, please immediately delete the message and notify the sender of such incident. Please be noted that any unauthorized use, dissemination, distribution or copying of this email is strictly prohibited. -- Tommy Becker Senior Software Engineer

Re: Review Request 40106: SAMZA-812 CachedStore flushes too often

---------- On Nov. 9, 2015, 4:30 p.m., Tommy Becker wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/40106/ >

Review Request 40106: SAMZA-812 CachedStore flushes too often

://reviews.apache.org/r/40106/diff/ Testing --- Unit tested Thanks, Tommy Becker

Re: How to set up RocksDB

tid=139728971949824 And we're out of ideas at the moment. Can you please help me set up RocksDB correctly? I suppose we are doing something completely wrong. Is it possibble to use another key-value store at least? Lukáš Havrlant -- Tommy Becker Senior Software Engineer Digitals

Re: Streaming question

tificationRichBolt_With_TrimSequence(),4).setNumTasks(4) .fieldsGrouping("StepOne", new Fields("UserName")); Output of stepOne send as input to steptwo in above line of code in storm. Thanks regards, Annadath rao -- Tommy Becker Senior

YARN scheduler configuration

monitoring was unable to detect that there were jobs that, despite being in a "RUNNING" state, were not doing anything because they were starved for containers. Is there a way to configure YARN to reject container requests that can't be immediately satisfied? -- Tommy Becker Senior So

Re: Coordinator URL always 127.0.0.1

ue on cluster expansion. I can understand your concern if JC and AM are running on 2 separate containers. Please correct me if I have misunderstood your statement. Thanks! Navina On Fri, Jul 31, 2015 at 6:05 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote: Hey Yan, -- I do not quite

Re: Coordinator URL always 127.0.0.1

find your hostname is mycomputer.xzy.com 2. Put your host name in your hosts file. /etc/hosts . Such as 10.50.16.136 mycomputer.xzy.com {code} -Yi On Thu, Jul 30, 2015 at 11:35 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote: We are testing some jobs on a YARN grid

Coordinator URL always 127.0.0.1

dress() in org.apache.samza.coordinator.server.HttpServer#getUrl Am I off base here? Because I don't see how this is ever going to work in scenarios where the AM is on a different node than the containers. -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www

Access to high-watermark from within a Samza job

etrics but this API would enable mine as well. This seems pretty trivial to add, is there some reason it hasn't been done yet? Otherwise I can take a stab at it. Or is there another way to do what I need that I'm unaware of? -- Tommy Becker Senior Software Engineer Dig

Re: Missing artifact? org.apache.samza:samza-serializers_2.10:jar:0.9.0

the maven repo by chance? Thanks in advance, -Mart -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> This email and any a

Re: Issues with certain characters in config values

nks! Navina On 6/9/15, 5:22 AM, "Tommy Becker" <mailto:tobec...@tivo.com> wrote: Ok, the issue with spaces appears to have been addressed by https://issues.apache.org/jira/browse/SAMZA-530. But quotes are still a problem, and I'm becoming increasingly convinced that Util.

Re: Issues with certain characters in config values

Ok, the issue with spaces appears to have been addressed by https://issues.apache.org/jira/browse/SAMZA-530. But quotes are still a problem, and I'm becoming increasingly convinced that Util.envVarEscape() method is unnecessary. On 06/08/2015 03:02 PM, Tommy Becker wrote: We have a s

Issues with certain characters in config values

the YARN client. I filed the following bug for this, can anyone shed any light? https://issues.apache.org/jira/browse/SAMZA-700 -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec.

Did 0.9.1 ever get released?

Did 0.9.1 ever get released? I don't see it in Maven Central, nor has the site been updated... -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tob

Re: Logback logger

s built in run-class.sh, it doesn’t include the lib folder where the logback.xml is stored in. Davide On 14 May 2015, at 3:55 pm, Tommy Becker <mailto:tobec...@tivo.com> wrote: We use logback as well and simply include logback.xml in the task's jar file. On 05/14/2015 08:48 AM, Da

Re: Logback logger

le to be in the class path and the run-class.sh script only includes .*jar packages from lib folder. Do you know how I can sort that issue? Should I use a custom run-class.sh script? Thanks Davide -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com

Re: Review Request 33674: Fix SAMZA-660 - Default serdes do not work for changelog streams.

; is not used anymore. remove it? Oops, I missed that. - Tommy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33674/#review81978 -------

Re: Review Request 33674: Fix SAMZA-660 - Default serdes do not work for changelog streams.

/TestSamzaContainer.scala 58d7fe8 Diff: https://reviews.apache.org/r/33674/diff/ Testing --- Unit tested; had trouble running master due to some issue with the CoordinatorStream commit. Thanks, Tommy Becker

Review Request 33674: Fix SAMZA-660 - Default serdes do not work for changelog streams.

/ Testing --- Unit tested; had trouble running master due to some issue with the CoordinatorStream commit. Thanks, Tommy Becker

Re: Dealing with partitioning mismatches between bootstrap and input streams

omfortable with it, though, then (1) would be your next-best bet. Cheers, Chris On Tue, Apr 7, 2015 at 10:16 AM, Tommy Becker <mailto:tobec...@tivo.com> wrote: We have a Kafka topic containing data needed by several Samza jobs. These jobs will essentially read the data and build up sta

Re: Review Request 32877: SAMZA-616

samza/container/TestRunLoop.scala 2a0897f Diff: https://reviews.apache.org/r/32877/diff/ Testing (updated) --- Tested manually Thanks, Tommy Becker

Dealing with partitioning mismatches between bootstrap and input streams

is mostly Scala code. Are there any alternatives that I'm missing? Note that we need to process the data stream as a bootstrap stream. Using it as a changelog is insufficient because we need to be able to manipulate the data before building up the state store. -- Tommy Becker Seni

Re: sending data to a different partitions of the output stream

titioner.class property? Many thanks in advance, Vladimir -- Vladimir Lebedev http://linkedin.com/in/vlebedev -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> __

Review Request 32878: SAMZA-634

b80d349 samza-core/src/test/scala/org/apache/samza/job/local/TestProcessJob.scala 18eedc0 Diff: https://reviews.apache.org/r/32878/diff/ Testing --- Thanks, Tommy Becker

Review Request 32877: SAMZA-616

0897f Diff: https://reviews.apache.org/r/32877/diff/ Testing --- Thanks, Tommy Becker

Re: SamzaException: no job factory class defined

he problem is with the task.class definition or I still have something missing in the system. Thanks in advance, Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es<mailto:jbl...@nextel.es> Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_

Review Request 32202: SAMZA-456 - Add samza-yarn jar to multi-node tutorial.

SAMZA-456 - Add samza-yarn jar to multi-node tutorial. Diffs - docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 969152b Diff: https://reviews.apache.org/r/32202/diff/ Testing --- Thanks, Tommy Becker

Re: Review Request 32102: SAMZA-589

g.toString() now operates on a sanitized copy. AM Web UI and API endpoints also use sanitized values. Thanks, Tommy Becker

Review Request 32102: SAMZA-589

change adds a sanitize() method to Config that returns a new Config with values for keys beginning with "sensitive." masked. Config.toString() now operates on a sanitized copy. AM Web UI and API endpoints also use sanitized values. Thanks, Tommy Becker

Storing sensitive data in the Config

these sort of incidental exposures can't happen. -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> This email and any a

RE: [DISCUSS] JDK7

I just added this comment to SAMZA-455, since I was listed as requesting JDK 6 support: We have recently migrated to JDK 8, so you may consider my request for JDK 6 support withdrawn (though it was really more of a clarification of whether JDK 6 is supported than a request for such support). Pe

RE: Not safe to access KV stores from InitiableTask.init()

#x27;ll take a looks today/tomorrow. If you want to test out reordering it, please share any findings. :) Cheers, Chris On Monday, February 16, 2015, Tommy Becker wrote: > I have need to do some initial processing of the entries in my KV store on > startup before processing messages. I p

Not safe to access KV stores from InitiableTask.init()

I have need to do some initial processing of the entries in my KV store on startup before processing messages. I put the code into my task's init() method, and although it worked with an empty KV store/changelog once I have entries in there it bombs with a rather obscure exception: java.util.N

Re: Testing dev@samza.apache.org

Yes. On 01/23/2015 12:49 PM, Chris Riccomini wrote: Hey all, Could you please confirm that you're seeing this? I'm trying to verify the TLP migration for: https://issues.apache.org/jira/browse/INFRA-9055 Cheers, Chris This email and any attachments may