In case someone else runs into this, we had to change the constructor we
used for the LocalApplicationRunner, and explicitly pass in the " new
CoordinatorStreamMetadataStoreFactory()'.
On Mon, Jan 6, 2020 at 3:18 PM Thunder Stumpges wrote:
> Hey dev team. Just upgrading our st
> > Brett
> >
> > From: Brett Konold
> > Sent: Tuesday, January 7, 2020 1:25 PM
> > To: dev@samza.apache.org
> > Subject: Re: Problem: upgrade 1.2 to 1.3 caused loss of clean shutdown on
> > SIGTERM
> >
> >
into this and will get back to you
> when I have something.
>
> Brett
>
> From: Thunder Stumpges
> Sent: Monday, January 6, 2020 6:54 PM
> To: dev@samza.apache.org
> Subject: Problem: upgrade 1.2 to 1.3 caused loss of clean shutdown on
> SI
We are attempting to upgrade from samza 1.2 to 1.3 in hopes of fixing
https://issues.apache.org/jira/browse/SAMZA-2198 where there was a deadlock
in the shutdown code which prevented completing a clean shutdown.
After the upgrade, it appears like NONE of the shutdown hooks / code are
being called
Hey dev team. Just upgrading our stand alone low-level tasks to Samza 1.3. We
use the LocalApplicationRunner and initialize most of our application within
SamzaApplication.describe() including setting up
"withDefaultSystem(systemDescriptor)"
However it seems that earlier on in the process, the
other PRs, and saw this one. It does
need a little work to make it up to Samza coding standards and more robust
unit tests, but we've been using it in production for some time now.
If there is interest, I will make a point to clean this up some and get in
a PR...
Let me know!
Thunder Stumpges
king.
Thanks!
Thunder
On Wed, Jun 5, 2019 at 11:20 AM Thunder Stumpges
wrote:
> Thanks, I think this all makes sense. I was able to get my application up
> and running with the exception of the Output SystemStream not finding its
> Serde.
>
> For a little more background, here&#x
yTask)' in your
> TaskApplication#describe with no other code changes. Please give that
> a shot and let us know if you run into any issues.
>
> Apologies for the confusion, we'll update the upgrade docs.
>
> Thanks,
> Prateek
>
> On Sat, Jun 1, 2019 at 11:13 AM Thu
on / properties...
One other thing, is that in a few of my jobs, we have dynamic output
SystemStream(s) created based on other runtime state. Is this not going to
be possible anymore?
A little more guidance would be most helpful.
Thanks!
Thunder Stumpges
ilable message history.
I.e., set 'samza.reset.offset' = true, and 'samza.offset.default' = oldest for
your stream. Let us know if this doesn't help.
Thanks,
Prateek
On Fri, Jul 6, 2018 at 11:43 AM, Thunder Stumpges wrote:
> Hi all,
>
>
> We've just run into a
Hi all,
We've just run into a strange problem with samza 0.14.1. We had a job down for
a bit, while kafka cleaned past our saved offsets. When starting the job now,
we get repeated org.apache.kafka.common.errors.OffsetOutOfRangeException. And
it just retries over and over again. We HAVE set
s
job.container.single.thread.mode = true?
On Fri, Jun 8, 2018 at 2:53 PM, Thunder Stumpges wrote:
> Thanks for the quick reply. That sounds very much like what I'm
> seeing. I'm merging in 0.14.1 to our branch now. I did try single
> threaded mode and unfortunately that di
o legacy single-threaded event loop.
Default is false, which enables the multithreading execution."
Let us know if this doesn't help.
Thanks,
Prateek
On Fri, Jun 8, 2018 at 1:35 PM, Thunder Stumpges
wrote:
> We have a new samza job which we just put into production. This job
>
We have a new samza job which we just put into production. This job processes
many topics (~30) but the total rate is not that high (~1200/sec in aggregate).
I am unable to get above ~700/sec and have a growing backlog.
We are running samza 0.12 (I have an update to 0.14 that is not tested or
p
ection on
>> K8s integration! There have been multiple prior asks on this, and
>> your learnings would be super-helpful.
>>
>> Best,
>> Jagdish
>>
>>
>>
>> On Tue, Mar 20, 2018 at 10:46 AM, Prateek Maheshwari <
>> prateek...@gmail.com>
&g
ld be super-helpful.
Best,
Jagdish
On Tue, Mar 20, 2018 at 10:46 AM, Prateek Maheshwari
wrote:
> Glad you were able to figure it out, that was very confusing. Thanks
> for the fix too.
>
> - Prateek
>
> On Mon, Mar 19, 2018 at 9:58 PM, Thunder Stumpges
>
> wrote:
>
And that last issue was mine. My setting override was not picked up and it was
using GroupByContainerCount instead.
-Thanks,
Thunder
-Original Message-
From: Thunder Stumpges
Sent: Monday, March 19, 2018 20:58
To: dev@samza.apache.org
Cc: Jagadish Venkatraman ; t...@recursivedream.com
1"
}, {
"system" : "kafka",
"partition" : 1,
"stream" : "test_topic2"
} ],
"changelog-partition" : 1
}
},
"container-id" : 0,
&quo
Can you please take and attach a thread dump with this?
Thanks,
Prateek
On Fri, Mar 16, 2018 at 4:47 PM, Thunder Stumpges
wrote:
> It appears it IS hung while serializing the JobModel... very strange!
> I added some debug statements around the calls:
>
> LOG.debug("Ge
l:1.9.13
Thanks!
Thunder
-Original Message-
From: Thunder Stumpges [mailto:tstump...@ntent.com]
Sent: Friday, March 16, 2018 15:29
To: dev@samza.apache.org; Jagadish Venkatraman
Cc: t...@recursivedream.com; yi...@linkedin.com; Yi Pan
Subject: RE: Old style "low level"
e more debug logging and try again.
Thanks for any guidance you all might have.
-Thunder
-Original Message-
From: Thunder Stumpges [mailto:tstump...@ntent.com]
Sent: Friday, March 16, 2018 14:43
To: dev@samza.apache.org; Jagadish Venkatraman
Cc: t...@recursivedream.com; yi...@linkedin.com; Yi
responses continue every 10 seconds, but no other activity or
messages occur.
It looks like it gets as far as confirming the JobModel and grouping the
partitions, but nothing actually starts up.
Any ideas?
Thanks in advance!
Thunder
-Original Message-----
From: Thunder Stumpges [mailto:t
r the guidance, this has been easier than I imagined! I’ll
report back when I get more of the Dockerization/Kubernetes running and test it
a bit more.
Cheers,
Thunder
From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com]
Sent: Thursday, March 15, 2018 14:46
To: Thunder Stumpges
Cc: dev@samza.apach
ad and
ask now anyway. Thanks!
-Thunder
From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com]
Sent: Thursday, March 15, 2018 1:41
To: dev@samza.apache.org; Thunder Stumpges ;
t...@recursivedream.com
Cc: yi...@linkedin.com; Yi Pan
Subject: Re: Old style "low level" Tasks with alter
d there
(there's only one container/configuration to deploy).
Best,
Tom
Thunder Stumpges writes:
> Hi all,
>
> We are using Samza (0.12.0) in about 2 dozen jobs implementing several
> processing pipelines. We have also begun a significant move of other
> services within
Hi all,
We are using Samza (0.12.0) in about 2 dozen jobs implementing several
processing pipelines. We have also begun a significant move of other services
within our company to Docker/Kubernetes. Right now our Hadoop/Yarn cluster has
a mix of stream and batch "Map Reduce" jobs (many reporting
effort is needed to make the move to 2.11, we would be glad to help out!
> That will be much appreciated! I don't think we have seen any strong
motivation for doing this within LinkedIn. If you guys can help us upgrade,
that will be awesome!!
Thanks!
Navina
On Mon, Nov 7, 2016 at 11:30
Hi all,
We are looking to get our projects upgraded to scala 2.11 as some of our
dependencies are dropping their compiling of 2.10 libraries. We found SAMZA-469
which looks to have had a patch for some time, however the last couple comments
seem pretty shaky about when (or if!?) support will be
28 matches
Mail list logo
