Great discussion !
Here are some more thoughts
The point that repartitioning is a more general purpose solution is surely
spot on. For many source systems (Kinesis, Google Pub-Sub, any of the
older queuing systems (rabbitMQ etc. etc.), repartitioning is anyways
functionally required to do even s
+1 (non binding)
Great to see the SEP process being followed.
cheers
Kartik
On Thu, Mar 30, 2017 at 1:48 PM, Renato Marroquín Mogrovejo <
renatoj.marroq...@gmail.com> wrote:
> Thanks for the answers Navina!
>
> +1 (non-binding)
>
> 2017-03-30 22:32 GMT+02:00 Navina Ramesh :
>
> > Hi Renato,
> >
KStreams was surely incubated based on early Samza learnings. Although
they have a common base in terms of architecture, they started evolving
independently a year ago.
In the last year there have been significant improvements in Samza in order
to make stateful stream processing more reliable an
The log compaction fixes in Kafka 0.9 were done by our linkedIn Kafka
developers to fix issues faced by Samza.
So Yes.. Samza 0.10 can be safely used with Kafka 0.9.
At LinkedIn we currently run kafka broker from apache kafka trunk.. So it
is a more recent Kafka version. But that should be fine.
Thanks for starting this discussion and great to see the integration with
Kubernetes. It is very timely because we(at LinkedIn) have recently
started to refocus on Samza-516 and were thinking of kicking off a similar
discussion soon.
I am in agreement with you that there are two possible levels t
We have been wanting to do something similar at LinkedIn. We however
haven't thought through the details.
if container == thread.. then we would need to change the AppMaster to
request the appropriate number of Yarn 'containers' (processes) .. i.e. we
would have to decouple the process count from
Looks like we won't have a streaming option. But we will surely
upload the videos and send a link on this mailing list.
> On Oct 9, 2015, at 1:02 AM, Renato Marroquín Mogrovejo
> wrote:
>
> Hi Ed,
>
> Will there be a streaming option for this? or will it be recorded? for us
> that are in diffe
Thanks Chris/Jay for sending this out.
I wanted to chime in to reflect some of the motivations from LinkedIn POV.
Late last year we started observing that there is a class of event driven
applications at LinkedIn which can't leverage some of the cool work that is
happening in Samza pertaining
