Re: MessageStream with multiple concurrent operation stacks

2019-10-22 Thread Eric Shieh
Thank you Bharath and Boris, I really appreciate your response and guidance! Regards, Eric On Tue, Oct 22, 2019 at 6:00 PM Boris S wrote: > Yes, to answer your second question , you should be able to fork the > streams. Message Streams are idempotent and can be used in different > pipelines. >

Re: MessageStream with multiple concurrent operation stacks

2019-10-22 Thread Boris S
Yes, to answer your second question , you should be able to fork the streams. Message Streams are idempotent and can be used in different pipelines. With regards to group id, Samza does set the group ids, but, I think, they are the same for the whole job. The idea behind it is, that Samza is not u

Re: MessageStream with multiple concurrent operation stacks

2019-10-22 Thread Bharath Kumara Subramanian
Hi Eric, Based on the source code, it appears that each job designates a unique > group id when subscribing to kafka topic, is my understanding correct? > Yes. Samza uses a combination of job name and job id to generate the group id. > is it possible to have 2 independent stack of operations a

MessageStream with multiple concurrent operation stacks

2019-10-18 Thread Eric Shieh
Hi, Based on the source code, it appears that each job designates a unique group id when subscribing to kafka topic, is my understanding correct? If so, since one cannot call appDescriptor.getInputStream with the same KafkaInputDescriptor twice, is it possible to have 2 independent stack of opera