On 05.09.2018 02:38, n...@afshartous.com wrote:
On Sep 4, 2018, at 4:20 PM, Jan Filipiak <jan.filip...@trivago.com> wrote:
what I meant is litterally this interface:
https://samza.apache.org/learn/documentation/0.7.0/api/javadocs/org/apache/samza/system/chooser/MessageChooser.html
<https://samza.apache.org/learn/documentation/0.7.0/api/javadocs/org/apache/samza/system/chooser/MessageChooser.html>
Hi Jan,
Thanks for the reply and I have a few questions. This Samza doc
https://samza.apache.org/learn/documentation/0.14/container/streams.html
<https://samza.apache.org/learn/documentation/0.14/container/streams.html>
indicates that the chooser is set via configuration. Are you suggesting adding
a new configuration for Kafka ? Seems like we could also have a method on
KafkaConsumer
public void register(MessageChooser messageChooser)
I don't have strong opinions regarding this. I like configs, i also
don't think it would be a problem to have both.
to make it more dynamic.
Also, the Samza MessageChooser interface has method
/* Notify the chooser that a new envelope is available for a processing. */
void update(IncomingMessageEnvelope envelope)
and I’m wondering how this method would be translated to Kafka API. In
particular what corresponds to IncomingMessageEnvelope.
I think Samza uses the envelop abstraction as they support other sources
besides kafka aswell. They are more
on the spark end of things when it comes to different input types. I
don't have strong opinions but it feels like
we wouldn't need such a thing in the kafka consumer but just use a
regular ConsumerRecord or so.
Best,
--
Nick
