This looks great. A couple of questions:
1. How will partition awareness be achieved? Is the plan to consume the stream
and then partition and transmit the data to where it needs to go? Or will there
be some way to work with 3rd party event streams to ensure that events are
delivered to the right place? For example, it is possible in Ignite 2 to
override the affinityfunction to align to Kafka partitions & consumer
assignment - thereby avoiding a second hop. 2. Is the plan to use consistent
key/thread assignment to provide the sequence guarantee? That would avoid the
need to use transactions or other locks in many cases - especially if the same
thread assignment could be used for accessing the data.
Regards,Colin.
On Tuesday, 23 May 2023 at 11:48:55 BST, Pavel Tupitsyn
<ptupit...@apache.org> wrote:
Igniters,
Please review Ignite 3.x Data Streamer proposal
https://cwiki.apache.org/confluence/display/IGNITE/IEP-102%3A+Data+Streamer
