Hi All, Any suggestions?
Thanks On Thu, Aug 1, 2019 at 8:58 PM Navneeth Krishnan <reachnavnee...@gmail.com> wrote: > Hi Guozhang, > > Thanks for the clarification. What I want to achieve is use of localized > data. We have much larger state which has to be used at a per instance > context. So if I can get the assignment upfront I can basically write data > to partitions in such a way that all data goes to that specific node which > handles the logic. > > I wouldn't be able to achieve my use case with just one stream worker, I'm > trying to spawn up multiple worker and wire up the instance with some > static data which will be used in the per message business logic. > > Thanks > > On Thu, Aug 1, 2019 at 9:51 AM Guozhang Wang <wangg...@gmail.com> wrote: > >> Hello Navneeth, >> >> I may be misunderstanding your intent from the previous emails here, so >> just a quick summary: >> >> 1) if you just want to "know" which partitions are assigned to which >> instance, this can be retrieved in multiple ways (e.g. the one mentioned >> by >> Matthias, and also one can get this info from JMX metrics which shows >> threads->tasks mapping). >> >> 2) if you want to "manipulate" the assignment so that a specific set of >> partitions to be assigned to a specific instance, today it is not doable >> directly as Streams library does not expose the task assignor customizable >> by users. >> >> Guozhang >> >> On Wed, Jul 31, 2019 at 4:48 PM Matthias J. Sax <matth...@confluent.io> >> wrote: >> >> > You cannot hook into partition assignment, and I am not sure what you >> > exactly want to do. >> > >> > You can get local assignment metadata via >> > `KafkaStreams#localThreadMetadata()` though. >> > >> > Hope this helps. >> > >> > >> > -Matthias >> > >> > On 7/29/19 11:29 PM, Navneeth Krishnan wrote: >> > > Hi All, >> > > >> > > The main reason for knowing the partitions is to have a localized >> routing >> > > based on partitions assigned to set a stream tasks. This would really >> > help >> > > in my use case. >> > > >> > > Thanks >> > > >> > > On Mon, Jul 29, 2019 at 8:58 PM Navneeth Krishnan < >> > reachnavnee...@gmail.com> >> > > wrote: >> > > >> > >> Hi, >> > >> >> > >> I'm using the processor topology for my use case and I would like to >> get >> > >> the partitions assigned to a particular stream instance. I looked at >> the >> > >> addSouce function but I don't see a way to add a callback to get >> > notified >> > >> when partition assignment or reassignment happens. Please advise. >> > >> >> > >> Thank you >> > >> >> > > >> > >> > >> >> -- >> -- Guozhang >> >
