JingsongLi commented on pull request #18412: URL: https://github.com/apache/flink/pull/18412#issuecomment-1019821611
> I think here we have two different thread-safety issues. One is that the consumer is invoked by the KafkaProducer that you probably need to fix and trigger the callback via the Mailbox. The other issue is around adding the subscribers concurrently. I don't think we need to put it inside the mailbox, it would be very performance intensive, it's a per record operation. A callback consumer, which I think has asynchronous processing reasonable. > I agree the Consumer needs to be serializable but if we pass an unmodifiable list to the sink writer it solves a lot of the thread-safety issues because it is not possible to add new subscribers after the sink translation. I guess if that works depends on your implementation that you have in mind to use the metadata. Yes, the problem is that this is a runtime statistic, which needs to expose information to an external caller, and it's hard to implement by a serializable class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
