Hey Daniel,

Thanks for the KIP! The current logging behavior seems like a
usability nightmare. I've only debugged single-worker JVM deployments,
and I see now that I took the uniqueness of a connector name for
granted.

I had a few questions about the change:

1. The MDC context change and the thread name change both seem to
solve the problem, and I assume that the majority of people would find
the thread name sufficient and leave things at their default. What is
the advantage of implementing both vs picking one?

2. I see that there is already a `workerId` that is passed around for
emitting to the status topic and to namespace the metrics. In MM2
dedicated mode, this contains the source and target. Do you think it
could be used to disambiguate the log context for
WorkerConnector/WorkerSinkTask as well?

3. Is the 'flow' (source->target) always more relevant than the
'source' and 'target' separately? Does it ever make sense to aggregate
logs for one source and multiple targets, or vice versa, and should we
expose these fields separately?

Overall, I think this feels like a shared-JVM problem that just
happens to appear in MM2 dedicated mode. I am interested to see if
there is a solution to this problem that also makes debugging easier
for other shared-JVM use-cases. We've also solved it in a number of
different ways and I wonder if there's a holistic way to namespace the
logging, metrics, and threads for workers in a shared JVM.

Thanks!
Greg

Reply via email to