For most operators yes. For sure, for all stateless operators like
`mapValues()` -- for stateful operators, it depends if data
repartitioning is required. If yes, the topology would be split into two
sub-topologies and thus, both `peek()` operations could run on different
threads.
If you want to d
Good to know, thanks Matthias!
You've mentioned a previous operator, but what about:
`peek().mapValues().peek()`, will both `peek`s be in the same thread as
well?
El mar., 25 sept. 2018 a las 23:14, Matthias J. Sax ()
escribió:
> Just for clarification:
>
> `peek()` would run on the same thread
Just for clarification:
`peek()` would run on the same thread and the previous operator. Even
if---strictly speaking---there is no public contract to guarantee this,
it would be the case in the current implementation, and I also don't see
any reason why this would change at any point in the future
Thanks, everyone!
@Bill, the main issue with using `KStraem#peek()` is that AFAIK each `peek`
processor runs on a potentially different thread, then passing the trace
between them could be challenging. It will also require users to add these
operators themselves, which could be too cumbersome to u
Jorge:
I have a crazy idea off the top of my head.
Would something as low-tech using KSteam.peek calls on either side of
certain processors to record start and end times work?
Thanks,
Bill
On Tue, Sep 18, 2018 at 4:38 PM Guozhang Wang wrote:
> Jorge:
>
> My suggestion was to let your users to
Jorge:
My suggestion was to let your users to implement on the
TracingProcessorSupplier
/ TracingProcessor directly instead of the base-line ProcessorSupplier /
Processor. Would that work for you?
Guozhang
On Tue, Sep 18, 2018 at 8:02 AM, Jorge Esteban Quilcate Otoya <
quilcate.jo...@gmail.com
Hi Jorge,
Thanks for the clarifications.
Yes, I'm also not sure what "built-in tracing" would look like, and it may
not be a good idea. FWIW, though, I was not thinking of something like
"rich functions". Rather, I was imagining that Streams could just always
record spans in headers as it process
final StreamsBuilder builder = kafkaStreamsTracing.builder();Thanks
Guozhang and John.
@Guozhang:
> I'd suggest to provide a
> WrapperProcessorSupplier for the users than modifying
> InternalStreamsTopology: more specifically, you can provide an
> `abstract WrapperProcessorSupplier
> implements P
If I understand the request, it's about tracking the latencies for a
specific record, not the aggregated latencies for each processor.
Jorge,
The diff you posted only contains the library-side changes, and it's not
obvious how you would use this to insert the desired tracing code.
Perhaps you cou
Hello Jorge,
>From the TracingProcessor implementation it seems you want to track
per-processor processing latency, is that right? If this is the case you
can actually use the per-processor metrics which include latency sensors.
If you do want to track, for a certain record, what's the latency of
Thanks for your answer, Matthias!
What I'm looking for is something similar to interceptors, but for Stream
Processors.
In Zipkin -and probably other tracing implementations as well- we are using
Headers to propagate the context of a trace (i.e. adding metadata to the
Kafka Record, so we can crea
>> I'm experimenting on how to add tracing to Kafka Streams.
What do you mean by this exactly? Is there a JIRA? I am fine removing
`final` from `InternalTopologyBuilder#addProcessor()` -- it's an
internal class.
However, the diff also shows
> public Topology(final InternalTopologyBuilder interna
Hi everyone,
I'm experimenting on how to add tracing to Kafka Streams.
One option is to override and access
`InternalTopologyBuilder#addProcessor`. Currently this method it is final,
and builder is not exposed as part of `StreamsBuilder`:
```
public class StreamsBuilder {
/** The actual top
13 matches
Mail list logo
