Tips for debugging per partition stream time?

2018-04-02 Thread dan bress
I have an app that is consuming multiple topics, and punctuating on stream time. I know that punctuate is driven by the min time of all the partitions of all the topics driving the transformer that I am punctuating on. When I deploy my app and punctuate is not called as I expect, what tools do I

Possible to enable client SSL when PLAINTEXT brokers exist in the cluster?

2018-04-02 Thread Alexander Maniates
Is it possible to enable client SSL when PLAINTEXT brokers exist in the cluster? Suppose you have 9 brokers in a cluster. 8 are configured with both SSL and PLAINTEXT endpoints. One broker is configured with only a PLAINTEXT endpoint, and this broker does NOT OWN any partitions. I supply a list of

Re: Tips for debugging per partition stream time?

2018-04-02 Thread Matthias J. Sax
Dan, Kafka Streams supports two types of punctuation: event-time and wall-clock time punctuation. Wall-clock time punctuation will be called even if event-time does not progress and even if there is no new input data available for processing. Not sure if this is what you are looking for. Beside t

Re: Tips for debugging per partition stream time?

2018-04-02 Thread dan bress
Hi Matthias, Yes, I'm aware of Wall-clock time punctuation, but that is not the behavior my customers want, so I can't use that unfortunately... OK. I'm going to try and write a utility "StreamTimeInspector" transformer that will do as you suggest and give me the stats the I want. If we find th

Re: Possible to enable client SSL when PLAINTEXT brokers exist in the cluster?

2018-04-02 Thread Martin Gainty
FWIR: is a plaintext broker supported with SASL Zookeeper frontend..assuming no here are the reasons: https://www.tutorialspoint.com/apache_kafka/apache_kafka_cluster_architecture.htm [https://www.tutorialspoint.com/apache_kafka/images/cluster_architecture.jpg]

Re: Tips for debugging per partition stream time?

2018-04-02 Thread Matthias J. Sax
Atm, Kafka does not ship any ready-to-use transformers. All operators that are ready-to-use are provided via the DSL only and their implementation itself is not public API. The question arises though, if a transformer like this would be the best way to expose this information or if there might be