Hi,

I wanted to provide some feedback about the new API introduced in Kafka
streaming 2.7 for adding a Processor to the Topology (using PAPI).

Prior to 2.7, the Processor was typed only by the input key values <K,V>,
and the ProcessorContext was untyped.

In 2.7 ProcessorSupplier & Processor now have the additional params <...,
KForward, VForward> and the ProcessorContext also has <KForward,
VForward>.  The *only* use of this appears to be to provide some typing for
the "forward" functions.
One of my common use cases is where I have multiple child nodes each of
which has different output <K,V>.  To migrate to the new 2.7 API, seems to
require adding <.., Object, Object> parameters to every type which looks
noisy in the code and does nothing for improving type safety of the API.

I would like to ask why the new type parameters were added when they do not
actually achieve typing for the multiple child use-case?

Personally, I would prefer new methods on the ProcessorContext that allow
me to create a "typed" forwarder per child node.

e.g.

public interface ProcessorContext {

    /** Default forwarder. */
    <K, V> Forwarder<K, V> forwarder();

    /** Forwarder for a specific child. */
    <K, V> Forwarder<K, V> forwarder(final String childName);

}



public interface Forwarder<K, V> {
    void forward(Record<K, V> record);
}


With my suggested approach you at least get some typing per-topic (child
node).  To me this also feels like it better represents the domain model of
single stream input with multiple outputs.  Forwarder also feels like a
real entity in the model (similar to Producer), rather than just being a
method hacked into the context.


To me, the new 2.7 API feels like a step backwards.  I also get the
impression that the streaming API continues to break each release without a
particular strong model / design behind it (particularly the sometimes
jarring disconnect between PAPI and streaming API).
Having learned from initial implementations, is it time to take a step back
to design a "V2" streaming API which blends the existing PAPI + streaming
API?
I am curious what others think?

Thanks,
Ross

Reply via email to