Hey John, thanks for writing this up! I like the proposal but there's one point that I think may be too restrictive:
"A processor that happens to use a typed store is actually emitting the same types that it is storing." I can imagine someone could want to leverage this new type safety without also limiting how they can interact with/use their store. As an (admittedly contrived) example, say you have an input stream of purchases of a certain type (entertainment, food, etc), and on seeing a new record you want to output how many types of purchase a shopper has made more than 5 purchases of in the last month. Your state store will probably be holding some more complicated PurchaseHistory object (keyed by user), but your output is just a <User, Long> I'm also not crazy about "processor2" as the package name ... not sure what a better one would be though (something with "typed"?) On Mon, Jun 17, 2019 at 12:47 PM John Roesler <j...@confluent.io> wrote: > Hi all, > > I'd like to propose KIP-478 (https://cwiki.apache.org/confluence/x/2SkLBw > ). > > This proposal would add output type bounds to the Processor interface > in Kafka Streams, which enables static checking of a number of useful > properties: > * A processor B that consumes the output of processor A is actually > expecting the same types that processor A produces. > * A processor that happens to use a typed store is actually emitting > the same types that it is storing. > * A processor is simply forwarding the expected types in all code paths. > * Processors added via the Streams DSL, which are not permitted to > forward results at all are statically prevented from doing so by the > compiler > > Internally, we can use the above properties to achieve a much higher > level of confidence in the Streams DSL implementation's correctness. > Actually, while doing the POC, I found a few bugs and mistakes, which > become structurally impossible with KIP-478. > > Additionally, the stronger types dramatically improve the > self-documentation of our Streams internal implementations, which > makes it much easier for new contributors to ramp up with confidence. > > Thanks so much for your consideration! > -John >
