Hey Matthias, Make sense, I'm more advocating for removing the word topology than any particular new replacement.
-Jay On Mon, Mar 13, 2017 at 12:30 PM, Matthias J. Sax <matth...@confluent.io> wrote: > Jay, > > thanks for your feedback > > > What if instead we called it KStreamsBuilder? > > That's the current name and I personally think it's not the best one. > The main reason why I don't like KStreamsBuilder is, that we have the > concepts of KStreams and KTables, and the builder creates both. However, > the name puts he focus on KStream and devalues KTable. > > I understand your argument, and I am personally open the remove the > "Topology" part, and name it "StreamsBuilder". Not sure what others > think about this. > > > About Processor API: I like the idea in general, but I thinks it's out > of scope for this KIP. KIP-120 has the focus on removing leaking > internal APIs and do some cleanup how our API reflects some concepts. > > However, I added your idea to API discussion Wiki page and we take if > from there: > https://cwiki.apache.org/confluence/display/KAFKA/ > Kafka+Streams+Discussions > > > > -Matthias > > > On 3/13/17 11:52 AM, Jay Kreps wrote: > > Two things: > > > > 1. This is a minor thing but the proposed new name for KStreamBuilder > > is StreamsTopologyBuilder. I actually think we should not put > topology in > > the name as topology is not a concept you need to understand at the > > kstreams layer right now. I'd think of three categories of concepts: > (1) > > concepts you need to understand to get going even for a simple > example, (2) > > concepts you need to understand to operate and debug a real > production app, > > (3) concepts we truly abstract and you don't need to ever understand. > I > > think in the kstream layer topologies are currently category (2), and > this > > is where they belong. By introducing the name in even the simplest > example > > it means the user has to go read about toplogies to really understand > even > > this simple snippet. What if instead we called it KStreamsBuilder? > > 2. For the processor api, I think this api is mostly not for end > users. > > However this are a couple cases where it might make sense to expose > it. I > > think users coming from Samza, or JMS's MessageListener ( > > https://docs.oracle.com/javaee/7/api/javax/jms/MessageListener.html) > > understand a simple callback interface for message processing. In > fact, > > people often ask why Kafka's consumer doesn't provide such an > interface. > > I'd argue we do, it's KafkaStreams. The only issue is that the > processor > > API documentation is a bit scary for a person implementing this type > of > > api. My observation is that people using this style of API don't do a > lot > > of cross-message operations, then just do single message operations > and use > > a database for anything that spans messages. They also don't factor > their > > code into many MessageListeners and compose them, they just have one > > listener that has the complete handling logic. Say I am a user who > wants to > > implement a single Processor in this style. Do we have an easy way to > do > > that today (either with the .transform/.process methods in kstreams > or with > > the topology apis)? Is there anything we can do in the way of trivial > > helper code to make this better? Also, how can we explain that > pattern to > > people? I think currently we have pretty in-depth docs on our apis > but I > > suspect a person trying to figure out how to implement a simple > callback > > might get a bit lost trying to figure out how to wire it up. A simple > five > > line example in the docs would probably help a lot. Not sure if this > is > > best addressed in this KIP or is a side comment. > > > > Cheers, > > > > -Jay > > > > On Fri, Feb 3, 2017 at 3:33 PM, Matthias J. Sax <matth...@confluent.io> > > wrote: > > > >> Hi All, > >> > >> I did prepare a KIP to do some cleanup some of Kafka's Streaming API. > >> > >> Please have a look here: > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >> 120%3A+Cleanup+Kafka+Streams+builder+API > >> > >> Looking forward to your feedback! > >> > >> > >> -Matthias > >> > >> > > > >
