This approach sounds nice at first but it would fail if you start sending the same message but partitioned in different (orthogonal) ways. How would you go about that?
Maciej On 25 February 2015 at 05:17, Gwen Shapira <gshap...@cloudera.com> wrote: > Nice :) I like the idea of tying topic name to avro schemas. > > I have experience with other people's data, and until now I mostly > recommended: > <app type>.<app name>.<data set name>.<stage of processing> > > So we end up with things like: > etl.onlineshop.searches.validated > > Or if I have my own test dataset that I don't want to share: > users.gshapira.newapp.testing1 > > Makes it relatively easy to share datasets across the organization, and > also makes white-listing and black-listing relatively simple because of the > hierarchy (until we add a real topic hierarchy to kafka...). > > Gwen > > On Tue, Feb 24, 2015 at 1:13 PM, Thunder Stumpges <tstump...@ntent.com> > wrote: > >> We have a global namespace hierarchy for topics that is exactly our Avro >> namespace with Class Name. The template is basically: >> >> <root_ns>.Core.<core_data_types_shared_across_company> >> <root_ns>.<product>.<product_specific_hierarchy> >> >> The up side of this for us is that since the topics are named based on the >> Avro schema namespace and type, we can look up the avro schema in the Avro >> Schema Repository using the topic name, and the schema ID coded into the >> message. Each product then also has the flexibility of defining whatever >> topics they find useful. >> >> Hope this helps, >> Thunder >> >> -----Original Message----- >> From: Taylor Gautier [mailto:tgaut...@yahoo.com.INVALID] >> Sent: Tuesday, February 24, 2015 12:11 PM >> To: kafka-us...@incubator.apache.org >> Subject: Stream naming conventions? >> >> Hello all, >> Just wondering if those with a good amount of experience using Kafka in >> production with many streams have converged on any sort of naming >> convention. If so would you be willing to share? >> Thanks in advance, >> Taylor >> -- Twitter: @mjaskowski
