Hi Jun, I also wanted to highlight the implication of having a dot "." as part of the topic and client.id with ".". Both topic name and client id need rule enforcement.
JMX Bean names are not correct: eg: I have topics called "topic.dot" and client.id as "my.client". JMX Bean name is: kafka.producer.*topic*:type=*dot* kafka.producer.*my.client*.topic.topic:type=dot With this developing a generic tool to flush the JMX values to some reporting tool for monitoring will be very hard to develop. Please let me know your recommendation. We had similar issues in Kafka old producer. At my company, we have convention to use "." to separation out functionality of website. eg "search.log" "checkout.log" etc. Please define/enforce rule for topic name and client.id. (Also, you might have to define a process of migrating topic name and client.id with new rules that you enforce with new producer). Thanks, Bhavesh On Mon, Sep 15, 2014 at 5:11 PM, Jun Rao <jun...@gmail.com> wrote: > Yes, that description is not precise. We do allow dots in general. However, > a topic can't be just "." or "..". > > Thanks, > > Jun > > On Mon, Sep 15, 2014 at 9:31 AM, Michael G. Noll < > michael+st...@michael-noll.com> wrote: > > > Ah -- I only tested with dots "." > > > > However, haven't you said earlier in this thread that only "-" and "_" > are > > allowed as special chars? This would exclude dots, although in the error > > message below they (dots) are said to be fine. Is this a bug or a > feature? > > ;-) > > > > --Michael > > > > > > > > > On 14.09.2014, at 20:41, Jun Rao <jun...@gmail.com> wrote: > > > > > > Actually, we do give you an error if an invalid topic is created > through > > > CLI. When we add a create topic api, we can return the correct error > code > > > too. > > > > > > bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic="te+dd" > > > --partitions 1 --replication-factor 1 > > > > > > Error while executing topic command topic name te+dd is illegal, > > contains a > > > character other than ASCII alphanumerics, '.', '_' and '-' > > > > > > kafka.common.InvalidTopicException: topic name te+dd is illegal, > > contains a > > > character other than ASCII alphanumerics, '.', '_' and '-' > > > > > > Thanks, > > > > > > Jun > > > > > > On Sun, Sep 14, 2014 at 1:46 AM, Michael G. Noll < > > mich...@michael-noll.com> > > > wrote: > > > > > >> Wouldn't it be helpful to throw an error or a warning if the user > > >> tries to create a topic with an invalid name? Currently neither the > > >> API nor the CLI tools inform you that you are naming a topic in a way > > >> you shouldn't. > > >> > > >> And as Otis pointed out elsewhere in this thread this ties back into > > >> the JMX/MBean issues related the usage of "-"/dashes and > > >> "_"/underscores, which are unfortunately the only non-alphanumeric > > >> characters that are at your disposal to add reasonable "structure" to > > >> your Kafka topic names. ( > > https://issues.apache.org/jira/browse/KAFKA-1481) > > >> > > >> Do you mind sharing your topic naming convention at LinkedIn? > > >> > > >> --Michael > > >> > > >> > > >> > > >> > > >> > > >>> On 11.09.2014 00:24, Jun Rao wrote: > > >>> We actually don't allow "." in the topic name. Topic name can be > > >>> alpha-numeric plus "-" and "_". > > >>> > > >>> Thanks, > > >>> > > >>> Jun > > >>> > > >>> On Tue, Sep 9, 2014 at 6:29 PM, Bhavesh Mistry > > >>> <mistry.p.bhav...@gmail.com> wrote: > > >>> > > >>>> Thanks, I was using without JMX. I will go through doc. But how > > >>>> about Topic or Metric name Topic Name Convention or Metric Name > > >>>> Convention ? The dot notation with topic having a ".". Any > > >>>> future plan to enforce some stand rules. > > >>>> > > >>>> Thanks, > > >>>> > > >>>> Bhavesh > > >>>> > > >>>> On Tue, Sep 9, 2014 at 3:38 PM, Jay Kreps <jay.kr...@gmail.com> > > >>>> wrote: > > >>>> > > >>>>> Hi Bhavesh, > > >>>>> > > >>>>> Each of those JMX attributes comes with documentation. If you > > >>>>> open up jconsole and attach to a jvm running the consumer you > > >>>>> should be able to read the descriptions for each attribute. > > >>>>> > > >>>>> -Jay > > >>>>> > > >>>>> On Tue, Sep 9, 2014 at 2:07 PM, Bhavesh Mistry > > >>>>> <mistry.p.bhav...@gmail.com> wrote: > > >>>>>> Kafka Team, > > >>>>>> > > >>>>>> Can you please let me know what each of following Metrics > > >>>>>> means ? Some > > >>>>> of > > >>>>>> them are obvious, but some are hard to understand. My Topic > > >>>>>> name is *TOPIC_NAME*. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> can we enforce a Topic Name Convention or Metric Name > > >>>>>> Convention. > > >>>>> Because > > >>>>>> in previous version of Kafka, we have similar issue of > > >>>>>> parsing Kafka Metrics name with host name issue (codahale > > >>>>>> lib) . I have topic name > > >>>>> with > > >>>>>> “.” So, it is hard to distinguish metric name and topic. > > >>>>>> Also, when guys get chance I would appreciate if you guys > > >>>>>> can explain metric description on wiki so community would > > >>>>>> know what to monitor. Please > > >>>> see > > >>>>>> below for full list of metrics from new producer. > > >>>>>> > > >>>>>> > > >>>>>> Thanks, > > >>>>>> > > >>>>>> Bhavesh > > >>>>>> > > >>>>>> > > >>>>>> record-queue-time-avg NaN *node-1.*request-latency-max > > >>>>>> -Infinity record-size-max -Infinity > > >>>>>> *node-1.*incoming-byte-rate NaN request-size-avg NaN > > >>>>>> *node-1.*request-latency-avg NaN *node-2.*request-size-avg > > >>>>>> NaN requests-in-flight 0.0 bufferpool-wait-ratio NaN > > >>>>>> network-io-rate NaN metadata-age 239.828 > > >>>>>> records-per-request-avg NaN record-retry-rate NaN > > >>>>>> buffer-total-bytes 6.7108864E7 buffer-available-bytes > > >>>>>> 6.7108864E7 topic.*TOPIC_NAME*.record-error-rate NaN > > >>>>>> record-send-rate NaN select-rate NaN > > >>>>>> node-2.outgoing-byte-rate NaN > > >>>>>> topic.*TOPIC_NAME*.record-retry-rate NaN batch-size-max > > >>>>>> -Infinity connection-creation-rate NaN > > >>>>>> node-1.outgoing-byte-rate NaN topic.*TOPIC_NAME*.byte-rate > > >>>>>> NaN waiting-threads 0.0 batch-size-avg NaN io-wait-ratio NaN > > >>>>>> io-wait-time-ns-avg NaN io-ratio NaN > > >>>>>> topic.TOPIC_NAME.record-send-rate NaN request-size-max > > >>>>>> -Infinity record-size-avg NaN request-latency-max -Infinity > > >>>>>> node-2.request-latency-max -Infinity record-queue-time-max > > >>>>>> -Infinity node-2.response-rate NaN node-1.request-rate NaN > > >>>>>> node-1.request-size-max -Infinity connection-count 3.0 > > >>>>>> incoming-byte-rate NaN compression-rate-avg NaN request-rate > > >>>>>> NaN node-1.response-rate NaN node-2.request-latency-avg NaN > > >>>>>> request-latency-avg NaN record-error-rate NaN > > >>>>>> connection-close-rate NaN *node-2.*request-size-max > > >>>>>> -Infinity topic.TOPIC_NAME.compression-rate NaN > > >>>>>> node-2.incoming-byte-rate NaN node-1.request-size-avg NaN > > >>>>>> io-time-ns-avg NaN outgoing-byte-rate NaN > > >>>>>> *node-2*.request-rate NaN response-rate NaN > > >> > > >
