Hi Bhavesh, Yes, if you look at https://issues.apache.org/jira/browse/KAFKA-1481 it describes this sort of a problem with various delimiters used by Kafka. As Jun suggested, I'll start a new thread and try to clearly explain the problem.
Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Mon, Sep 15, 2014 at 9:41 PM, Bhavesh Mistry <mistry.p.bhav...@gmail.com> wrote: > Hi Jun, > > I also wanted to highlight the implication of having a dot "." as part of > the topic and client.id with ".". Both topic name and client id need > rule > enforcement. > > JMX Bean names are not correct: > > eg: > > I have topics called "topic.dot" and client.id as "my.client". > > JMX Bean name is: > > kafka.producer.*topic*:type=*dot* > kafka.producer.*my.client*.topic.topic:type=dot > > > With this developing a generic tool to flush the JMX values to some > reporting tool for monitoring will be very hard to develop. Please let me > know your recommendation. We had similar issues in Kafka old producer. > > At my company, we have convention to use "." to separation out > functionality of website. eg "search.log" "checkout.log" etc. Please > define/enforce rule for topic name and client.id. (Also, you might have > to > define a process of migrating topic name and client.id with new rules that > you enforce with new producer). > > Thanks, > > Bhavesh > > On Mon, Sep 15, 2014 at 5:11 PM, Jun Rao <jun...@gmail.com> wrote: > > > Yes, that description is not precise. We do allow dots in general. > However, > > a topic can't be just "." or "..". > > > > Thanks, > > > > Jun > > > > On Mon, Sep 15, 2014 at 9:31 AM, Michael G. Noll < > > michael+st...@michael-noll.com> wrote: > > > > > Ah -- I only tested with dots "." > > > > > > However, haven't you said earlier in this thread that only "-" and "_" > > are > > > allowed as special chars? This would exclude dots, although in the > error > > > message below they (dots) are said to be fine. Is this a bug or a > > feature? > > > ;-) > > > > > > --Michael > > > > > > > > > > > > > On 14.09.2014, at 20:41, Jun Rao <jun...@gmail.com> wrote: > > > > > > > > Actually, we do give you an error if an invalid topic is created > > through > > > > CLI. When we add a create topic api, we can return the correct error > > code > > > > too. > > > > > > > > bin/kafka-topics.sh --zookeeper localhost:2181 --create > --topic="te+dd" > > > > --partitions 1 --replication-factor 1 > > > > > > > > Error while executing topic command topic name te+dd is illegal, > > > contains a > > > > character other than ASCII alphanumerics, '.', '_' and '-' > > > > > > > > kafka.common.InvalidTopicException: topic name te+dd is illegal, > > > contains a > > > > character other than ASCII alphanumerics, '.', '_' and '-' > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > On Sun, Sep 14, 2014 at 1:46 AM, Michael G. Noll < > > > mich...@michael-noll.com> > > > > wrote: > > > > > > > >> Wouldn't it be helpful to throw an error or a warning if the user > > > >> tries to create a topic with an invalid name? Currently neither the > > > >> API nor the CLI tools inform you that you are naming a topic in a > way > > > >> you shouldn't. > > > >> > > > >> And as Otis pointed out elsewhere in this thread this ties back into > > > >> the JMX/MBean issues related the usage of "-"/dashes and > > > >> "_"/underscores, which are unfortunately the only non-alphanumeric > > > >> characters that are at your disposal to add reasonable "structure" > to > > > >> your Kafka topic names. ( > > > https://issues.apache.org/jira/browse/KAFKA-1481) > > > >> > > > >> Do you mind sharing your topic naming convention at LinkedIn? > > > >> > > > >> --Michael > > > >> > > > >> > > > >> > > > >> > > > >> > > > >>> On 11.09.2014 00:24, Jun Rao wrote: > > > >>> We actually don't allow "." in the topic name. Topic name can be > > > >>> alpha-numeric plus "-" and "_". > > > >>> > > > >>> Thanks, > > > >>> > > > >>> Jun > > > >>> > > > >>> On Tue, Sep 9, 2014 at 6:29 PM, Bhavesh Mistry > > > >>> <mistry.p.bhav...@gmail.com> wrote: > > > >>> > > > >>>> Thanks, I was using without JMX. I will go through doc. But how > > > >>>> about Topic or Metric name Topic Name Convention or Metric Name > > > >>>> Convention ? The dot notation with topic having a ".". Any > > > >>>> future plan to enforce some stand rules. > > > >>>> > > > >>>> Thanks, > > > >>>> > > > >>>> Bhavesh > > > >>>> > > > >>>> On Tue, Sep 9, 2014 at 3:38 PM, Jay Kreps <jay.kr...@gmail.com> > > > >>>> wrote: > > > >>>> > > > >>>>> Hi Bhavesh, > > > >>>>> > > > >>>>> Each of those JMX attributes comes with documentation. If you > > > >>>>> open up jconsole and attach to a jvm running the consumer you > > > >>>>> should be able to read the descriptions for each attribute. > > > >>>>> > > > >>>>> -Jay > > > >>>>> > > > >>>>> On Tue, Sep 9, 2014 at 2:07 PM, Bhavesh Mistry > > > >>>>> <mistry.p.bhav...@gmail.com> wrote: > > > >>>>>> Kafka Team, > > > >>>>>> > > > >>>>>> Can you please let me know what each of following Metrics > > > >>>>>> means ? Some > > > >>>>> of > > > >>>>>> them are obvious, but some are hard to understand. My Topic > > > >>>>>> name is *TOPIC_NAME*. > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> can we enforce a Topic Name Convention or Metric Name > > > >>>>>> Convention. > > > >>>>> Because > > > >>>>>> in previous version of Kafka, we have similar issue of > > > >>>>>> parsing Kafka Metrics name with host name issue (codahale > > > >>>>>> lib) . I have topic name > > > >>>>> with > > > >>>>>> “.” So, it is hard to distinguish metric name and topic. > > > >>>>>> Also, when guys get chance I would appreciate if you guys > > > >>>>>> can explain metric description on wiki so community would > > > >>>>>> know what to monitor. Please > > > >>>> see > > > >>>>>> below for full list of metrics from new producer. > > > >>>>>> > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> > > > >>>>>> Bhavesh > > > >>>>>> > > > >>>>>> > > > >>>>>> record-queue-time-avg NaN *node-1.*request-latency-max > > > >>>>>> -Infinity record-size-max -Infinity > > > >>>>>> *node-1.*incoming-byte-rate NaN request-size-avg NaN > > > >>>>>> *node-1.*request-latency-avg NaN *node-2.*request-size-avg > > > >>>>>> NaN requests-in-flight 0.0 bufferpool-wait-ratio NaN > > > >>>>>> network-io-rate NaN metadata-age 239.828 > > > >>>>>> records-per-request-avg NaN record-retry-rate NaN > > > >>>>>> buffer-total-bytes 6.7108864E7 buffer-available-bytes > > > >>>>>> 6.7108864E7 topic.*TOPIC_NAME*.record-error-rate NaN > > > >>>>>> record-send-rate NaN select-rate NaN > > > >>>>>> node-2.outgoing-byte-rate NaN > > > >>>>>> topic.*TOPIC_NAME*.record-retry-rate NaN batch-size-max > > > >>>>>> -Infinity connection-creation-rate NaN > > > >>>>>> node-1.outgoing-byte-rate NaN topic.*TOPIC_NAME*.byte-rate > > > >>>>>> NaN waiting-threads 0.0 batch-size-avg NaN io-wait-ratio NaN > > > >>>>>> io-wait-time-ns-avg NaN io-ratio NaN > > > >>>>>> topic.TOPIC_NAME.record-send-rate NaN request-size-max > > > >>>>>> -Infinity record-size-avg NaN request-latency-max -Infinity > > > >>>>>> node-2.request-latency-max -Infinity record-queue-time-max > > > >>>>>> -Infinity node-2.response-rate NaN node-1.request-rate NaN > > > >>>>>> node-1.request-size-max -Infinity connection-count 3.0 > > > >>>>>> incoming-byte-rate NaN compression-rate-avg NaN request-rate > > > >>>>>> NaN node-1.response-rate NaN node-2.request-latency-avg NaN > > > >>>>>> request-latency-avg NaN record-error-rate NaN > > > >>>>>> connection-close-rate NaN *node-2.*request-size-max > > > >>>>>> -Infinity topic.TOPIC_NAME.compression-rate NaN > > > >>>>>> node-2.incoming-byte-rate NaN node-1.request-size-avg NaN > > > >>>>>> io-time-ns-avg NaN outgoing-byte-rate NaN > > > >>>>>> *node-2*.request-rate NaN response-rate NaN > > > >> > > > > > >
