Hello Anton, first of all, my apologies for the late reply. I wanted to think a little more about your message and then the vacation season got into my way. The plus is that I have some additional references to post.
So now let's finally begin with the answer (I am not removing any of your text so that it is probably better to get the idea after such a long time...): > Rainer: > > Sorry if I did not follow earlier discussion and misunderstand > something obvious. But here is my feedback. > > I like the key=value approach. A lot! This would be hugely welcome in > syslog and this is a great undertaking! To begin with, we could plug > things like severity and facility because, otherwise, they don't even > get stored for standard UDP syslog messages! I have dozens of > practical examples of other things that could be put into these tags. Yes, I think this is one of the main benefits of the new "cookie"/metadata format. It allows for many extensions and fits them in very nicely. Also, non-metadata-aware (and older) syslogds can simply store the metadata as part of the message content. > I am not clear why it has to be enclosed specifically in XML-like tags > though. See what we did below for an alternative. The orginal idea was that it is relatively easy to get an XML parser AND XML already has thought-out, practically prooven things like escape sequences for characters which are used for special purposes (e.g. > for ">"). However, using XML raises some other questions. I will actually raise them in a soon-to-be-written reply to chris, because I think his message was highly focussed just on this. > > If "/>" is a special sequence, are there any rules for escaping it if > one needs to put it in tag values? "/>" denotes the end of a tag that is not a container. So you may say "<tag p="aaa"></tag>" or just "<tag p="aaa" />". I am not 100% sure if the SP before "/" is needed, I don't think so. > I am also not clear on the use of word "cookie" in general. Can you > explain where does this name concept come from? The only association > that comes to my mind is with web browser cookies, and I don't see the > connection. Can we call these key-value pairs "tags" or something > else? The term "cookie" stems back to syslog-sign. As it looks now, "metadata" seems to be an accepted name change (but few comments so far). I am tracking this issue at http://www.syslog.cc/ietf/protocol/issue3.html (will receive an update with the new messages soon). > At Cisco, we are evangelizing a new logging specification for logging > to file or through syslog. It is called CiscoLog. It specifies > optional use of tags. We are looking at delimiting individual tags > with [] and the tag section with ": ". Here is an example of a > message with tags as seen on the wire, but without the syslog PRI > field: > > 12: host.cisco.com: *Jun 13 2003 23:11:52.454 UTC: > %BACC-4-BAD_REQUEST: RDU: > [comp=parser][mac=1,6,aa:bb:cc:11:22:33][txn=mytxn123]: Bad request > received from device [1,6,aa:bb:cc:11:22:33]. Header missing. > > Things in square brackets after "RDU: " are tags. We don't follow > some of the optional syslog RFC3164 time and other header formats > partly for legacy reasons and also largely due to the fact that syslog > server implementations vary in what they store (or how they forward) > messages while we need a minimum set of data to always be present in > the log file and in a consistent format. I can give more details about > our format if needed. I am allowed to disclose it. > > It is also useful to differentiate tag syntax from tag meaning. We > addressed this with the use of "semantic extensions". For example, > tag with key "ip" just means that the value is a properly formatted IP > related to message. Tag "ip.orig" means it is IP of host originating > message. Tag "ip.client" means it is a tag from a client talking to > server which originated the message. You can also see things like > "ip.from", "ip.to". Would it be useful to make the syntax/semantic > distinction in the standard? This paragraph was one of the main raesons for me not replying immediately. I thought quite a bit about it. On my initial look, I thought, yes, this is header data that needs to be specified in the protocol itself. On second thought, I think it is too verbose to be actually part of the header. A good indication is that it is optional. Don't misunderstand me: I think it is an excellent idea - I just think it is too "upper layer" to be included into -protocol. In fact, I think this addresses a payload issue, which is currently outside the scope of the WG (but may come into the scope ones the other issues are finalized, at least Chris has already allowed on-list discussion, so I think there is a chance... ;)). I am a strong advocate for some payload specification (which can also solve some of the "how to store it" questions around [I think Albert asked, but I am not sure]). However, I think we really need to finish the lower layers first. Once this is done, we can focus (and have ressources!) to look at the payload. Anyhow, I would be very interested to read your format specification. I would appreciate if you could post it. Not only for the payload part but also on ideas that could/should be used in -protocol. One question I have (for example) is if you have escapes defined for the special characters ('['/']'/':']. > > BTW, we also use tags to mark parts of a multi-part messages. So, > there is a lot of uses for them. > > > <cookie MSGNO=234 VENDOR=example vendorparam=name%nnn > > /> msg.msg.msg > > Are spaces allowed for values? They should be, I think. Actually, this sample was done to quickly. The message orginally sent by Chris was just a rough idea and I should have better edited it while quoting. You need to have parantheses around all values. So the sample would actually be <cookie MSGNO="234" VENDOR="example vendor" param="name%nnn" />msg.msg.msg ... and your question had probably never appeared ;) > Why a different syntax for experimental tags? I thought to include a class for cross-vendors tags that are "born" in IETF and other non-vendor specific places. People feel strong about names. So I thought it doesn't hurt to allow for a class which is neiter standardized nor vendor specific. I had the following picture on my mind: On an IETF list, vendeorA, vendorB and vendorC have a bright idea and create a new cookie to implement it. Technically, everything looks bright. But then begins the political issue of wheter it should be placed in the vendorA, vendorB or vendorC namespace... maybe this expresses what I mean ;) I still have the feeling that a special category for such cookies/metadata would at least not hurt. What does the rest of the list mean? > Getting back to syslog-international, I think tag values should also > be allowed to be in foreign languages eventually. Preferably UTF-8. > Tags could be used to provide any easily parseable parameters. If I > use "username" tag, for example, then in Japan the value could be in > Japanese. We had some specific requirements from some Cisco apps to > support this. We support foreign languages in tag values and > specifying UTF-8 encoding for entire message (US-ASCII is preserved as > is). YES. This is why -protocol does not enforce 7 bit encoding. -international is very easy doable with this. UTF-8 is the solution to everything as soon as 8 bit encoding is allowed. -international will become a very thin document ;-) Rainer > > Thanks, > Anton. > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] Behalf Of Rainer Gerhards > > Sent: Wednesday, December 17, 2003 12:55 PM > > To: [EMAIL PROTECTED] > > Subject: syslog-protocol: Cookie Format > > > > > > Hi WG, > > > > Chris has proposed a XLM-like cookie format in private > > mail. I received > > permission to quote him on this. I would appreciate any > > feedback from > > the WG. > > > > Chris said (Chris, please correct me if you feel I am quoting out of > > context): > > > > #### > > The thought just struck me about the use of cookies, languages, etc. > > The > > use of the special characters is what John Kelsey proposed. > > We're not > > limited to that if you can come up with something better. I was > > thinking > > of something like the following at the start of the MSG: > > > > <cookie MSGNO=123 ENCODING=USASCII /> msg.msg.msg > > or > > <cookie MSGNO=234 VENDOR=example vendorparam=name%nnn > > /> msg.msg.msg > > or even > > <cookie block=certblock /> certificate.block.msg > > > > syslog-protocol would then define the IANA held cookie > > parameters and > > vendors would be able to add their own. Experimental > > parameters could > > be > > done like "vendorparam" (above) where the "%" replaces the > > "=" of a real > > parameter. > > #### > > > > Please note: In this quote, "msg.msg.msg" is a syslog > > message. It is NOT > > (necessarily) multiple messages (it may for fragmented messages, but > > that is a separate issue). > > > > A key point in Chris suggestion is that this is NOT actual > > XML. It just > > looks so. But we could specify the exact sequence of parameters and > > such, so that no XML parser is needed to process it. After > > some initial > > scepticism, I have to say that I like this approach. It > > looks very clean > > and extensible. > > > > In this mail, I am trying to gather some feedback from the WG if we > > should move into the proposed direction. If so, I will > > focus on some of > > the details. > > > > Questions so: Do you like this? Do you think it is useful? Should we > > proceed into this direction? > > > > Rainer > > > > > > >
