Hey David, I read that documentation. As far as I can tell, it only discusses using rsyslog/adiscon-built templates. I don't think I can use those templates - regardless of "bracketing". The SAM we use doesn't appear to support multi-line matching through PCRE(2) via the /s flag.
Hence why I'm looking to build and use my own (normalized) JSON template. The JSON template that adiscon uses is both multi line and isn't actual JSON. It contains elements of JSON. Cheers, Mike [cid:e628dcc4-f053-42b6-90a6-79771b1ae670] Michael Redbourne (he/him) Senior Security Analyst Office: +1 (506) 606-0384 Cell: +61 04 2647 3071 SOC: 1-833-415-2424 www.bulletproofsi.com<http://www.bulletproofsi.com/> Book a Meeting<https://outlook.office365.com/owa/calendar/michaelredbou...@bulletproofsolutions.onmicrosoft.com/bookings/> [cid:ea725086-9fec-4d13-8b97-056dba7feac4] Notes: Please be advised that I live in Sydney, Australia. My normal hours are 8AM-5PM (Australia/Sydney). I allow 24-hour calendar bookings, but bookings made outside of official working hours should be discussed with me prior to booking. ________________________________ From: David Lang <da...@lang.hm> Sent: Monday, January 20, 2025 12:04 PM To: Redbourne,Michael via rsyslog <rsyslog@lists.adiscon.com> Cc: Redbourne,Michael <michael.redbou...@bulletproofsi.com> Subject: Re: [rsyslog] Building Custom pstats Template CAUTION: The Sender is located Outside The Organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. see https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rsyslog.com%2Fdoc%2Fconfiguration%2Fmodules%2Fimpstats.html&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7C9781aa28dc084468627308dd38ee6516%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638729318758914939%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=7XiAjfDEAt8QZyju2Od1NnGFNhxjlpZUz8CT1Xd0i1Q%3D&reserved=0<https://www.rsyslog.com/doc/configuration/modules/impstats.html> Format type default mandatory obsolete legacy directive word legacy no none New in version 8.16.0. Specifies the format of emitted stats messages. The default of “legacy” is compatible with pre v6-rsyslog. The other options provide support for structured formats (note the “cee” is actually “project lumberjack” logging). The json-elasticsearch format supports the broken ElasticSearch JSON implementation. ES 2.0 no longer supports valid JSON and disallows dots inside names. The “json-elasticsearch” format option replaces those dots by the bang (“!”) character. So “discarded.full” becomes “discarded!full”. Options: json/json-elasticsearch/cee/legacy David Lang On Mon, 20 Jan 2025, Redbourne,Michael via rsyslog wrote: > Date: Mon, 20 Jan 2025 01:01:47 +0000 > From: "Redbourne,Michael via rsyslog" <rsyslog@lists.adiscon.com> > To: "Redbourne,Michael via rsyslog" <rsyslog@lists.adiscon.com> > Cc: "Redbourne,Michael" <michael.redbou...@bulletproofsi.com> > Subject: [rsyslog] Building Custom pstats Template > > Hey Folks, > > I'm wondering if anyone has implemented (or knows how to) implement their own > pstats template? The current formatting works well for human-readable > monitoring but isn't so good for machine-based monitoring. What I want to do > is essentially transform the current format json to a single line of properly > formatted JSON that I can do LLD (low level discovery) on in our SAM. This is > necessary so I can tell SAM to dynamically add or remove entries anytime it > sees or doesn't see [a given item], as compared to the last entries. I'm > aware of the 'bracketing' option, however, I can't get my SAM to play nicely > with multi-line log matching, not to mention the post-processing > (pre-processing) needed would be a bit of a nightmare to handle. > > The current pstats formatting looks like this: > Sun Jan 19 23:57:54 2025: { "name": "action-15-builtin:omfile", "origin": > "core.action", "processed": 32079, "failed": 0, "suspended": 0, > "suspended.duration": 0, "resumed": 0 } > [...] > Sun Jan 19 23:57:54 2025: { "name": "action-16-builtin:omfile", "origin": > "core.action", "processed": 0, "failed": 0, "suspended": 0, > "suspended.duration": 0, "resumed": 0 } > > > I want to transform it to something like this: > { > "time": "Sun Jan 19 23:40:26 2025", > "host": "$HOSTNAME", > "stats": [ > { [...] }, > { "name": "action-15-builtin:omfile", "origin": "core.action", > "processed": 32079, "failed": 0, "suspended": 0, "suspended.duration": 0, > "resumed": 0 }, > { "name": "action-16-builtin:omfile", "origin": "core.action", > "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, > "resumed": 0 }, > { [...] } > ] > } > > I put it in multi-line form (for us humans), but the actual log one would a > single line for processing purposes > ________________________________________ > This e-mail communication (including any or all attachments) is intended only > for the use of the person or entity to which it is addressed and may contain > confidential and/or privileged material. If you are not the intended > recipient of this e-mail, any use, review, retransmission, distribution, > dissemination, copying, printing, or other use of, or taking of any action in > reliance upon this e-mail, is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete the original and any > copy of this e-mail and any printout thereof, immediately. If you have any > questions or concerns, please contact our Customer Service Desk at > 1-877-274-2349. Your co-operation is appreciated. > > Le présent courriel (y compris toute pièce jointe) s'adresse uniquement à son > destinataire, qu'il soit une personne ou un organisme, et pourrait comporter > des renseignements privilégiés ou confidentiels. Si vous n'êtes pas le > destinataire du courriel, il est interdit d'utiliser, de revoir, de > retransmettre, de distribuer, de disséminer, de copier ou d'imprimer ce > courriel, d'agir en vous y fiant ou de vous en servir de toute autre façon. > Si vous avez reçu le présent courriel par erreur, prière de communiquer avec > l'expéditeur et d'éliminer l'original du courriel, ainsi que toute copie > électronique ou imprimée de celui-ci, immédiatement. Si vous avez des > questions ou des préoccupations, veuillez contacter notre centre de service à > la clientèle au 1-877-274-2349. Nous sommes reconnaissants de votre > collaboration. > ________________________________________ > _______________________________________________ > rsyslog mailing list > https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7C9781aa28dc084468627308dd38ee6516%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638729318758942409%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gNipllHXQ1gqvgzw1qTsQIVZNCLnSG2MKdnvYLi0FOQ%3D&reserved=0<https://lists.adiscon.net/mailman/listinfo/rsyslog> > https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7C9781aa28dc084468627308dd38ee6516%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638729318758956309%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=e0voCn8CQ8utb1XenUEOnDVJ8iFem48E8WUnfq45YwA%3D&reserved=0<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow > https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7C9781aa28dc084468627308dd38ee6516%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638729318758968361%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=%2BSXJKV2srCDSajX%2B8mKhj3NbDqwD3r%2BVIHzKvhN%2BTRc%3D&reserved=0<https://twitter.com/rgerhards> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. >
_______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
