I have made some progress in getting it to work, I removed the leading
space in the rule and it now matches. I totally missed that on log output.
rule=:%kerntime:word% Denied: IN=%in:word% OUT=%out:word% MAC=%mac:word%
SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% TOS=%tos:word%
PREC=%prec:word% TTL=%ttl:number% ID=%id:number% %DF:word%
PROTO=%proto:word% SPT=%src-port:number% DPT=%dst-port:number%
WINDOW=%window:number% RES=%res:word% %pkt-type:word% URGP=%urgp:number%
I did find how to use the iptables type but my results where not very good.
using this rule my output was has a lot of garbage output in the log, looks
like it wrote out binary data:
rule=:%kerntime:word% Denied: %iptables:iptables%
{ "IN": "eth0", "OUT": "", "MAC":
"00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00", "SRC": "10.10.10.10", "DST":
"10.10.10.11", "LEN": "60", "TOS": "0x10", "PREC": "0x00", "TTL": "64",
"ID": "10181", "DF": "[*PRESENT*]", "PROTO": "TCP", "SPT": "51765", "DPT":
"444", "WINDOW": "14600", "RES": "0x00", "SYN": "[*PRESENT*]", "URGP": "0",
"U)@00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00ü«^?xü«^?ÿÿÿÿÿÿÿÿ%\u0003\u0010LENü«^?<80>¸ü«^?%\u0002\u001060ü«^?¼ü«^?%LEN«^?xü«^?@µü«^?%60ü«^?xü«^?¹ü«^?%\býý\u0001p¾ü«^?<80>ºü«^?%\u0003\bLENü«^?<80>»ü«^?%`´ü«^?ð¿ü«^?ðbü«^?%\u0003\u0010TOSü«^?ÿÿÿÿÿÿÿÿ%\u0004\u00100x10«^?ÿÿÿÿÿÿÿÿ%TOS«^?xü«^?¸ü«^?%0x10^?xü«^?ÿÿÿÿÿÿÿÿ%\býý\u0001P¿ü«^?ÿÿÿÿÿÿÿÿ%\u0003\bTOSü«^?»ü«^?%":
"[*PRESENT*]", "kerntime": "[1151929.993763]" }
Using the full rule I get results like this, nice and clean.
{ "urgp": "0", "pkt-type": "SYN", "res": "0x00", "window": "14600",
"dst-port": "444", "src-port": "51954", "proto": "TCP", "DF": "DF", "id":
"31890", "ttl": "64", "prec": "0x00", "tos": "0x10", "len": "60", "dst-ip":
"10.10.10.10", "src-ip": "10.10.10.11", "mac":
"00:30:48:90:cc:a6:00:30:48:da:48:e8:08:00", "in": "eth0", "kerntime":
"[1152127.460873]" }
I will continue my journey using the other method for now.
On Tue, Jul 16, 2013 at 3:19 PM, Todd Mortensen <[email protected]>wrote:
> I am not sure, I honestly stole the examples from the sagan project.
>
> I just tried using the following to test.
>
> rule=: %kerntime:word% Denied: %iptables%
> rule=: [%kerntime:word%] Denied: %iptables%
>
> I also tried adding [] to the above full line as well. Still doesn't
> return any parsed values.
>
> I forgot to mention this is version 7.4.2 on ubuntu 12.04 from my ppa
>
> ii rsyslog-mmnormalize 7.4.2-0ubuntu1ppa2
>
>
>
>
> On Tue, Jul 16, 2013 at 3:12 PM, David Lang <[email protected]> wrote:
>
>> it looks to me like your ruleset is not matching the rule
>>
>> are you sure that %kerntime:word% includes the square brackets and it
>> shouldn't be [%kerntime:word%] instead?
>>
>> there is a special type 'iptables' that was created explicitly for
>> matching the name=value format of iptables rules.
>>
>> I think your rule can be simplified to something like:
>>
>> rule=: [%kerntime:word%] Denied: %iptables%
>>
>> David Lang
>>
>> On Tue, 16 Jul 2013, Todd Mortensen wrote:
>>
>> Date: Tue, 16 Jul 2013 14:53:36 -0700
>>> From: Todd Mortensen <[email protected]>
>>> Reply-To: rsyslog-users <[email protected]>
>>> To: rsyslog-users <[email protected]>
>>> Subject: [rsyslog] mmnormalize and iptable logs
>>>
>>>
>>> I am trying my first attempts at using mmnormalize.
>>>
>>> My goal is to parse out my iptable firewall logs into a cee format so I
>>> can
>>> then send them to elasticsearch.
>>>
>>> But so far my logs just show unparsed items. Can anyone shed any light
>>> to
>>> where I am going wrong here.
>>>
>>> Relevant sections from rsyslog.conf
>>>
>>> module(load="mmnormalize")
>>> action(type="mmnormalize" UseRawMsg="off"
>>> ruleBase="/etc/rsyslog.d/**normalize.rb")
>>>
>>> module(load="mmjsonparse")
>>>
>>>
>>> #template(name="cee" type="string" string="%$!all-json%\n")
>>> template(name="cee" type="string" string="%$!%\n")
>>>
>>> *.* { action (name="cee" type="omfile"
>>> Template="cee" file="/var/log/cee.log") }
>>>
>>> I have also tried a couple variations of this but they still don't match
>>>
>>> normalize.rb
>>> rule=: %kerntime:word% Denied: IN=%in:word% OUT= MAC=%mac:word%
>>> SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% TOS=%tos:word%
>>> PREC=%prec:word% TTL=%ttl:number% ID=%id:number% %DF:word%
>>> PROTO=%proto:word% SPT=%src-port:number% DPT=%dst-port:number%
>>> WINDOW=%window:number% RES=%res:word% %pkt-type:word% URGP=%urgp:number%
>>>
>>> I also tried to use the iptales type, but I am not able to find many docs
>>> on its usage.
>>>
>>> rule=: %kerntime:word% Denied: %iptables:iptables%
>>>
>>>
>>> Sample log line:
>>>
>>> [1084540.211910] Denied: IN=eth0 OUT=
>>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10
>>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF PROTO=TCP
>>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0
>>>
>>>
>>> This is what I see in my cee logfile and it shows unpared-data.
>>>
>>> # { "originalmsg": "[1084540.211910] Denied: IN=eth0 OUT=
>>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10
>>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF PROTO=TCP
>>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0 ", "unparsed-data":
>>> "[1084540.211910] Denied: IN=eth0 OUT=
>>> MAC=00:30:48:90:cc:a6:00:30:**48:da:48:e8:08:00 SRC=10.10.10.10
>>> DST=10.10.10.11 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=28843 DF PROTO=TCP
>>> SPT=44075 DPT=444 WINDOW=14600 RES=0x00 SYN URGP=0 " }
>>> ______________________________**_________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
>>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>> ______________________________**_________________
>> rsyslog mailing list
>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
