Found it - was a stupid admin error :D Rsyslog runs as: "11331 ? Sl 111:23 /sbin/rsyslogd -i /var/run/syslogd.pid -x
Logrotate says: kill -HUP `cat /var/run/syslog.pid` "syslog.pid" vs "syslogd.pid" - difference of a "d" Thanks for your patience and help. On Tue, Jun 25, 2013 at 1:06 PM, Xuri Nagarin <[email protected]> wrote: > On Tue, Jun 25, 2013 at 11:51 AM, David Lang <[email protected]> wrote: > >> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >> >> Yes, a manual HUP released them all but not instantaneously. Took a few >>> seconds. >>> >> >> given how many files you have open, It's not too surprising that it takes >> a little time >> >> A logrotate bug? >>> >> >> double check the contents (and timestamp) of /var/run/syslog.pid, it may >> not be what you are expecting >> > > /var/run/syslog.pid contents reflect the right PID number. What's more > funny is that if I run "watch -d 'sudo lsof | grep deleted | grep > rsyslog'", I can see the deleted log files growing in size. Logrotate > should never create a logfile.1 since rotate is set to 0. Right? > > > >> >> since the HUP does free the files, my suspicion is that rsyslog is not >> getting the HUP in the first place. This would point me at logrotate, or >> the way that logrotate is getting the PID >> > > Going to run logrotate with strace and see what shows up. > > Thanks. > > > > >> >> David Lang >> >> >> >> On Tue, Jun 25, 2013 at 11:38 AM, David Lang <[email protected]> wrote: >>> >>> If you manually do a kill -HUP 11311 do these files get released? >>>> >>>> >>>> David Lang >>>> >>>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>>> >>>> More fun info: >>>> >>>>> >>>>> $ sudo lsof -s | awk '$5 == "REG"' | sort -n -r -k 7,7 | head -n 50 | >>>>> grep >>>>> deleted >>>>> rsyslogd 11331 root 20w REG 9,2 >>>>> 117175430149 >>>>> 8847368 /var/log/joe/CISCO/app7/****logfile.1 (deleted) >>>>> >>>>> rsyslogd 11331 root 9w REG 9,2 >>>>> 53533745904 >>>>> 8839187 /var/log/joe/Microsoft/app1/****logfile.1 (deleted) >>>>> >>>>> rsyslogd 11331 root 10w REG 9,2 >>>>> 16268001760 >>>>> 8839237 /var/log/joe/Microsoft/app2/****logfile.1 (deleted) >>>>> >>>>> rsyslogd 11331 root 18w REG 9,2 >>>>> 1831944964 >>>>> 8847369 /var/log/joe/Microsoft/app3/****logfile.1 (deleted) >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Jun 25, 2013 at 12:10 PM, Xuri Nagarin <[email protected]> >>>>> wrote: >>>>> >>>>> Happening again: >>>>> >>>>>> >>>>>> $ sudo du -sh * >>>>>> 9.8M bin >>>>>> 26M boot >>>>>> 220K dev >>>>>> 6.9M etc >>>>>> 3.5G home >>>>>> 129M lib >>>>>> 22M lib64 >>>>>> 20K lost+found >>>>>> 8.0K media >>>>>> 8.0K mnt >>>>>> 6.4G opt >>>>>> du: cannot access `proc/2713/task/9576': No such file or directory >>>>>> du: cannot access `proc/9490/task/9490/fd/4': No such file or >>>>>> directory >>>>>> du: cannot access `proc/9490/task/9490/fdinfo/4'****: No such file or >>>>>> >>>>>> directory >>>>>> du: cannot access `proc/9490/fd/4': No such file or directory >>>>>> du: cannot access `proc/9490/fdinfo/4': No such file or directory >>>>>> 0 proc >>>>>> 116K root >>>>>> 13M sbin >>>>>> 4.0K selinux >>>>>> 4.0K srv >>>>>> 0 sys >>>>>> 296K tmp >>>>>> 1.3G usr >>>>>> 39G var >>>>>> >>>>>> $ sudo df -kh >>>>>> Filesystem Size Used Avail Use% Mounted on >>>>>> /dev/md2 913G 212G 655G 25% / >>>>>> tmpfs 16G 0 16G 0% /dev/shm >>>>>> /dev/md0 97M 31M 62M 34% /boot >>>>>> >>>>>> I am seeing the same baheviour on a second system. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 25, 2013 at 12:04 PM, Xuri Nagarin <[email protected]> >>>>>> wrote: >>>>>> >>>>>> My suspicion was something to do with dynamic file creation but >>>>>> looking >>>>>> >>>>>>> at the code, both static and dynamic file creation use the same >>>>>>> function of >>>>>>> file "open": >>>>>>> >>>>>>> From tools/omfile.c >>>>>>> ------------xxxxxxxxxxxxxxxxx-****-------------- >>>>>>> fd = open((char*) newFileName, >>>>>>> O_WRONLY|O_APPEND|O_CREAT|O_****NOCTTY|O_CLOEXEC, >>>>>>> 556 pData->fCreateMode); >>>>>>> ------------xxxxxxxxxxxxxxxxx-****-------------- >>>>>>> >>>>>>> >>>>>>> "O_APPEND" should take care of file being rotated while rsyslog is >>>>>>> trying >>>>>>> to write to it. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 25, 2013 at 11:47 AM, Xuri Nagarin <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Not a typo or error, in RHEL the rsyslogd start up script in init.d >>>>>>> has >>>>>>> >>>>>>>> an explicit variable "PIDFILE" set as: >>>>>>>> PIDFILE=/var/run/syslogd.pid >>>>>>>> >>>>>>>> :) >>>>>>>> >>>>>>>> I did not write the script, using whatever was bundled in the RPM I >>>>>>>> grabbed from Adiscon. >>>>>>>> >>>>>>>> To confirm: >>>>>>>> $ ps ax | grep rsyslog >>>>>>>> 11331 ? Sl 61:18 /sbin/rsyslogd -i /var/run/syslogd.pid -x >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 25, 2013 at 11:41 AM, Soham Chakraborty < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>>> >>>>>>>>> In the logrotate config, change it to /var/run/rsyslog.pid and >>>>>>>>> test. >>>>>>>>> On >>>>>>>>> a >>>>>>>>> quick skim, it looks like a typo. >>>>>>>>> >>>>>>>>> Soham >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jun 25, 2013 at 11:07 PM, David Lang <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> well, one thing that looks wrong is that logrotate is looking for >>>>>>>>> >>>>>>>>>> /var/run/syslog.pid, but on my systems the pid is in >>>>>>>>>> >>>>>>>>>> /var/log/rsyslog.pid >>>>>>>>> >>>>>>>>> >>>>>>>>>> are you sure that rsyslog is actually getting the HUP? >>>>>>>>>> >>>>>>>>>> can you try sending it a HUP manually and see if it closes the >>>>>>>>>> files? >>>>>>>>>> >>>>>>>>>> David Lang >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>>>>>>>>> >>>>>>>>>> Date: Tue, 25 Jun 2013 11:34:08 -0700 >>>>>>>>>> >>>>>>>>>> From: Xuri Nagarin <[email protected]> >>>>>>>>>>> Reply-To: rsyslog-users <[email protected]> >>>>>>>>>>> To: rsyslog-users <[email protected]> >>>>>>>>>>> Subject: Re: [rsyslog] HUP-ing rsyslog does not free up disk >>>>>>>>>>> space >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> The master conf file is simple with few additions that are >>>>>>>>>>> self-explanatory. The other conf in rsyslog.d, I have added >>>>>>>>>>> comments >>>>>>>>>>> inline >>>>>>>>>>> in the conf that explain the config logic. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> /etc/rsyslog.conf >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> module(load="impstats" interval="600" severity="7" >>>>>>>>>>> >>>>>>>>>>> log.syslog="off" /* need to turn log stream logging off! */ >>>>>>>>>>> log.file="/var/log/rsyslog-******stats.log") >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> $ModLoad imuxsock.so # Unix sockets >>>>>>>>>>> $ModLoad imklog.so # Kernel logger >>>>>>>>>>> $MainMsgQueueSize 1000000 >>>>>>>>>>> $OMFileIOBufferSize 512k >>>>>>>>>>> $MaxMessageSize 8k >>>>>>>>>>> $MainMsgQueueWorkerThreads 64 >>>>>>>>>>> $umask 0000 >>>>>>>>>>> $FileOwner joe >>>>>>>>>>> $FileGroup joe >>>>>>>>>>> $DirOwner joe >>>>>>>>>>> $DirGroup joe >>>>>>>>>>> $DirCreateMode 0755 >>>>>>>>>>> $FileCreateMode 0644 >>>>>>>>>>> >>>>>>>>>>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >>>>>>>>>>> $IncludeConfig /etc/rsyslog.d/*.conf >>>>>>>>>>> >>>>>>>>>>> $RuleSet local >>>>>>>>>>> *.info;mail.none;authpriv.******none;cron.none >>>>>>>>>>> /var/log/messages >>>>>>>>>>> authpriv.* >>>>>>>>>>> >>>>>>>>>>> /var/log/secure >>>>>>>>>> >>>>>>>>> >>>>>>>>> mail.* >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -/var/log/maillog >>>>>>>>>> >>>>>>>>> >>>>>>>>> cron.* >>>>>>>>>> /var/log/cron >>>>>>>>>> >>>>>>>>>>> *.emerg * >>>>>>>>>>> uucp,news.crit >>>>>>>>>>> >>>>>>>>>>> /var/log/spooler >>>>>>>>>> >>>>>>>>> >>>>>>>>> local7.* >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> /var/log/boot.log >>>>>>>>>> >>>>>>>>> >>>>>>>>> $DefaultRuleset local >>>>>>>>>> >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> >>>>>>>>>>> /etc/rsyslog.d/cef.conf >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> # Senders are four Arcsight Logger devices that send logs in CEF >>>>>>>>>>> >>>>>>>>>>> format >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> #template for writing CEF formatted logs >>>>>>>>>>> template(name="cefdynfile" type="string" >>>>>>>>>>> string="/var/log/joe/%$!******vendor%/%$!product%/logfile") >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> #template for writing logs from non-CEF sources >>>>>>>>>>> template(name="noncefdynfile" type="string" >>>>>>>>>>> string="/var/log/joe/noncef/%******hostname%/%programname%/***** >>>>>>>>>>> * >>>>>>>>>>> >>>>>>>>>>> logfile") >>>>>>>>>>> >>>>>>>>>>> ruleset(name="tcpcef") { >>>>>>>>>>> >>>>>>>>>>> #CEF uses the pipe delimiter, fields 2 and 3 are product vendor >>>>>>>>>>> and >>>>>>>>>>> product >>>>>>>>>>> type respectively >>>>>>>>>>> set $!vendor = field($msg, 124, 2); >>>>>>>>>>> set $!product = field($msg, 124, 3); >>>>>>>>>>> >>>>>>>>>>> # Rules to write CEF formatted logs to disk and send logs by app >>>>>>>>>>> >>>>>>>>>>> type to >>>>>>>>>> >>>>>>>>> >>>>>>>>> their flume destinations that are listening on the same box >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> if $syslogtag=="CEF:" then { action (type="omfile" >>>>>>>>>>> ASyncWriting="on" >>>>>>>>>>> IOBufferSize="8192K" FileOwner="joe" FileGroup="joe" >>>>>>>>>>> DirOwner="joe" >>>>>>>>>>> DirGroup="joe" DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>>> DynaFile="cefdynfile") } >>>>>>>>>>> if $!product == "app1" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5161" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app2" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5146" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app3" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5172" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app4" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5162" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app5" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5166" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app6" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5163" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app7" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5164" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app8" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5177" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app9" then { action (type="omfwd" >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>>> Port="5144" Protocol="tcp") stop } >>>>>>>>>>> if $!product == "app10" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5145" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app11" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5148" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app12" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5180" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app13" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5147" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app14" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5149" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app15" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5150" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app16" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5151" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app17" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5152" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app18" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5153" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app19" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5155" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app20" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5156" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app21" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5157" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app22" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5158" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app23" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5159" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app24" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5160" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app25" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5178" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app26" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5165" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app27" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5165" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app28" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5167" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app29" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5167" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app30" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5179" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app31" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5169" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app32" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5170" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app33" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5171" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app34" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5174" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app35" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5173" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app36" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5175" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app37" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5176" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app38" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5154" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> if $!product == "app39" then { action (type="omfwd" >>>>>>>>>>> >>>>>>>>>>> Target="127.0.0.1" >>>>>>>>>> >>>>>>>>> >>>>>>>>> Port="5181" Protocol="tcp") stop } >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> # Unfortunately, the four Arcsight Loggers also send us garbage >>>>>>>>>>> so >>>>>>>>>>> whatever >>>>>>>>>>> could not be parsed/classified correctly by the rules above, gets >>>>>>>>>>> >>>>>>>>>>> parked >>>>>>>>>> >>>>>>>>> >>>>>>>>> in >>>>>>>>>> >>>>>>>>>>> a catchall file. >>>>>>>>>>> >>>>>>>>>>> if $fromhost-ip == '10.1.1.100' or $fromhost-ip == '10.1.1.101' >>>>>>>>>>> or >>>>>>>>>>> $fromhost-ip == '10.1.1.102' or $fromhost-ip == '10.1.1.103' >>>>>>>>>>> then { >>>>>>>>>>> >>>>>>>>>>> action >>>>>>>>>> >>>>>>>>> >>>>>>>>> (type="omfile" FileOwner="joe" FileGroup="joe" DirOwner="joe" >>>>>>>>>> >>>>>>>>>>> DirGroup="joe" DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>>> file="/var/log/joe/fallback/******logfile") >>>>>>>>>>> >>>>>>>>>>> & action (type="omfwd" Target="127.0.0.1" Port="5182" >>>>>>>>>>> >>>>>>>>>>> Protocol="tcp" >>>>>>>>>> >>>>>>>>> >>>>>>>>> ) >>>>>>>>>> >>>>>>>>>>> stop } >>>>>>>>>>> >>>>>>>>>>> # Take care of all the non-CEF / BSD Syslog formatted streams >>>>>>>>>>> coming >>>>>>>>>>> >>>>>>>>>>> in >>>>>>>>>> >>>>>>>>> >>>>>>>>> else { >>>>>>>>>> >>>>>>>>>>> action (type="omfile" ASyncWriting="on" IOBufferSize="8192K" >>>>>>>>>>> FileOwner="joe" FileGroup="joe" DirOwner="joe" DirGroup="joe" >>>>>>>>>>> DirCreateMode="0755" FileCreateMode="0644" >>>>>>>>>>> DynaFile="noncefdynfile") >>>>>>>>>>> & action (type="omfwd" Target="127.0.0.1" Port="5182" >>>>>>>>>>> >>>>>>>>>>> Protocol="tcp") >>>>>>>>>> >>>>>>>>> >>>>>>>>> } >>>>>>>>>> >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> module(load="imtcp" ) # needs to be done just once >>>>>>>>>>> input(type="imtcp" port="514" ruleset="tcpcef") >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Here's the logrotate code: >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> /var/log/joe/*/*/* >>>>>>>>>>> /var/log/joe/*/*/*/* >>>>>>>>>>> { >>>>>>>>>>> missingok >>>>>>>>>>> size 1G >>>>>>>>>>> rotate 0 >>>>>>>>>>> sharedscripts >>>>>>>>>>> postrotate >>>>>>>>>>> if [ -f /var/run/syslog.pid ]; then \ >>>>>>>>>>> kill -HUP `cat /var/run/syslog.pid`; \ >>>>>>>>>>> fi; >>>>>>>>>>> endscript >>>>>>>>>>> } >>>>>>>>>>> ==============================******==========================** >>>>>>>>>>> ==** >>>>>>>>>>> >>>>>>>>>>> ==** >>>>>>>>>>> =========== >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jun 25, 2013 at 10:12 AM, David Lang <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> On Tue, 25 Jun 2013, Xuri Nagarin wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On RHEL 6.2 64-bit, I have Rsyslog 7.4.1 (actually, the issue >>>>>>>>>>>> has >>>>>>>>>>>> existed >>>>>>>>>>>> >>>>>>>>>>>> for earlier versions too). >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> After logrotation, the logrotate script HUPs the rsyslogd pid >>>>>>>>>>>>> but >>>>>>>>>>>>> >>>>>>>>>>>>> the >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> disk >>>>>>>>>> >>>>>>>>>>> space doesn't free up until I restart rsyslog. After a few hours >>>>>>>>>>>>> >>>>>>>>>>>>> "df" >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> reports a full filesystem whereas "ls" shows much smaller file >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> sizes. In >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> between this confusion, rsyslog seems to stop receiving log >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>> streams. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>> Is there a way for rsyslog to switch file handles when it is >>>>>>>>>>>>> >>>>>>>>>>>>> HUP-ed? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>>> rsyslog is already supposed to close and re-open files when >>>>>>>>>>>>> it's >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> HUP-ed, >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> and it seems to be working for me and many others. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> could you send your config file so we can see if there is >>>>>>>>>>>> anything >>>>>>>>>>>> unusual >>>>>>>>>>>> in it? >>>>>>>>>>>> >>>>>>>>>>>> David Lang >>>>>>>>>>>> ______________________________********_________________ >>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog<http://lists.adiscon.net/******mailman/listinfo/rsyslog> >>>>>>>>>>>> <http**://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>>>> > >>>>>>>>>>>> < >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> <http:**//lists.adiscon.net/****mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/**listinfo/rsyslog> >>>>>>>>>> <htt**p://lists.adiscon.net/mailman/****listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>>> < >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://www.rsyslog.com/********professional-services/<http://www.rsyslog.com/******professional-services/> >>>>>>>>>>>>> <http://**www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>>>>> > >>>>>>>>>>>>> >>>>>>>>>>>> < >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> <http://**www.rsyslog.com/****professional-**services/<http://www.rsyslog.com/**professional-**services/> >>>>>>>>>> <http:**//www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>>> < >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>>>> >>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>>>>>> a >>>>>>>>>>>> >>>>>>>>>>>> myriad >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> you >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> DON'T LIKE THAT. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> ______________________________******_________________ >>>>>>>>>>>> >>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>>> > >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>> >>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>>> >>>>>>>>>>> myriad >>>>>>>>>> >>>>>>>>> >>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> you >>>>>>>>>> >>>>>>>>> >>>>>>>>> DON'T LIKE THAT. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ______________________________******_________________ >>>>>>>>>>> >>>>>>>>>>> rsyslog mailing list >>>>>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>>> > >>>>>>>>>> < >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>>> > >>>>>>>>>> < >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>> >>>>>>>>>> myriad >>>>>>>>> >>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>> you >>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>> >>>>>>>>>> ______________________________****_________________ >>>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>>> > >>>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>>> > >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>> POST if >>>>>>>>> you DON'T LIKE THAT. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> ______________________________****_________________ >>>>>> >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>> > >>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
