Or logrotate could be stopping and restarting the rsyslog process? Check to see how long rsyslog has been running.
-- Wantegrity Inc. Michael A Hawkins, President Stamford, CT 06902 USA Mobile: 203-550-5502 On Wed, 2025-02-05 at 09:55 -0800, David Lang via rsyslog wrote: > we would need to see your config to have any idea what's going on (are you > sending via UDP, TCP, RELP, other??) > > But there are a few basic things that can go wrong > > 1. your VM server gets overloaded and stops scheduling your VM for a chunk of > time > > 2. you have a network problem (overload, interupption) that causes the > packets > to not get through. > > 3. you have a flood of messages that are arriving faster than they can be > processed and your network buffers on your OS are overflowing (from your > description, this doesn't seem likely) > > for the network problem, this doesn't have to be on your VM server. > > For example, if you are sending the logs via UDP and have a router that gets > overloaded with a nightly backup, it is allowed to drop UDP packets. > > So look at what else is going on on the network/systems around the time of > your > log outage? is that when a nightly backup runs somewhere? other big batch job? > > David Lang > > On Wed, 5 Feb 2025, Levi Wilbert via rsyslog wrote: > > > Date: Wed, 5 Feb 2025 16:18:15 +0000 > > From: Levi Wilbert via rsyslog <rsyslog@lists.adiscon.com> > > To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com> > > Cc: Levi Wilbert <lwilb...@uwyo.edu> > > Subject: [rsyslog] Rsyslog Losing Messages > > > > Greetings, > > > > We are using Rsyslog on RHEL9.3 to gather logs in an environment of around > > 600 or so servers. All of these servers > > forward directly to our single syslog server, which then forwards the logs > > along to a mysql db (runs on a separate > > server), ELK stack, and to file locally on the system. > > > > I've noticed at around the same time each night, rsyslog begins dropping > > most of the incoming logs, and there is a > > gap where almost all logs simply aren't recorded. > > > > Network connectivity seems ok, as I am able to connect to the syslog > > server, connect to 514/TCP port (we also use > > UDP), and I can see logs hitting the server using tcpdump on the NIC. > > > > The syslog server is a virtualized server w/ 4 CPU cores and 8G of RAM. > > > > Would anyone have any idea on how to tune rsyslog to avoid these periods of > > log loss? > > > > > > > > Levi Wilbert > > HPC & Linux Systems Administrator > > ARCC - Division of Research and Economic Development > > Information Technology Ctr 226 > > 1000 E. University Avenue, Laramie, WY 82071-200 > > > > > > > > _______________________________________________ > > rsyslog mailing list > > https://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE > > UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE > UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
