No interruption to the rsyslog service, rsyslog service never goes down.
Levi Wilbert HPC & Linux Systems Administrator ARCC - Division of Research and Economic Development Information Technology Ctr 226 1000 E. University Avenue, Laramie, WY 82071-200 ________________________________ From: rsyslog <rsyslog-boun...@lists.adiscon.com> on behalf of Michael A Hawkins via rsyslog <rsyslog@lists.adiscon.com> Sent: Wednesday, February 5, 2025 12:19 PM To: rsyslog-users <rsyslog@lists.adiscon.com> Cc: Michael A Hawkins <mhawk...@wantegrity.com> Subject: Re: [rsyslog] Rsyslog Losing Messages ◆ This message was sent from a non-UWYO address. Please exercise caution when clicking links or opening attachments from external sources. Or logrotate could be stopping and restarting the rsyslog process? Check to see how long rsyslog has been running. -- Wantegrity Inc. Michael A Hawkins, President Stamford, CT 06902 USA Mobile: 203-550-5502 On Wed, 2025-02-05 at 09:55 -0800, David Lang via rsyslog wrote: > we would need to see your config to have any idea what's going on (are you > sending via UDP, TCP, RELP, other??) > > But there are a few basic things that can go wrong > > 1. your VM server gets overloaded and stops scheduling your VM for a chunk of > time > > 2. you have a network problem (overload, interupption) that causes the packets > to not get through. > > 3. you have a flood of messages that are arriving faster than they can be > processed and your network buffers on your OS are overflowing (from your > description, this doesn't seem likely) > > for the network problem, this doesn't have to be on your VM server. > > For example, if you are sending the logs via UDP and have a router that gets > overloaded with a nightly backup, it is allowed to drop UDP packets. > > So look at what else is going on on the network/systems around the time of > your > log outage? is that when a nightly backup runs somewhere? other big batch job? > > David Lang > > On Wed, 5 Feb 2025, Levi Wilbert via rsyslog wrote: > > > Date: Wed, 5 Feb 2025 16:18:15 +0000 > > From: Levi Wilbert via rsyslog <rsyslog@lists.adiscon.com> > > To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com> > > Cc: Levi Wilbert <lwilb...@uwyo.edu> > > Subject: [rsyslog] Rsyslog Losing Messages > > > > Greetings, > > > > We are using Rsyslog on RHEL9.3 to gather logs in an environment of around > > 600 or so servers. All of these servers > > forward directly to our single syslog server, which then forwards the logs > > along to a mysql db (runs on a separate > > server), ELK stack, and to file locally on the system. > > > > I've noticed at around the same time each night, rsyslog begins dropping > > most of the incoming logs, and there is a > > gap where almost all logs simply aren't recorded. > > > > Network connectivity seems ok, as I am able to connect to the syslog > > server, connect to 514/TCP port (we also use > > UDP), and I can see logs hitting the server using tcpdump on the NIC. > > > > The syslog server is a virtualized server w/ 4 CPU cores and 8G of RAM. > > > > Would anyone have any idea on how to tune rsyslog to avoid these periods of > > log loss? > > > > > > > > Levi Wilbert > > HPC & Linux Systems Administrator > > ARCC - Division of Research and Economic Development > > Information Technology Ctr 226 > > 1000 E. University Avenue, Laramie, WY 82071-200 > > > > > > > > _______________________________________________ > > rsyslog mailing list > > https://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE > > UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE > UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
