nothing in the config involves kafka, you you also need to provide all the files
that are included.
David Lang
On Mon, 27 Jan 2020, Nino Bosteels via rsyslog wrote:
Date: Mon, 27 Jan 2020 13:01:16 +0000
From: Nino Bosteels via rsyslog <[email protected]>
To: rsyslog-users <[email protected]>
Cc: Nino Bosteels <[email protected]>
Subject: Re: [rsyslog] Disconnect
My bad, this is thus the relevant config file:
# /etc/rsyslog.conf Configuration file for rsyslog.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#################
#### MODULES ####
#################
module(load="imuxsock") # provides support for local system logging
module(load="imklog") # provides kernel logging support
#module(load="immark") # provides --MARK-- message capability
# provides UDP syslog reception
#module(load="imudp")
#input(type="imudp" port="514")
# provides TCP syslog reception
#module(load="imtcp")
#input(type="imtcp" port="514")
###########################
#### GLOBAL DIRECTIVES ####
###########################
#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
#
# Set the default permissions for all log files.
#
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog
#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
###############
#### RULES ####
###############
#
# First some standard log files. Log by facility.
#
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
#cron.* /var/log/cron.log
daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
lpr.* -/var/log/lpr.log
mail.* -/var/log/mail.log
user.* -/var/log/user.log
#
# Logging for the mail system. Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info<http://mail.info> -/var/log/mail.info
mail.warn -/var/log/mail.warn
mail.err /var/log/mail.err
#
# Some "catch-all" log files.
#
*.=debug;\
auth,authpriv.none;\
news.none;mail.none -/var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none -/var/log/messages
#
# Emergencies are sent to everybody logged in.
#
*.emerg :omusrmsg:*
From: rsyslog <[email protected]> on behalf of Nino Bosteels via
rsyslog <[email protected]>
Reply to: rsyslog-users <[email protected]>
Date: Friday, 24 January 2020 at 16:59
To: "[email protected]" <[email protected]>
Cc: Nino Bosteels <[email protected]>
Subject: [rsyslog] Disconnect
Dear list,
We’ve configured rsyslog to log to kafka on our Debian jessie and stretch
systems, though we’re getting a lot of level 3 (error) Disconnect messages:
omkafka: kafka message <hereisanip>:9092/11: Receive failed: Disconnected [v8.24.0
try http://www.rsyslog.com/e/2422<http://www.rsyslog.com/e/2422>
omkafka: kafka message <hereisanip>:9092/10: Receive failed: Disconnected [v8.24.0
try http://www.rsyslog.com/e/2422<http://www.rsyslog.com/e/2422>
omkafka: kafka message kafka.local.<hereisadomainname>:9092/bootstrap: Receive
failed: Connection reset by peer [v8.23.0 try
http://www.rsyslog.com/e/2422<http://www.rsyslog.com/e/2422> ]
Everything comes from the rsyslog repos (packages) > rsyslog rsyslog-kafka
The kafka config:
module(load="omkafka")
template(name="json_lines" type="string"
string="{\"@timestamp\":\"%TIMESTAMP:::date-rfc3339%\",\"hostname\":\"%HOSTNAME%\",\"programname\":\"%programname%\",\"syslogfacility-text\":\"%syslogfacility-text%\",\"syslogseverity-text\":\"%syslogseverity-text%\",\"syslogseverity\":%syslogseverity%,\"syslogfacility\":%syslogfacility%,\"app-name\":\"%app-name%\",\"message\":\"%msg:::json%\",\"Region\":\"eu-west-1\",\"Environment\":\"alphasearch\",\"AvZone\":\"NA\",\"Service\":\"raas\",\"Subgroup\":\"activity\",\"Colour\":\"NA\"}"
)
*.warn,*.err,*.crit,*.alert,*.emerg action(
broker=["kafka.local. <hereisadomainname>:9092"]
type="omkafka"
topic="syslog_messages"
template="json_lines"
)
I am very grateful if you could help us out!
Nino
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow
https://twitter.com/rgerhards<https://twitter.com/rgerhards>
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
