On 05/11/2013 11:12 PM, David Lang wrote:
On Sat, 11 May 2013, Erik Steffl wrote:
On 05/10/2013 06:58 PM, David Lang wrote:
On Fri, 10 May 2013, Erik Steffl wrote:
On Ubuntu (uses upstart):
I would like to have a dynamically created data as part of rsyslog
(essentially host id that I don't know before the host is
created/started).
What is the best way to achieve this? I see that there is a way to
get environment variable in rsyslog config (using getenv) but since
rsyslog is started by init it has no env variables (i.e. the variables
in /etc/environment or /etc/profile.d/* are not set).
There is /etc/default/rsyslog but I would like to avoid changing
system files plus not sure how to make sure it's changed before the
system starts (it's a shell script so I guess I could add a line there
HOST_ID=$(getHostId))
thanks!
Currently, the only way to set a variable that rsyslog would use across
the board would be to modify the config file (or an included config
file) and then restart rsyslog.
But I'm not understanding what it is that you are trying to do.
Rsyslog doesn't have any internal variable for the system hostname, it
asks the OS.
It can extract data from logs and use it, but that is a per log message
thing, not something you can set to use with future log messages
(although, that concept has been discussed)
With a better understanding of the bigger problem (as opposed to the
narrow question of how to set a variable inside rsyslog), it may be
possible to come up with some other solution.
we plan to use rsyslog on our hosts that are automatically created
(EC2) and send these logs to a remote location (aggregate them
somewhere).
The messages that are being sent out should have some kind of host
identification. I think the best way to identify EC2 host is EC2
public ip (I can get that by running ec2metadata).
Hostname of EC2 hosts is not very useful (example of what hostname
command prints: domU-12-31-39-0A-50-42)
This should be hostname or IP of the host where rsyslog is running
(i.e. not from the received message)
Ok, that's actually easier
so you have a couple of possibilities
1. you can set the hostname as part of your startup config that you pass
to the EC2 instance.
don't think that's possible with autoscaling groups, it's just an
image and autoscaling group terminates and creates instances as needed.
Lot of our hosts are in autoscaling groups.
2. on the system receiving the logs from these systems, you can change
the template that it uses to store or forward the logs to have
%fromhost-ip% instead of %hostname% and it will log the IP address that
the log comes from (if you are logging inside a VPC you will get the
internal address.
not sure how good internal address is to identify the machine, but
that might work. However we plan to use Flume to aggregate logs (cause
it can write to S3 and has fairly flexible plugin system to write to
different destinations). Was thinking of this as last resort solution
just can't find how to get incoming IP from Flume's syslogTcp plugin.
but I would really like the rsyslog to identify machine in case we
later on decide to forward this in a more complex manner...
3. before you start rsyslog, run a process that looks up the EIP and
sets the hostname based off of that (or otherwise determines what
hostname to use) so that the hostname that rsyslog gets from the system
will be meaningful to you
that might work just not sure how to make sure the process that sets
hostname runs before rsyslog (using adiscon packages with upstart script).
what to set hostname to though? it shouldn't be completely bogus
(e.g. external.ip.nnn.nnn.nnn.nnn) because then the host would not be
accessible via that name (I think EX2 sets it so that the hostname is
valid in the given region)
now, if the systems are behind a load balancer, they won't all have
EIPs, and if they are logging through NAT you won't see their real IPs
as the fromhost-ip (besides, the non-EIP ips of the box are not much
better than random hostnames)
So I woul either make the hostname be something you pass in at startup
time, or have a script that runs very early in the boot that decides
what you want the hostname value to be and sets that before you start
rsyslog.
do you think these will work for you?
seems like creating a file in /etc/rsyslog.d would work, just need to
make sure the file generation runs before rsyslog starts or that script
that creates it also restarts rsyslog.
alternatively adding a line to /etc/default/rsyslog that exports some
variable (I see that file is sourced in /etc/init/rsyslog) would also
work I guess.
erik
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
