[toaster] Issues with qmailmrtg7 statistics
Hi all,
I have a system set-up according to the guide, with qmail logging using
multilog to /var/log/qmail and using qmailmrtg7 for creating graphs.
I've been investigating a few peaks in my messages/hour graph..
Looking at the qmailmrtg7 code for counting messages... if increments a
counter for every line that has a success, failure or deferral.
Then at the end it returns two numbers, each multiplied by 12...
(success*12, (failure+success)*12).
It took me a few minutes to figure this out; it only counts messages
from the last 300 seconds (5 minutes), and as we graph messages per hour
it multiplies by 12 to get an hours worth... This turns 54 messages in
the last 5 minutes into a plot of 648 messages/hour.
So when interpreting the 5 minute average graph, you have to look at any
peak and divide by 12, to get the actual number of messages sent in the
5 minutes it represents
Wouldn't it be of more use to graph messages/5 minutes or a running
total of messages per hour based on the actual previous 12 5-minute totals?
Has anyone patched qmailmrtg7.c to allow it to take into account CHKUSER
rejecting messages when counting SMTP totals? At the moment it only
counts tcpserver and rblsmtpd deny messages.
Would simply adding the following(before line 291) to qmailmrtg7.c
cover this?
if ((tmpstr1 = strstr(TmpBuf, "CHKUSER rejected rcpt"))!=NULL) {
--tallow;
++tdeny;
}
Regards,
Tom Beardshall
B2B Internet Services Ltd
Re: [toaster] Simscan Spam Rejection
Hank wrote on 27/09/2005 15:19: That looks like a SpamAssassin message, not simscan. -Hank On 9/27/05, Carlos J. Muentes <[EMAIL PROTECTED]> wrote: Hey All, I'm successfully running the latest toaster. We recently transfered our email from an external hosting company to our in-house email server (with fresh toaster install). We're getting alot of spam on a couple of the accounts, and this is what it says in the email body: "Spam detection software, running on the system "server2.domain.com", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see [EMAIL PROTECTED] for details." My question is, I thought Simscan would reject email that it detected as Spam? What am I missing? P.S. Sorry for masking the domains, I hate when people do that to me lol What does it say in the file /var/qmail/control/simcontrol ? Mine is.. :clam=yes,spam=yes,spam_passthru=yes,attach=.vbs:.lnk:.scr:.wsh:.hta:.pif The configure options included: -spam-passthru=y|nPass spam email thru or reject Default: disable (reject) Try changing to spam_passthru=no And run /var/qmail/bin/simscanmk to rebuild the simcontrol.cdb Regards, Tom Beardshall
Re: [toaster] Simscan Spam Rejection
You can putting the following at the top of /service/qmail-smtpd/run (after #!/bin/sh) and looking for any errors SIMSCAN_DEBUG=2 export SIMSCAN_DEBUG And restart with svc -h /service/qmail-smtpd It may give you an indication of any errors in your qmail-smtpd log file. Carlos J. Muentes wrote on 27/09/2005 16:04: Spam passthrough was already set to no, but I lowered the spam hits and did the simscanmk. Hopefully that will fix it. Here is my simcontrol :clam=yes,spam=yes,spam_passthru=no, spam_hits=6,attach=.mp3:.vbs:.lnk:.scr:.wsh:.hta:.pif Sloath wrote: Hank wrote on 27/09/2005 15:19: That looks like a SpamAssassin message, not simscan. -Hank On 9/27/05, Carlos J. Muentes <[EMAIL PROTECTED]> wrote: Hey All, I'm successfully running the latest toaster. We recently transfered our email from an external hosting company to our in-house email server (with fresh toaster install). We're getting alot of spam on a couple of the accounts, and this is what it says in the email body: "Spam detection software, running on the system "server2.domain.com", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see [EMAIL PROTECTED] for details." My question is, I thought Simscan would reject email that it detected as Spam? What am I missing? P.S. Sorry for masking the domains, I hate when people do that to me lol What does it say in the file /var/qmail/control/simcontrol ? Mine is.. :clam=yes,spam=yes,spam_passthru=yes,attach=.vbs:.lnk:.scr:.wsh:.hta:.pif The configure options included: -spam-passthru=y|nPass spam email thru or reject Default: disable (reject) Try changing to spam_passthru=no And run /var/qmail/bin/simscanmk to rebuild the simcontrol.cdb Regards, Tom Beardshall
Re: [toaster] spam/virus-check outsourcing ???
Michael D Schleif wrote on 13/11/2005 07:57:
* Tom Collins <[EMAIL PROTECTED]> [2005:10:27:22:30:26-0700] scribed:
On Oct 27, 2005, at 5:02 PM, Michael D Schleif wrote:
We have a client that runs Groupwise on a Netware server on their LAN.
We manage everything on this LAN; and, we handle their DNS for their
domains.
We want to scrub all of their _incoming_ email on our remote toaster,
doing what it does best, then forward *ALL* of their email onto that
Groupwise box.
Use simscan. Make your server the only MX for the domain. Put the
domain in /var/qmail/control/rcpthosts (or morercpthosts and run
qmail-newmrh). Do *not* add their domain to locals or virtualdomains.
Add an entry in /var/qmail/control/smtproutes to route the scanned mail
to their Groupwise server.
--
Tom Collins - [EMAIL PROTECTED]
QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/
You don't need a laptop to troubleshoot high-speed Internet:
sniffter.com
I finally got back on this one. Yes, setup was simple; and, yes,
messages _do_ go through our toaster, and they _do_ end up in the
Groupwise server.
This setup does *not* appear to work. I am still waiting for the
Groupwise admin to send to me the complete headers for the test
messages; but, I am told that *none* of these test messages include
spamassassin headers.
In fact, the following does *not* show _any_ activity for test messages:
tail -f /var/log/{cl,sp}amd/current
I am not clear which process comes first,
/var/qmail/control/smtproutes
or simscan ???
What do you think?
The diagram of the found at http://www.nrg4u.com/ does a very good job
of explaining the qmail process
man qmail-control will also tell what each control file is used by.
In your case:
qmail-smtp check incoming email envelope recipient addresses against the
contents of rcpthosts to see if it should be accepted.
It then passes the email onto qmail-queue, unless $QMAILQUEUE is set in
which case it will invoke the program named.
A simple /home/vpopmail/etc/tcp.smtp should include
:allow,QMAILQUEUE="/var/qmail/bin/simscan"
The message is then passed to qmail-send which determines whether the
message is local or remote by the presence( or lack of ) the
envelope recipient in the virtualdomains or locals file.
If it is determined to be remote, it is passed to qmail-rspawn and then
qmail-remote.
Qmail-remote checks smtproutes for the domain and sends the message as
appropriate.
The short answer was, simscan comes first.
You turn on debug in simscan by putting the following in
/service/qmail-smtpd/run before the call to exec.
SIMSCAN_DEBUG=2
export SIMSCAN_DEBUG
The other thing to check is the /var/qmail/control/simcontrol file
Make sure clam and spam are set to yes, otherwise they won't be invoked.
Regards
Tom Beardshall
Re: [toaster] bounce from: whence comes this address ???
Michael D Schleif wrote on 11/11/2005 05:51: * Rick Widmer <[EMAIL PROTECTED]> [2005:11:10:19:38:31-0700] scribed: Michael D Schleif wrote: I have a toaster on czar.platinumaire.net. Bounces show a different system: From: [EMAIL PROTECTED] This is one of three (3) retired toasters that are now consolidated/migrated to czar. Where does this bounce-from: come from? Why is it the wrong system? What do you think? Look in /var/qmail/control/me Don't forget to restart/reload qmail-send after you change it. (qmailctl restart or qmailctl reload) Rick I know; but, that's not it: # uname -a ; cat /var/qmail/control/me Linux czar 2.6.8-2-386 #1 Thu May 19 17:40:50 JST 2005 i686 GNU/Linux czar.platinumaire.net Do the files bouncefrom and bouncehost exist in your qmail control dir? And are set to something? Tom Beardshall
Re: [toaster] spam/virus-check outsourcing ???
Michael D Schleif wrote on 15/11/2005 00:01:
* Sloath <[EMAIL PROTECTED]> [2005:11:14:18:44:49+0100] scribed:
Michael D Schleif wrote on 13/11/2005 07:57:
* Tom Collins <[EMAIL PROTECTED]> [2005:10:27:22:30:26-0700] scribed:
On Oct 27, 2005, at 5:02 PM, Michael D Schleif wrote:
We have a client that runs Groupwise on a Netware server on their LAN.
We manage everything on this LAN; and, we handle their DNS for their
domains.
We want to scrub all of their _incoming_ email on our remote toaster,
doing what it does best, then forward *ALL* of their email onto that
Groupwise box.
Use simscan. Make your server the only MX for the domain. Put the
domain in /var/qmail/control/rcpthosts (or morercpthosts and run
qmail-newmrh). Do *not* add their domain to locals or virtualdomains.
Add an entry in /var/qmail/control/smtproutes to route the scanned mail
to their Groupwise server.
--
Tom Collins - [EMAIL PROTECTED]
QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/
You don't need a laptop to troubleshoot high-speed Internet:
sniffter.com
I finally got back on this one. Yes, setup was simple; and, yes,
messages _do_ go through our toaster, and they _do_ end up in the
Groupwise server.
This setup does *not* appear to work. I am still waiting for the
Groupwise admin to send to me the complete headers for the test
messages; but, I am told that *none* of these test messages include
spamassassin headers.
In fact, the following does *not* show _any_ activity for test messages:
tail -f /var/log/{cl,sp}amd/current
I am not clear which process comes first,
/var/qmail/control/smtproutes
or simscan ???
What do you think?
The diagram of the found at http://www.nrg4u.com/ does a very good job
of explaining the qmail process
man qmail-control will also tell what each control file is used by.
In your case:
qmail-smtp check incoming email envelope recipient addresses against the
contents of rcpthosts to see if it should be accepted.
It then passes the email onto qmail-queue, unless $QMAILQUEUE is set in
which case it will invoke the program named.
A simple /home/vpopmail/etc/tcp.smtp should include
:allow,QMAILQUEUE="/var/qmail/bin/simscan"
The message is then passed to qmail-send which determines whether the
message is local or remote by the presence( or lack of ) the
envelope recipient in the virtualdomains or locals file.
If it is determined to be remote, it is passed to qmail-rspawn and then
qmail-remote.
Qmail-remote checks smtproutes for the domain and sends the message as
appropriate.
The short answer was, simscan comes first.
You turn on debug in simscan by putting the following in
/service/qmail-smtpd/run before the call to exec.
SIMSCAN_DEBUG=2
export SIMSCAN_DEBUG
The other thing to check is the /var/qmail/control/simcontrol file
Make sure clam and spam are set to yes, otherwise they won't be invoked.
Regards
Tom Beardshall
OK.
# cat /home/vpopmail/etc/tcp.smtp
127.:allow,RELAYCLIENT=""
:allow,QMAILQUEUE="/var/qmail/bin/simscan"
24.15.252.147:allow,RELAYCLIENT=""
Remember, this is a functioning toaster. It has a couple dozen virtual
sites, and hundreds of users. *ALL* of the functionality, including
spamassassin _and_ clamav, work successfully, and as expected, for these
users.
In fact, doing this:
tail -f /var/log/{cl,sp}amd/current
I can watch messages pass through simscan, clamav and spamassassin --
provided that they are destined for a virtual user.
Point of fact, GTUBE tests to virtual users result in spamassassin
counts of 999 -- for virtual users; but, I do not see them pass through
simscan; nor are they marked by spamassassin at the Groupwise
destination.
However, the Groupwise administrator cannot seem to get complete
messages and headers to me for inspection. Does anybody here know how I
can have him extract a message from Groupwise?
What am I missing?
How can I test this further?
What do you think?
I've just tried your setup, using rcpthosts and smtproutes to accept &
forward messages for a domain.
It works fine, with simcan calling both spam and clamav on each message.
Sorry to ask this... Can we assume your're not testing your setup by
sending emails from 24.15.252.147 or localhost?
If they can't get you a copy of a message... have you thought of adding
the qmail-tap patch? (Details on shupp.org)
Then you can take a copy of each messsage being forwarded to your
clients domain and inspect it.
Regards,
Tom Beardshall
Re: [toaster] spam/virus-check outsourcing ???
Michael D Schleif wrote on 15/11/2005 16:09:
* Sloath <[EMAIL PROTECTED]> [2005:11:15:15:07:15+0100] scribed:
Michael D Schleif wrote on 15/11/2005 00:01:
* Sloath <[EMAIL PROTECTED]> [2005:11:14:18:44:49+0100] scribed:
Michael D Schleif wrote on 13/11/2005 07:57:
* Tom Collins <[EMAIL PROTECTED]> [2005:10:27:22:30:26-0700] scribed:
On Oct 27, 2005, at 5:02 PM, Michael D Schleif wrote:
We have a client that runs Groupwise on a Netware server on their LAN.
We manage everything on this LAN; and, we handle their DNS for their
domains.
We want to scrub all of their _incoming_ email on our remote toaster,
doing what it does best, then forward *ALL* of their email onto that
Groupwise box.
Use simscan. Make your server the only MX for the domain. Put the
domain in /var/qmail/control/rcpthosts (or morercpthosts and run
qmail-newmrh). Do *not* add their domain to locals or virtualdomains.
Add an entry in /var/qmail/control/smtproutes to route the scanned mail
to their Groupwise server.
--
Tom Collins - [EMAIL PROTECTED]
QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/
You don't need a laptop to troubleshoot high-speed Internet:
sniffter.com
I finally got back on this one. Yes, setup was simple; and, yes,
messages _do_ go through our toaster, and they _do_ end up in the
Groupwise server.
This setup does *not* appear to work. I am still waiting for the
Groupwise admin to send to me the complete headers for the test
messages; but, I am told that *none* of these test messages include
spamassassin headers.
In fact, the following does *not* show _any_ activity for test messages:
tail -f /var/log/{cl,sp}amd/current
I am not clear which process comes first,
/var/qmail/control/smtproutes
or simscan ???
What do you think?
The diagram of the found at http://www.nrg4u.com/ does a very good job
of explaining the qmail process
man qmail-control will also tell what each control file is used by.
In your case:
qmail-smtp check incoming email envelope recipient addresses against the
contents of rcpthosts to see if it should be accepted.
It then passes the email onto qmail-queue, unless $QMAILQUEUE is set in
which case it will invoke the program named.
A simple /home/vpopmail/etc/tcp.smtp should include
:allow,QMAILQUEUE="/var/qmail/bin/simscan"
The message is then passed to qmail-send which determines whether the
message is local or remote by the presence( or lack of ) the
envelope recipient in the virtualdomains or locals file.
If it is determined to be remote, it is passed to qmail-rspawn and then
qmail-remote.
Qmail-remote checks smtproutes for the domain and sends the message as
appropriate.
The short answer was, simscan comes first.
You turn on debug in simscan by putting the following in
/service/qmail-smtpd/run before the call to exec.
SIMSCAN_DEBUG=2
export SIMSCAN_DEBUG
The other thing to check is the /var/qmail/control/simcontrol file
Make sure clam and spam are set to yes, otherwise they won't be invoked.
Regards
Tom Beardshall
OK.
# cat /home/vpopmail/etc/tcp.smtp
127.:allow,RELAYCLIENT=""
:allow,QMAILQUEUE="/var/qmail/bin/simscan"
24.15.252.147:allow,RELAYCLIENT=""
Remember, this is a functioning toaster. It has a couple dozen virtual
sites, and hundreds of users. *ALL* of the functionality, including
spamassassin _and_ clamav, work successfully, and as expected, for these
users.
In fact, doing this:
tail -f /var/log/{cl,sp}amd/current
I can watch messages pass through simscan, clamav and spamassassin --
provided that they are destined for a virtual user.
Point of fact, GTUBE tests to virtual users result in spamassassin
counts of 999 -- for virtual users; but, I do not see them pass through
simscan; nor are they marked by spamassassin at the Groupwise
destination.
However, the Groupwise administrator cannot seem to get complete
messages and headers to me for inspection. Does anybody here know how I
can have him extract a message from Groupwise?
What am I missing?
How can I test this further?
What do you think?
I've just tried your setup, using rcpthosts and smtproutes to accept &
forward messages for a domain.
It works fine, with simcan calling both spam and clamav on each message.
OK ...
Sorry to ask this... Can we assume your're not testing your setup by
sending emails from 24.15.252.147 or localhost?
Yes, I am sending mail through czar.platinumaire.net via my comcast
account on 24.15.252.147. czar _does_ accept mail from that address;
empirical evidence of which is that toaster virtual domains/users *all*
successfully receive messages from me ...
If they can't get you a copy of a message... have you thought of adding
the qmail-tap patch? (Details on shupp.org)
Then you can take a cop
