Re: Trying to understand how bayes works.

[Reindl Harald]

Am 11.12.2015 um 17:40 schrieb Kris Deugau:

Marc Perkel wrote:

I've had bayes disabled in SA because it seems to not be able to stay
working in a high volume situation. The MySQL server can't seem to keep
up with it even on very fast computers.

I'm curious where you started seeing performance issues (number of
messages, users;  hardware platform stats), because it's something I'm
keeping in mind for "things to watch out for" locally.  Particularly in
light of:

Yes MariaDB was better than MySQL but not good enough to keep up. I even tried 
putting the database on ram disk and still didn't work.

where we switched from MySQL (either ISAM or InnoDB) on disk to ramdisk,
and immediately dropped 90%+ of the performance issues we were having.

We also found that MySQL replication was such an I/O hog that it was
better to single-host the SA Bayes DB, keep reasonable backups, and plan
to restore from backup instead of trying for hot or warm failover

well, try the default bayes-backend on a tmpfs





0      54694    SPAM
0      21111    HAM
0    2379839    TOKEN

-rw------- 1 sa-milt sa-milt 10M 2015-12-11 17:39 bayes_seen
-rw------- 1 sa-milt sa-milt 81M 2015-12-11 17:39 bayes_toks
-rw------- 1 sa-milt sa-milt  39 2015-12-07 21:10 user_prefs



[spamass-milter@mail-gw:~]$ cat /etc/systemd/system/bayes.service
[Unit]
Description=Bayes RAM-Disk Manager
Before=spamassassin.service

[Service]
Type=oneshot
RemainAfterExit=yes
User=sa-milt
Group=sa-milt


[Install]
WantedBy=multi-user.target

when anything is producing load here, the bayes-db don't while it's likely one of the largest bayes databases out there caused by no auto-expire/auto-learn and no limited to just 150000 tokens last night it took 4300 seconds to test 75000 samples against the current bayes which means 17/second BUT the most expensive part is for sure fire up 75000 times the spamc-process by a script and not the bayes itself "/var/lib/spamass-milter/.spamassassin/" where the sit-wide bayes is stored is a read-only tmpfs (read-only for SA) restored at boot from a persistent folder and besides shut down rsynced also when the learning script is called ExecStart=/usr/bin/rsync --quiet --recursive --times /var/lib/bayes-persistent/ /var/lib/spamass-milter/.spamassassin/ ExecStop=/usr/bin/rsync --quiet --recursive --times /var/lib/spamass-milter/.spamassassin/ /var/lib/bayes-persistent/

signature.asc
Description: OpenPGP digital signature