RE: postres bayes db and high load

[Michael Scheidell]

Title: Message

 

-----Original Message-----
From: Dan [mailto:[EMAIL PROTECTED]
Sent: Thursday, August 03, 2006 12:24 PM
To: users@spamassassin.apache.org
Subject: postres bayes db and high load

Over the past few weeks, my company's mail server has been experiencing high loads that result in SA skipping emails.  I use a postgres database to manage bayes, awl and userprefs.  I am pretty sure that it is the bayes db that is causing the high load and resultant skipping, but I have no idea how to fix the problem.  I installed the SA DBI plugin in hopes this would decrease the load, but it hasn't.  I have also tried increasing spamd's max-children parameter from 8 up to 27.

It's appears that if all of the spamd's children become busy SA skips the message all together.  Or spamd stops working on a message when bayes times out.  If the latter is the case, is there a way to tell spamd to continue processing the message without bayes?

I have included some details below.  Any suggestions would be very helpful.

I have systems running 800K email/day, no problem,  I use mysql, it seemed to scale a lot better with one of our other products (which has postgess issues when busy).  if using mysql, use the correct .cfg file (mysql-large.cfg,  )

 

 

 The mail servers stats:
~3500 email/day
2GHz Intel Celeron
768M ram
SA v3.1.0
postgreSQL v8.0.4
database size: 333M
bayes_seen: 378275 rows
bayes_token: 172484 rows


a snippet of maillog when the disruption began:
Aug  2 14:47:59 mail spamd[32613]: prefork: child states: BBBBBBBBBBBBBBBBBBBBBBBBBBB
Aug  2 14:47:59 mail spamd[32613]: prefork: server reached --max-clients setting, consider raising it
Aug  2 14:47:59 mail spamd[3577]: spamd: connection from localhost.localdomain [127.0.0.1 ] at port 49872
Aug  2 14:47:59 mail spamd[3577]: spamd: processing message < [EMAIL PROTECTED]
om> for steve:0
Aug  2 14:48:16 mail spamd[3675]: bayes: child processing timeout at /usr/bin/spamd line 1088.
Aug  2 14:48:19 mail spamd[3675]: spamd: identified spam ( 25.9/5.0) for bug:0 in 5525.1 seconds, 2163 bytes.
Aug  2 14:48:19 mail spamd[3675]: spamd: result: Y 25 - BAYES_99,MY_ALL_CAPS,MY_CASINO,MY_OFFER, MY_URI_2CHAR,MY_URI_ALPHNM,MY_URI_CHARNUM,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR 2_CHECK,URIBL_BLACK,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SC_SURBL,URIBL_WS_SURBL scantime= 5525.1,size=2163,user=bug,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1 ,rport=48946, mid=<[EMAIL PROTECTED]>,bayes=1,autolearn=failed
Aug  2 14:48:35 mail spamd[3675]: __alarm__
Aug  2 14:48:35 mail spamd[3675]: __alarm__
Aug  2 14:48:36 mail spamd[32613]: prefork: child states: BBBBBBBBBBBBBBBBBBBBBBBBBBB
Aug  2 14:48:36 mail spamd[32613]: prefork: server reached --max-clients setting, consider raising it
Aug  2 14:48:36 mail spamd[3675]: spamd: connection from localhost.localdomain [127.0.0.1 ] at port 49881
Aug  2 14:48:40 mail spamd[3675]: spamd: processing message < [EMAIL PROTECTED]> for harriet:0
Aug  2 14:50:06 mail spamd[3835]: bayes: child processing timeout at /usr/bin/spamd line 1088, <GEN1402> line 59.