Re: bayes sync is hogging cpu

2006-09-29 Thread Andreas Pettersson
Bret Miller wrote: I used to have problems with bayes locking and journaling. When it finally corrupted the database, I decided it was time to put it into a real SQL database instead of using DB_File. Haven't had a single problem with bayes CPU or locking since. Maybe it's time you consider usi

Re: bayes sync is hogging cpu

2006-09-26 Thread John Mok
When using Berkeley DB, the default values of database config. might lead to database corruption in a high-load situation, especially the cache size. You may try to put the following in the DB_CONFIG :- set_cachesize 0 536870912 1 set_lg_regionmax 262144 set_lg_bsize 2097152 Note : Cache siz

Re: bayes sync is hogging cpu

2006-09-26 Thread Andreas Pettersson
Bret Miller wrote: I used to have problems with bayes locking and journaling. When it finally corrupted the database, I decided it was time to put it into a real SQL database instead of using DB_File. Haven't had a single problem with bayes CPU or locking since. May

Re: bayes sync is hogging cpu

2006-09-26 Thread Andreas Pettersson
Fabien GARZIANO wrote: Ok, I may say something dumb, but have you tried to clear the bayes db with : sa-learn --clear --dbpath -- Fab No, not yet, but that would be the last option if nothing else helps. I have already prepared a few 100 spams and hams for immediate training after wip

Re: bayes sync is hogging cpu

2006-09-26 Thread Fabien GARZIANO
Ok, I may say something dumb, but have you tried to clear the bayes db with : sa-learn --clear --dbpath -- Fab

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Logan Shaw wrote: One thing you could try is running db4_recover (or db_recover, depending on how it's installed) on the Bayes database. Seems like something to try. But I don't understand the utility: usage: db_recover [-ceVv] [-h home] [-P password] [-t [[CC]YY]MMDDhhmm[.SS]] How can I spec

Re: bayes sync is hogging cpu

2006-09-25 Thread Logan Shaw
On Mon, 25 Sep 2006, Andreas Pettersson wrote: Same Bus error (core dumped) as before when running manual expire. When I make another try it hogs, and is still doing so after 5 minutes. But this time I'll wait at least 30 minutes, just to make sure. And just to make it clear; the spamd daemon is

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Bret Miller wrote: Are you sure you have enough RAM to handle the number of threads you are running? Yes, I'm pretty sure 512MB is enough. No swapping going on, and I only scan msgs smaller than 500 KB. Avg scan time is about 3-4 sec and I scan less than 1 a day. Regards, Andreas

RE: bayes sync is hogging cpu

2006-09-25 Thread Bret Miller
> >I used to have problems with bayes locking and journaling. When it > >finally corrupted the database, I decided it was time to put > it into a > >real SQL database instead of using DB_File. Haven't had a > single problem > >with bayes CPU or locking since. > > > >Maybe it's time you consider usi

Re: bayes sync is hogging cpu

2006-09-25 Thread David B Funk
On Mon, 25 Sep 2006, Andreas Pettersson wrote: > Bret Miller wrote: > > >I used to have problems with bayes locking and journaling. When it > >finally corrupted the database, I decided it was time to put it into a > >real SQL database instead of using DB_File. Haven't had a single problem > >with

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Jonas Eckerman wrote: Andreas Pettersson wrote: Bus error (core dumped) This *can* be the symnptom of a hardware problem, such as bad memory or a bad disk. If you have a disk thats going bad, the symptoms often are corrupt files and extremeley slow writes (because the disk controller re

Re: bayes sync is hogging cpu

2006-09-25 Thread Jonas Eckerman
Andreas Pettersson wrote: Bus error (core dumped) This *can* be the symnptom of a hardware problem, such as bad memory or a bad disk. If you have a disk thats going bad, the symptoms often are corrupt files and extremeley slow writes (because the disk controller retries the write operation

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Bret Miller wrote: I used to have problems with bayes locking and journaling. When it finally corrupted the database, I decided it was time to put it into a real SQL database instead of using DB_File. Haven't had a single problem with bayes CPU or locking since. Maybe it's time you consider usi

RE: bayes sync is hogging cpu

2006-09-25 Thread Bret Miller
> Me again. Since I'm not getting any responses I better keep > posting more > information as I've made some more investigating today. > > Sometimes when I run sa-learn --force-expire I get this > response almost immediately: > Bus error (core dumped) > When I run again the process just hogs until

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Here's an interesting observation. I set bayes_auto_expire to 0 as a temporary solution, I thought, and restarted spamd. The hogging occurs at least as often as before. Am I looking in the wrong direction or wouldn't this have helped something? Another observation: # sa-learn --dump magic: bay

Re: bayes sync is hogging cpu

2006-09-25 Thread Andreas Pettersson
Me again. Since I'm not getting any responses I better keep posting more information as I've made some more investigating today. Sometimes when I run sa-learn --force-expire I get this response almost immediately: Bus error (core dumped) When I run again the process just hogs until I break it

Re: bayes sync is hogging cpu

2006-09-23 Thread Andreas Pettersson
Ok, more information here. I found in spamd.log this line when the problem started: Fri Sep 22 19:55:22 2006 [74581] warn: bayes: expire_old_tokens: child processing timeout at /usr/local/bin/spamd line 1082 which was followed by lots of these: Fri Sep 22 19:55:52 2006 [74581] warn: bayes: can

Re: bayes sync is hogging cpu (was: Some mail seems to hog spamd process)

2006-09-23 Thread Andreas Pettersson
Hi, me again ;) I'm pretty confident that the hogging occurs when SA is trying to sync the bayes. The bayes_journal is cleared exactly when the hogging begins. And when I run sa-learn --sync I get the very same hogging effect. The permissions seems ok, doesn't it? -rw--- 1 spamd wheel