Re: [SAtalk] bayes expiry error

2004-01-07 Thread Theo Van Dinter
On Wed, Jan 07, 2004 at 09:09:32AM -0500, Adam Denenberg wrote: > i am getting the following bayes error when trying to do a > force-expire. Is there any way around this so i can expire my old > tokens? > > debug: bayes: expiry check keep size, 75% of max: 75 > debug: bayes: token count: 0,

[SAtalk] bayes expiry error

2004-01-07 Thread Adam Denenberg
Hello, i am getting the following bayes error when trying to do a force-expire. Is there any way around this so i can expire my old tokens? debug: bayes: found bayes db version 2 synced Bayes databases from journal in 0 seconds: 569 unique entries (569 total entries) debug: bayes: expiry chec

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Theo Van Dinter
On Wed, Dec 03, 2003 at 05:07:25PM -0800, Justin Mason wrote: > I think the implementation of "each" in DB_File must be inefficient, > creating an in-memory list. It might be a good idea to check out > DB_File's built-in iterator methods, now that we only support that > db module. Haven't check

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Justin Mason
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Theo Van Dinter writes: >The expiry should, theoretically, not take a lot of memory since it >really only needs to have 1 token in memory at a time while it copies >between database files. An expiry forces a journal sync though, so an >expiry run wil

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Theo Van Dinter
On Wed, Dec 03, 2003 at 04:11:48PM -0600, Smart,Dan wrote: > So if I set the Journal to a really big number, like 10MB, then the > once-per-day opportunistic sync rule should kick in and sync once per day. > > So a sync takes as much or more memory than an expiration run? Yeah, fun question: The

RE: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Smart,Dan
So if I set the Journal to a really big number, like 10MB, then the once-per-day opportunistic sync rule should kick in and sync once per day. So a sync takes as much or more memory than an expiration run? <> | -Original Message- | From: Theo Van Dinter [mailto:[EMAIL PROTECTED] | Sent

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Theo Van Dinter
On Wed, Dec 03, 2003 at 01:47:55PM -0600, Smart,Dan wrote: > Is there a reason that the journal needs to be synced more than once per > day? Why is the default set to 100KB? The default was reasonable for most users. There's no need, really, to sync more than once a day, but a large journal mean

RE: [SAtalk] Bayes Expiry limits and Journal size

2003-12-03 Thread Smart,Dan
ct: Re: [SAtalk] Bayes Expiry limits and Journal size | | [EMAIL PROTECTED] (Justin Mason) writes: | | > Smart,Dan writes: | >> I'm running Bayes site wide with about 45,000 messages per | day. Is | >> there any harm in increasing the bayes_expiry_max_db_size and | >&g

Re: [SAtalk] Bayes expiry

2003-12-02 Thread Ted Cabeen
Yevgeniy Miretskiy <[EMAIL PROTECTED]> writes: > sa-learn stopped learning messages. Debugging shows that it can > successfully tie Bayes db, extracts tokens, etc, but never actually > writes data to the database. > > No matter what bayes_expiry_max_db_size is set to (I tried anything from 100K t

Re: [SAtalk] Bayes expiry

2003-11-26 Thread David B Funk
On Tue, 25 Nov 2003, Yevgeniy Miretskiy wrote: > Hello, > > sa-learn stopped learning messages. Debugging shows that it can > successfully tie Bayes db, extracts tokens, etc, but never actually > writes data to the database. > > I had a db corruption issue some time ago, so, this could very > wel

Re: [SAtalk] Bayes expiry

2003-11-25 Thread Theo Van Dinter
On Tue, Nov 25, 2003 at 02:45:17PM -0500, Yevgeniy Miretskiy wrote: > No matter what bayes_expiry_max_db_size is set to (I tried anything from 100K to > 3Mil), > sa-learn reports, after running for quite some time: > bayes: couldn't find a good delta atime, need more token difference, skipping

[SAtalk] Bayes expiry

2003-11-25 Thread Yevgeniy Miretskiy
Hello, sa-learn stopped learning messages. Debugging shows that it can successfully tie Bayes db, extracts tokens, etc, but never actually writes data to the database. I had a db corruption issue some time ago, so, this could very well be remnants of that. Anyway, I'm trying to run "sa-learn -

RE: [SAtalk] Bayes Expiry limits and Journal size

2003-11-21 Thread Smart,Dan
I'm working on just that. My thought, and I must admit I don't know all the theory involved, is that the effectiveness of Bayes and the FN/FP is purely a function of how you trained it. The *ONLY* way (in my mind) to score the Bayes functions would be to train it and run the Mass-check. I'm work

RE: [SAtalk] Bayes Expiry limits and Journal size

2003-11-20 Thread Bob Apthorpe
Hi, On Thu, 20 Nov 2003, Smart,Dan wrote: > Is there a reason that the Bayes scoring is NOT a normal distribution from > 50% to 100%, and negative from 0% to 50%? Yes, check the [SAtalk] list archives; this may well be a FAQ. Short answer: all scores including those from Bayes are generated by a

RE: [SAtalk] Bayes Expiry limits and Journal size

2003-11-20 Thread Smart,Dan
0 3.0 score BAYES_98 4.0 score BAYES_99 5.4 - Comments? <> | -Original Message- | From: Ted Cabeen [mailto:[EMAIL PROTECTED] | Sent: Friday, November 14, 2003 6:22 PM | To: [EMAIL PROTECTED] | Cc: Smart,Dan; [EMAIL PROTECTED] | Subj

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-11-14 Thread Ted Cabeen
[EMAIL PROTECTED] (Justin Mason) writes: > Smart,Dan writes: >> I'm running Bayes site wide with about 45,000 messages per day. Is there >> any harm in increasing the bayes_expiry_max_db_size and >> bayes_journal_max_size limits? >> >> If it's ok to make bigger, what is the maximum before somet

Re: [SAtalk] Bayes Expiry limits and Journal size

2003-11-14 Thread Justin Mason
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Smart,Dan writes: > I'm running Bayes site wide with about 45,000 messages per day. Is there > any harm in increasing the bayes_expiry_max_db_size and > bayes_journal_max_size limits? > > If it's ok to make bigger, what is the maximum before someth

[SAtalk] Bayes Expiry limits and Journal size

2003-11-14 Thread Smart,Dan
Title: Blank Stationery I'm running Bayes site wide with about 45,000 messages per day.  Is there any harm in increasing the bayes_expiry_max_db_size and bayes_journal_max_size limits?   If it's ok to make bigger, what is the maximum before something bad starts happening?   <>