Re: Integrating high perf patch

2005-03-09 Thread David Sparks
I've been running this code for a day now without crashes or signs of memory growth on about 3M messages. What I've noticed is that certain messages are almost "stuck" on the sending MTA because processing of them stalls (in 15000 out of 3Million). The envelope sender and recipient are always

Re: Integrating high perf patch

2005-03-09 Thread David Sparks
Matt Sergeant wrote: On 9 Mar 2005, at 19:41, David Sparks wrote: I've been running this code for a day now without crashes or signs of memory growth on about 3M messages. What I've noticed is that certain messages are almost "stuck" on the sending MTA because processin

IPC::DirQueue queue plugin?

2005-04-21 Thread David Sparks
Hi all, I've been investigating using IPC::DirQueue in a queue plugin with the high_perf branch. Does such a plugin already exist? Some performance #s spamtrap queue plugin: local queue: 6000 msg/min nfs queue: 4000 msg/min IPC::DirQueue queue plugin (nfs only): $dq->enqueue_file: 2600

Re: IPC::DirQueue queue plugin?

2005-04-22 Thread David Sparks
> oh, btw, I've just released IPC::DirQueue 0.05, which includes support for > the enqueue_sub() API as used in http://taint.org/wk/QpsmtpdIpcDirqueue . I'm getting ~5% (whoopee!) better throughput by calling the enqueue_string method with a full message. Tested with 11k and 50k messages. This i

Re: IPC::DirQueue queue plugin?

2005-04-22 Thread David Sparks
>>>I've been investigating using IPC::DirQueue in a queue plugin with the >>>high_perf branch. Does such a plugin already exist? > > > Not that I know of --- I haven't even looked into the details of > high_perf, and what may be required there, yet ;) I thought it all just worked. :) > oh, bt

PATCH: high_perf: avoid warning when no plugins run on body

2005-05-11 Thread David Sparks
I'm getting a warning on every message: Use of uninitialized value in array element at [...]lib/Qpsmtpd/Transaction.pm line 164. I actually don't use any plugins that run on the body. Perhaps _body_current_pos is being initialized by plugins that parse the body? --- Transaction.pm 2005-05

Re: Opinion regarding multiple recipients per connection

2005-05-12 Thread David Sparks
> 550 Spam detected. This almost never does what you think it does or want it to do. On a bad day I personally get 100+ bounces caused by people who though rejecting spams (and virii) was a good idea. Accept and discard. ds

$ENV{HOME} not correct when run uses setuidgid

2005-06-07 Thread David Sparks
When running qpsmtpd under `setuidgid` (from daemontools) it doesn't reset ENV{HOME} so it tries to write to /root/tmp. root # setuidgid smtpd echo $HOME /root I'm not a daemontools expert, is this expected? This is my run file and the change I needed to make it work: dave $ cat run #!/

persistent configs?

2005-06-07 Thread David Sparks
I wanted to add some kind of extended config support for plugins that I'm writing that run at connect, mail from and rcpt to stages. I hacked in YAML support into lib/Qpsmtpd.pm (the diff is below but I'm not submitting this as a patch ... at this point at least). I've noticed that the config fil

Re: $ENV{HOME} not correct when run uses setuidgid

2005-06-08 Thread David Sparks
John Peacock wrote: > David Sparks wrote: > >> I'm not a daemontools expert, is this expected? > > > Yes. The documentation for setuidgid: > > <http://cr.yp.to/daemontools/setuidgid.html> > > doesn't mention anything about resetting E

Re: persistent configs?

2005-06-08 Thread David Sparks
John Peacock wrote: > David Sparks wrote: > >> I wanted to add some kind of extended config support for plugins that >> I'm writing that run at connect, mail from and rcpt to stages. I hacked >> in YAML support into lib/Qpsmtpd.pm (the diff is below but I'm

Re: status of trunk ?

2006-03-10 Thread David Sparks
> I have been running it for a while however I don't have good news. I > lose mail. I have no idea why yet and I haven't had chance to debug it. > I'm pretty frustrated by this issue. It seems to work fine for a while > and eventually start failing in very very weird ways (mostly to do with > r

config questions

2006-05-01 Thread David Sparks
(using poll server) I'm wondering why when I hook_config() in a plugin that hook is called whenever qpsmtpd calls ->config() somewhere? my hook_config() gets called for requests for 'me' and 'smtpgreeting'. Is there no way to have this hook only called from the plugin that defined it? It just s

Re: config questions

2006-05-02 Thread David Sparks
Ask Bjørn Hansen wrote: > > On May 1, 2006, at 6:10 PM, David Sparks wrote: > >> I'm wondering why when I hook_config() in a plugin that hook is called >> whenever qpsmtpd calls ->config() somewhere? my hook_config() gets >> called for requests for 'me&

transaction obj in hook_connect()

2006-05-02 Thread David Sparks
The transaction object you get in hook_connect, ie: sub hook_connect { my ($self, $transaction) = @_; what is it used/good for? Any values put in the notes section seem to be gone when accessed during the mail transaction. I would hazard a guess that the transaction object is reset when the M

Re: config questions

2006-05-03 Thread David Sparks
John Peacock wrote: > David Sparks wrote: > >>Is there a preferred way to get the config dir? To call config_dir() I >> seem to need to "use Qpsmtpd;" in my plugin and call it via >>Qpsmtpd::configdir(). > > > Perhaps you could tell us what you inten

define fatal problem during register?

2006-12-13 Thread David Sparks
I'm using the register method in a plugin to do some pre-configuration. One of the things it does is check that its special working directories are there and permissions are as expected. This works fine, but I'm not sure what to do if there is a problem? What is the best practice if you encounte

getting strict on angle brackets and spaces

2007-02-16 Thread David Sparks
Hi, I've just upgraded to the latest version and noticed that mail from and rcpt to checking is now very strict. Bear in mind that I use qpsmtpd on a spamtrap, so getting "strict" with the protocol isn't what I want. I found the workaround for optional brackets by adding them in hook_rcpt_pre whe

Re: getting strict on angle brackets and spaces

2007-02-18 Thread David Sparks
(as a side note I have another speedup coming for you shortly - I have re-written the DNS resolver in XS, which should provide a significant speedup). Sweet, I haven't looked too closely at whats going on, but %INC suggests that PollServer still uses Net::DNS for something. Will this depend

poll server not happy receiving pipelined msg from postfix

2007-03-14 Thread David Sparks
I was having problems getting a postfix mta to forward messages to qpsmtpd-async. Postfix was reporting that the connection was lost after end of data. plugins for hook_connect, hook_mail and hook_rcpt would run, but the plugins for hook_data_post, hook_queue and hook_disconnect were not run.

Re: poll server not happy receiving pipelined msg from postfix

2007-03-20 Thread David Sparks
Try latest SVN. There were some bugs in this area I just fixed. Yup that seems to have fixed it, I'm unable to reproduce the problem. Cheers, ds

Re: Transaction IDs

2007-08-23 Thread David Sparks
JT Moree wrote: > Hanno Hecker wrote: >>> Is there a message ID that is unique to each message? >> Not until the sending client has submitted a 'Message-ID' header ;-) > >> But it would be easy to add / generate a transaction id after every >> reset_transaction() call. This could be logged instead

Re: Transaction ID suggestions

2007-08-24 Thread David Sparks
>> = sprintf("%.4f", time()) .".". $self->qp->config("me") . \ >> sprintf("%08X", rand(2**32 - 1)); #how expensive is this? >> >> These are the approaches suggested so far. I added the last one as a >> combination of the others. Can we see a show of hands for the one > > Using rand is bogus.

Re: Transaction ID suggestions

2007-08-29 Thread David Sparks
A UUID is preferable to the other solutions because you can condense it down to 128 bits of binary data ... and put it in a database. :) The other solutions are not as database friendly. It seems to me if we're trying to solve the problem of guaranteeing unique transaction ids for extremely high

once only initialization hooks?

2007-09-18 Thread David Sparks
I forgot why init() is called per connection in the async server? Something to do with changing plugins on the fly? Right now my plugins are 30% management code to avoid reloading their config in init(). Does anyone else see a need for once only initialization hooks? Cheers, ds

poll server smtp session runs out of order?

2007-12-18 Thread David Sparks
Hi, Has anyone noticed the poll server running "out of order" when under high load? I can connect to one of my qp servers and successfully do most of a smtp transaction before the banner even arrives. Session dump: # telnet 1.2.3.4 25 Trying 1.2.3.4... Connected to 1.2.3.4. Escape character is

Re: poll server smtp session runs out of order?

2007-12-18 Thread David Sparks
Juerd Waalboer wrote: > David Sparks skribis 2007-12-18 16:36 (-0800): >> # telnet 1.2.3.4 25 >> Trying 1.2.3.4... >> Connected to 1.2.3.4. >> Escape character is '^]'. >> helo b.c >> mail from:<[EMAIL PROTECTED]> >> rcpt to:<[EMAIL

Re: poll server smtp session runs out of order?

2008-01-03 Thread David Sparks
Hi Matt, > I just checked this in which should fix it: Indeed that seems to fix the ordering problem. Cheers, ds > > --- qpsmtpd-async (revision 823) > +++ qpsmtpd-async (working copy) > @@ -369,6 +369,7 @@ > > $client->process_line("Connect\n"); > $client->watch_read(

too accept() happy?

2008-07-08 Thread David Sparks
Sometime I notice that qpsmtpd is very busy accepting new connections but never getting around to processing them (before they timeout). I wonder if anyone sees any value in being able to tweak the number of accept()s per event loop cycle? I tried the patch below with some success. As I unde