Postfix sendmail cannot be called from Sieve redirect
Sending (in my case: forwarding) messages from Sieve via sendmail does not
work with version 2.3.1.
I have narrowed it down to this simple test case:
# dovecot -n
# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# OS: Linux 4.4.0-119-generic x86_64 Ubuntu 16.04.4 LTS
# Hostname: tuxi.topfen.net
first_valid_gid = 200
first_valid_uid = 200
mail_location = mbox:~/mail
passdb {
args = password=secret
driver = static
}
plugin {
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = lmtp
service lmtp {
inet_listener lmtp {
address = 127.0.0.1
port = 24
}
}
userdb {
args = uid=200 gid=200 home=/home/vmail
driver = static
}
protocol lmtp {
mail_plugins = sieve
}
# cat /home/vmail/.dovecot.sieve
redirect "test@example.invalid";
stop;
$ swaks --protocol lmtp -s 127.0.0.1 -f ulr...@topfen.net -t ulr...@topfen.net
=== Trying 127.0.0.1:24...
=== Connected to 127.0.0.1.
<- 220 tuxi.topfen.net Dovecot ready.
-> LHLO tuxi.topfen.net
<- 250-tuxi.topfen.net
<- 250-8BITMIME
<- 250-CHUNKING
<- 250-ENHANCEDSTATUSCODES
<- 250-PIPELINING
<- 250 VRFY
-> MAIL FROM:
<- 250 2.1.0 OK
-> RCPT TO:
<- 250 2.1.5 OK
-> DATA
<- 354 OK
-> Date: Tue, 17 Apr 2018 18:44:42 +0200
-> To: ulr...@topfen.net
-> From: ulr...@topfen.net
-> Subject: test Tue, 17 Apr 2018 18:44:42 +0200
-> X-Mailer: swaks v20130209.0 jetmore.org/john/code/swaks/
->
-> This is a test mailing
->
-> .
<** 451 4.3.0 Temporary internal error
-> QUIT
<- 221 2.0.0 Bye
=== Connection closed with remote host.
$ tail -n 10 /var/log/mail.log
Apr 17 18:45:36 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create
file maildrop/421882.22643: Permission denied
Apr 17 18:45:36 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter:
create file maildrop/421882.22643: Permission denied
Apr 17 18:45:36 tuxi dovecot:
lmtp(ulr...@topfen.net)<22639>: Error: program
`/usr/sbin/sendmail' was forcibly terminated with signal 15
Apr 17 18:45:36 tuxi dovecot:
lmtp(ulr...@topfen.net)<22639>: Error: sieve:
msgid=unspecified: failed to redirect message to : Failed to
execute sendmail (temporary failure)
Apr 17 18:45:36 tuxi dovecot:
lmtp(ulr...@topfen.net)<22639>: Error: sieve: Execution
of script /home/vmail/.dovecot.sieve was aborted due to temporary failure (user
logfile /home/vmail/.dovecot.sieve.log may reveal additional details)
Apr 17 18:45:36 tuxi dovecot: lmtp(22639): Disconnect from 127.0.0.1: Client
has quit the connection (state = READY)
Apr 17 18:45:46 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create
file maildrop/422091.22643: Permission denied
Apr 17 18:45:46 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter:
create file maildrop/422091.22643: Permission denied
Apr 17 18:45:56 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create
file maildrop/47.22643: Permission denied
Apr 17 18:45:56 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter:
create file maildrop/47.22643: Permission denied
As far as I can tell, this is because Dovecot somehow does not allow suid
programs to run (Postfix' postdrop(1), which is invoked by sendmail(1),
requires set-gid to write the queue file).
What can I do to change this (besides using SMTP from Dovecot directly)?
I am using Ubuntu 16.04 and the Dovecot packages from the community
repository:
~$ dpkg -l dovecot* | grep ^ii
ii dovecot-core 2:2.3.1-1amd64secure POP3/IMAP server -
core files
ii dovecot-lmtpd 2:2.3.1-1amd64secure POP3/IMAP server -
LMTP server
ii dovecot-sieve 2:2.3.1-1amd64secure POP3/IMAP server -
Sieve filters support
Ulrich
Re: Postfix sendmail cannot be called from Sieve redirect
On Tue, Apr 17, 2018 at 07:04:56PM +0200, Alexander Dalloz wrote: > Am 17.04.2018 um 19:00 schrieb Ulrich Zehl: > > Sending (in my case: forwarding) messages from Sieve via sendmail does not > > work with version 2.3.1. > > https://dovecot.org/pipermail/dovecot/2018-April/111482.html Thanks, Alexander. I apologize for the noise; I somehow did not think of searching the archive for "forwarding." For the archive: Removing NoNewPrivileges from the systemd unit makes it work fine. Ulrich
[Dovecot] deliver: handling of quota_full_tempfail and -e
Hi,
while testing out dovecot 1.1, I found that when you set
quota_full_tempfail = yes in dovecot.conf and use the deliver -e command
line flag, no deferral reason is printed, thus breaking my fancy
quota_exceeded_message.
This appears to be because of the if() block in lines 1003-1009 in
src/deliver/deliver.c, and should be easily fixable (if it is indeed a bug,
and not intended behavior).
However, I shied away from creating a patch because either there is a
mismatch between the documentation and deliver's actual behavior, or I
misunderstood the code (likely, since I'm not really a C programmer).
The lines in question (in src/deliver/deliver.c) read:
| 1003 if (error != MAIL_ERROR_NOSPACE ||
| 1004 getenv("QUOTA_FULL_TEMPFAIL") != NULL) {
Does this mean that the corresponding block gets executed whenever
quota_full_tempfail is set? If so, setting it transforms all errors into
TEMPFAILs, and not only those relating to quota/space issues. I think
dovecot-example.conf should at least mention this it's inteded; otherwise,
the code should be changed (I'm not completely sure what Timo's intensions
are, so I won't comment on possible solutions).
Regards,
Ulrich
PS: All line numbers are taken from rev 080dd4d2fd94, which was the most
recent at the time of writing.
[Dovecot] [PATCH] Re: deliver: handling of quota_full_tempfail and -e
On Wed, Apr 30, 2008 at 05:53:37PM +0200, Johannes Berg wrote:
>
> > The lines in question (in src/deliver/deliver.c) read:
> > | 1003 if (error != MAIL_ERROR_NOSPACE ||
> > | 1004 getenv("QUOTA_FULL_TEMPFAIL") != NULL) {
> [...]
>
> Hence, what happens is that any error other than "out of space" gives
> TEMPFAIL and when quota-full-tempfail is set "out of space" also gives
> tempfail.
You're right, of course; thanks for clearing it up. I missed that the other
day: While the code is always executed if quota_full_tempfail is set, this
doesn't matter if there was any other error than "out of space", as a
TEMPFAIL is intended in this case anyway.
Now that the intial confusion is cleared up, I'd like to propose the
following patch which changes deliver's -e flag behavior to also include
the error message when TEMPFAILing.
This is useful in setups like mine where deliver is called from some MTA
(e.g., Postix) that includes the command response in bounces generated for
mails that have been in the queue for too long. Without the patch, the
sender will not get a meaningful error message back other than "temporary
error".
Ulrich
--- deliver.c.orig 2008-05-02 13:30:37.0 +0200
+++ deliver.c 2008-05-02 12:35:36.0 +0200
@@ -1005,6 +1005,10 @@
/* Saving to INBOX should always work unless
we're over quota. If it didn't, it's probably a
configuration problem. */
+
+ if (stderr_rejection)
+ fprintf(stderr, "%s\n", error_string);
+
return EX_TEMPFAIL;
}
Re: [Dovecot] LDA/Posfix error: bounce, user unknown
On Thu, May 08, 2008 at 09:01:23PM +0200, Colin Brace wrote: > > I am trying to get LDA working in conjuction with Postfix, but whenever I > activate Dovecot LDA in main.cf and send myself a test messsage, I get an > error message in maillog: > [...] > Postfix's virtual LDA has no such problem: > [...] > What might be going wrong with delivery though? My crystal ball has one idea: You followed the instructions on the wiki page (http://wiki.dovecot.org/LDA/Postfix), which results in Postfix calling deliver with the full mail address of the recipient ([EMAIL PROTECTED] in your example), and you've set up Dovecot's user database to only recognise usernames without domains (e.g., "cb"). This could explain the error you're seeing. If the above is not the problem, please post your dovecot -n output, your userdb configuration file (if seperate), and the relevant postfix configuration settings (especially your master.cf transport definition).
[Dovecot] Multiple auth client sockets
I use two seperate Postfix instances as MSAs (so that different policies
are simpler to configure), and both of them require Dovecot SASL support.
Since Postfix' smtpd is chrooted (at least in the Debian package), the
client socket needs to be somewhere under the instance's queue_directory.
This means I need two different auth client socket paths.
When I simply specify two "client { ... }" in dovecot.conf, the second
takes precedence over the first.
Besides using a second dovecot instance, which I'd like to avoid, what ways
are there to run a second client socket?
Ulrich
Re: [Dovecot] rejecting mail due to quota exceeded
On Wed, Sep 17, 2008 at 09:55:16AM +0100, martin f krafft wrote:
> When a message is delivered to an account that has reached its
> quotum, deliver issues a failure message saying:
>
> [...]
>
> considering that it doesn't know the envelope recipient, this would
> best be solved by
>
> 2. don't accept the mail and send out a rejection, just exit 69 and
>write the reason to stderr!
>
> Did I miss something and is this already possible with dovecot's
> deliver in a virtual setting?
To exit with EX_TEMPFAIL instead of sending a rejection message, use
deliver's -e flag and the following dovecot.conf snippet, taken straight
from our mail server.
protocol lda {
# If user is over quota, return with temporary failure instead of
# bouncing the mail.
quota_full_tempfail = yes
quota_exceeded_message = 4.2.2 Mailbox full
}
We only changed quota_exceeded_message because of Postfix' support of
enhanced status codes, which gets you a nice bounce message with the right
diagnostic code.
This requires dovecot 1.0.1 or later, according to
http://wiki.dovecot.org/LDA; additionally writing the rejection reason to
stderr requires dovecot 1.1.1 or later.
Re: [Dovecot] Courier->dovecot migration script
On Wed, Nov 12, 2008 at 01:12:45PM +0200, Warren Baker wrote: > I just need to confirm that the script > (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works > for Dovecot v1.0 and not for v1.1? > As the dovecout-uidlist file that is created (by the script) is not in > the same format as dovecot-uidlist file that is created by the 1.1.6 > server. The migration script generates the files using uidlist format "Version 1". While Dovecot 1.1.x normally uses "Version 3" format, it can also read the older formats (to maintain backward compability), and thus auto-convert it to the newest version. FWIW, we have used version 1.1.7 of the script for our Couier to Dovecot 1.1.x migration, and it worked well, execpt for cases where Couier's uidlist was too old. Ulrich
[Dovecot] Possible bug in Maildir++ quota?
While researching a possible bug in our custom-made Maildir++ expiration
script, I found the following in src/plugins/quota/quota-maildir.c.
| static const char *
| maildir_list_next(struct maildir_list_context *ctx, time_t *mtime_r)
| {
| [...]
| *mtime_r = st.st_size;
| return str_c(ctx->path);
| }
As far as I unterstand, this seems incorrect, because the value in mtime_r
is used, for example, in maildirs_check_have_changed to check whether any
maildirs or folders have changed.
If this isn't a bug, please satisfy my own curiosity and tell me what I'm
missing.
Ulrich
[Dovecot] Maildir++ quota: When is it recalculated?
We use Maildir++ quota, with the rules taken from our LDAP backend. We also have an old expunge script that's not quota-aware; it removes old messages from the maildir by simply deleting the files. Sometimes, a mailbox is over quota before the script runs, but well under quota after the old messages are deleted. This change does not seem to be picked up, however: When I try to deliver a new message (using Dovecot's deliver), the attempt fails with an "over quota" error. I tried finding out from the source code whether this is intended behavior, but unfortunately, I'm not enough of a programmer, so I have to ask here: Should I be able to delete mails from a maildir, and expect the quota plugin to notice the change? We use Dovecot 1.1.8, by the way. Ulrich
Re: [Dovecot] Maildir++ quota: When is it recalculated?
On Wed, Jan 14, 2009 at 10:18:30AM -0500, Timo Sirainen wrote: > > but unfortunately, I'm not enough of a programmer, so I have to ask here: > > Should I be able to delete mails from a maildir, and expect the quota > > plugin to notice the change? > > I think it should notice most times, but not necessarily always. Can you > reproduce this with simple steps? Meaning that if maildirsize file is > either older than 15 minutes or longer than two lines, does Dovecot not > recalculate the quota when user is over quota? Yes, I can. However, the user is, strictly speaking, not over quota, since there are still a few bytes free. The new message would push it over quota, though. Does this make a difference? These are the steps I took, starting from a "fresh" account (one whose home directory I deleted), and using swaks[0] to generate the mails. We use Postfix to pipe the messages to deliver, and have set quota_full_tempfail = yes. 1. Delete the user's home directory 2. Deliver 3 messages to the user, using # swaks -f ulr...@topfen.net -t grm...@example.net -s localhost 3. maildirsize now shows: # cat Maildir/maildirsize 2000S 0 0 510 1 510 1 510 1 4. Try to deliver a fourth message (also 510 bytes). As expected, it fails. # tail -n 2 /var/log/mail.log Jan 14 17:15:02 carolyn deliver(grm...@example.net): msgid=<20090114161502.7f2aa1c...@carolyn.beispiel.at>: save failed to INBOX: Quota exceeded (mailbox for user is full) Jan 14 17:15:02 carolyn postfix/pipe[8461]: 7F2AA1C104: to=, relay=dovecot, delay=0.18, delays=0.12/0.03/0/0.03, dsn=4.3.0, status=deferred (temporary failure. Command output: Quota exceeded (mailbox for user is full) ) 5. Remove one of the files: # rm Maildir/new/1231949258.M463705P8296.carolyn\,S\=510\,W\=524 6. Try to deliver the message again, and watch it fail again: # postqueue -f # tail -n 2 /var/log/mail.log Jan 14 17:15:55 carolyn deliver(grm...@example.net): msgid=<20090114155631.bcdb71c...@carolyn.beispiel.at>: save failed to INBOX: Quota exceeded (mailbox for user is full) Jan 14 17:15:55 carolyn postfix/pipe[8461]: BCDB71C0DA: to=, relay=dovecot, delay=1164, delays=1164/0.04/0/0.02, dsn=4.3.0, status=deferred (temporary failure. Command output: Quota exceeded (mailbox for user is full) ) 7. Try ageing maildirsize and a new delivery: # touch -t 01010101 Maildir/maildirsize # postqueue -f # tail -n 2 /var/log/mail.log Jan 14 17:18:42 carolyn postfix/pipe[8553]: 7F2AA1C104: to=, relay=dovecot, delay=220, delays=220/0.05/0/0.03, dsn=4.3.0, status=deferred (temporary failure. Command output: Quota exceeded (mailbox for user is full) ) Jan 14 17:18:42 carolyn postfix/pipe[8554]: BCDB71C0DA: to=, relay=dovecot, delay=1331, delays=1331/0.01/0/0.03, dsn=4.3.0, status=deferred (temporary failure. Command output: Quota exceeded (mailbox for user is full) ) 8. Try deleting maildirsize and a new delivery: # rm Maildir/maildirsize # tail -n 3 /var/log/mail.log Jan 14 17:19:48 carolyn deliver(grm...@example.net): msgid=<20090114161502.7f2aa1c...@carolyn.beispiel.at>: saved mail to INBOX Jan 14 17:19:48 carolyn postfix/pipe[8553]: 7F2AA1C104: to=, relay=dovecot, delay=286, delays=286/0.04/0/0.13, dsn=2.0.0, status=sent (delivered via dovecot service) Jan 14 17:19:48 carolyn postfix/qmgr[3599]: 7F2AA1C104: removed [0] http://jetmore.org/john/code/#swaks
Re: [Dovecot] Maildir++ quota: When is it recalculated?
On Wed, Jan 14, 2009 at 11:31:43AM -0500, Timo Sirainen wrote: > On Wed, 2009-01-14 at 17:24 +0100, Ulrich Zehl wrote: > > On Wed, Jan 14, 2009 at 10:18:30AM -0500, Timo Sirainen wrote: > > > > but unfortunately, I'm not enough of a programmer, so I have to ask > > > > here: > > > > Should I be able to delete mails from a maildir, and expect the quota > > > > plugin to notice the change? > > > > > > I think it should notice most times, but not necessarily always. Can you > > > reproduce this with simple steps? Meaning that if maildirsize file is > > > either older than 15 minutes or longer than two lines, does Dovecot not > > > recalculate the quota when user is over quota? > > > > Yes, I can. However, the user is, strictly speaking, not over quota, since > > there are still a few bytes free. The new message would push it over quota, > > though. Does this make a difference? > > Yes, it makes a difference. The recalculation isn't done unless user > really is over quota. Just out of curiosity: How can I actually make the maildir go over quota, short of changing the limits after delivery or using a non-aware LDA? > Hmm. I did just consider adding code to > recalculate the quota also if adding a new mail would bring user over > quota, but it would require pretty large changes to the code. Well, for what it's worth, I'd be very happy if this was implemented in Dovecot in the future, but for now ... > How about you simply delete maildirsize file after deleting files from > maildir? ... I'll do that.
Re: [Dovecot] Dovecot discards mail over quota
On Sun, Jan 18, 2009 at 12:26:46PM -0700, Gary V wrote: > Just as a matter of interest. On my Postfix system: > > a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> > Subject: Undelivered Mail Returned to Sender. Partial body: > ": permission denied. Command output: Quota exceeded > (mailbox for user is full)". Postfix does not retain the message. > > b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user > makes room for the message, then it will eventually be delivered. If > they don't, then _eventually_ a bounce will be sent. In this case the > bounce is less informative. Partial body: ": > temporary failure". In the case where the message is not delivered, > using default settings in Postfix, the sender will be notified 5 days > after they sent the message. > > c) For over quota with a+b, it behaves the same way as b, but the > bounce notice will be more informative: Partial body: > ": temporary failure. Command output: Quota exceeded > (mailbox for user is full)". > > I would say this is expected. You can also quota_exceeded_message to something like "4.2.2 Mailbox full" (or "5.2.2 ..."). This works whenever -e is specified, i.e. for a) and c), and you use Postfix 2.3 or later (see pipe(8)). > Each of the four possibilites has advantages and disadvantages, and > personally I think a) might be closest to "doing the right thing", but > it would be cool to have the option of deferring the mail (using > option a+b) and additionally have deliver immediately send a message > to the sender notifying them that their mail has been delayed due to > the recipient being over quota. Something like: This is also easy with Postfix, if you use delay_warning_time (see postconf(5)).
[Dovecot] Broken dovecot-uidlist files
I have just upgraded to 1.1.8, hoping to fix these types of errors: Jan 20 10:39:27 laura deliver(xxx...@example.net): Broken file /srv/storage/mail/store/net/example/xx/xx/Maildir/dovecot-uidlist line 156: Invalid data: Jan 20 10:41:19 laura dovecot: POP3(yy...@example.at): Broken file /srv/storage/mail/store/at/example/yy.yy/Maildir/dovecot-uidlist line 2: Invalid data: Unfortunately, they still pop up. Since I suspect these errors to be responsible for customers complaining about downloading mails twice, I'd like to fix them. Our setup has multiple servers sharing the same NFS-mounted directory for maildirs and indexes; we have set mmap_disable = yes dotlock_use_excl = yes fsync_disable = no mail_nfs_index = yes lock_method = fcntl As far as I can tell, for at least one of these errors, there was no concurrent access to the mailbox in question. What can I do to solve this problem, or to analyze it further? Ulrich
Re: [Dovecot] Broken dovecot-uidlist files
On Sun, Jan 25, 2009 at 09:28:27PM -0500, Timo Sirainen wrote: > On Tue, 2009-01-20 at 11:11 +0100, Ulrich Zehl wrote: > > Our setup has multiple servers sharing the same NFS-mounted directory for > > maildirs and indexes; we have set > > mmap_disable = yes > > dotlock_use_excl = yes > > fsync_disable = no > > mail_nfs_index = yes > > lock_method = fcntl > > What about mail_nfs_storage=yes? Yes, that's set too. (Sorry, forgot about that.) > > As far as I can tell, for at least one of these errors, there was no > > concurrent access to the mailbox in question. > > Are you also using Dovecot deliver? Yes, for almost all deliveries (see e.g. the first log entry in my original post). There's only one nightly script that delivers one message directly to some of the maildirs, but the errors also pop up on mailboxes that the script doesn't touch.
Re: [Dovecot] Broken dovecot-uidlist files
On Mon, Jan 26, 2009 at 10:02:19AM -0500, Timo Sirainen wrote: > Perhaps the NFS cache flushing doesn't work then for some reason. What > OS (kernel) are you using on the Dovecot servers? How big values have > you set to attribute cache? On the client side, it's Linux 2.6.23.16. All attribute cache related values are at their default, as far as I can tell. The entry in fstab reads: nfs-server:/srv/storage /srv/storage nfs rw,nfsvers=3,hard,intr,nosuid,noexec,nodev,noatime 0 0 > I was just thinking that when you said there was no concurrent access > did you also consider deliver processes and not just imap/pop3? I just checked again. See below for an example of the logs for one mailbox. $ grep 123...@example.net mail.log Jan 27 08:00:17 allina dovecot: pop3-login: Login: user=<123...@example.net>, method=PLAIN, rip=80.x.x.x, lip=10.x.x.x Jan 27 08:00:19 allina dovecot: POP3(123...@example.net): Disconnected: Logged out top=0/0, retr=1/1283428, del=0/1, size=1283408 Jan 27 08:00:57 laura deliver(123...@example.net): msgid=<497eb077.09.01...@ernesto>: saved mail to INBOX Jan 27 08:00:57 laura postfix-store/pipe[18734]: 753522531F: to=<123...@example.net>, relay=dovecot, delay=0.91, delays=0.28/0.06/0/0.57, dsn=2.0.0, status=sent (delivered via dovecot service) Jan 27 08:04:35 laura deliver(123...@example.net): msgid=<497eb05f.05.01...@ernesto>: saved mail to INBOX Jan 27 08:04:35 laura postfix-store/pipe[19818]: 0ABB3251E0: to=<123...@example.net>, relay=dovecot, delay=0.42, delays=0.11/0.06/0/0.25, dsn=2.0.0, status=sent (delivered via dovecot service) Jan 27 08:15:16 ellia dovecot: pop3-login: Login: user=<123...@example.net>, method=PLAIN, rip=80.x.x.x, lip=10.x.x.x Jan 27 08:15:16 ellia dovecot: POP3(123...@example.net): Broken file /srv/storage/mail/store/net/example/12/123456/Maildir/dovecot-uidlist line 2: Invalid data:
Re: [Dovecot] redelivered mail results in mail-forwarding-loop
On Mon, Feb 09, 2009 at 09:32:43PM +0100, Reto Glauser wrote: > The above is working when the "Delivered-To" line is being removed, and my > question is if this can't be done automatically? You can avoid generating a Delivered-To header in the first place by removing "D" from the "flags=..." part of the Postfix master.cf entry for Dovecot. This will, of course, disable this loop avoidance mechanism for all messages. If you don't want this, you'll have to teach the script that reinjects the message to remove the header.
Re: [Dovecot] Broken dovecot-uidlist files
On Tue, Jan 27, 2009 at 08:30:14AM +0100, Ulrich Zehl wrote: > On Mon, Jan 26, 2009 at 10:02:19AM -0500, Timo Sirainen wrote: > > Perhaps the NFS cache flushing doesn't work then for some reason. What > > OS (kernel) are you using on the Dovecot servers? How big values have > > you set to attribute cache? > > On the client side, it's Linux 2.6.23.16. All attribute cache related > values are at their default, as far as I can tell. The entry in fstab > reads: > > nfs-server:/srv/storage /srv/storage nfs > rw,nfsvers=3,hard,intr,nosuid,noexec,nodev,noatime 0 0 As much as I hate replying to myself, I'm at a loss here. What can I do to debug this issue? Could this even be a problem on the NFS server side?
Re: [Dovecot] LDAP, MD5-CRYPT, invalid credentials, BUG or config issue?
On Mon, Feb 16, 2009 at 12:28:54PM +0100, Guenther Falk wrote: > My dovecot-ldap.conf is: > auth_bind = yes > auth_bind_userdn = uid=%u,ou=People,dc=mydomain,dc=com > [...] > default_pass_scheme = CRYPT > [...] > It seems dovecot ignores the CRYPT password scheme. The password seems > to be encrypted in SSHA. If you use auth_bind, then the password scheme specified in dovecot-ldap.conf is irrelevant, as the password comparison will be done by the LDAP server (cf. http://wiki.dovecot.org/AuthDatabase/LDAP). Also, your logs show the communication between the two Dovecot components responsible for the auth process, not between Dovecot and the LDAP server. You may want to check you OpenLDAP logs to see what's going on.
Re: [Dovecot] Broken dovecot-uidlist files
On Fri, Feb 20, 2009 at 05:50:28PM -0500, Timo Sirainen wrote: > On Tue, 2009-01-27 at 08:30 +0100, Ulrich Zehl wrote: > > On Mon, Jan 26, 2009 at 10:02:19AM -0500, Timo Sirainen wrote: > > > Perhaps the NFS cache flushing doesn't work then for some reason. What > > > OS (kernel) are you using on the Dovecot servers? How big values have > > > you set to attribute cache? > > > > On the client side, it's Linux 2.6.23.16. All attribute cache related > > values are at their default, as far as I can tell. The entry in fstab > > reads: > > > > nfs-server:/srv/storage /srv/storage nfs > > rw,nfsvers=3,hard,intr,nosuid,noexec,nodev,noatime 0 0 > > Setting actimeo=0 probably fixes this, but also probably increases the > load a lot. actimeo=1 might work ok and reduce how often these problems > happen, but not eliminate them completely. > > Dovecot's nfs settings should avoid this problem though. You could see > if upgrading your kernel helps. Some kernels have somewhat broken NFS > code. I did # mount -o remount,actimeo=0 /srv/storage around 9 this morning, but I'm still seeing these errors pop up. (I verified that there are mailboxes where this happened more than once since 10 today.) Is it time to upgrade my kernel, or are there other options left? > > $ grep 123...@example.net mail.log > > Jan 27 08:00:17 allina dovecot: pop3-login: Login: > > user=<123...@example.net>, method=PLAIN, rip=80.x.x.x, lip=10.x.x.x > > Jan 27 08:00:19 allina dovecot: POP3(123...@example.net): Disconnected: > > Logged out top=0/0, retr=1/1283428, del=0/1, size=1283408 > > Jan 27 08:00:57 laura deliver(123...@example.net): > > msgid=<497eb077.09.01...@ernesto>: saved mail to INBOX > > So allina modified dovecot-uidlist and soon afterwards laura probably > was using a cached dovecot-uidlist and corrupted it. Since the corrupted files are available for a little while (in the example, it was ~ 15 minutes), will it help if I repeatedly check all dovecot-uidlists and save those found to be corrupted to a special directory, so that we can see what the corruption actually is?
Re: [Dovecot] Broken dovecot-uidlist files
On Mon, Feb 23, 2009 at 12:10:29PM -0500, Timo Sirainen wrote: > On Mon, 2009-02-23 at 16:27 +0100, Ulrich Zehl wrote: > > I did > > > > # mount -o remount,actimeo=0 /srv/storage > > To both servers? There's three Dovecot servers (NFS clients), actually, and I remounted /srv/storage on all of them. > > > So allina modified dovecot-uidlist and soon afterwards laura probably > > > was using a cached dovecot-uidlist and corrupted it. > > > > Since the corrupted files are available for a little while (in the example, > > it was ~ 15 minutes), will it help if I repeatedly check all > > dovecot-uidlists and save those found to be corrupted to a special > > directory, so that we can see what the corruption actually is? > > I suppose looking at a couple of those could verify if it's really just > NFS caching related corruption or something else. Okay, I will post results when I have them.
Re: [Dovecot] Broken dovecot-uidlist files
On Tue, Feb 24, 2009 at 08:08:52AM +0100, Ulrich Zehl wrote: > On Mon, Feb 23, 2009 at 12:10:29PM -0500, Timo Sirainen wrote: > > I suppose looking at a couple of those could verify if it's really just > > NFS caching related corruption or something else. > > Okay, I will post results when I have them. I found a lot of files that hat null bytes in them, but no other problems (which could be due to the fact that my uidlist checking script isn't all that great). I have put two corrupted dovecot-uidlist and the corresponding log files, and the check script up at http://www.topfen.net/user/ul/dovecot/. I hope that's enough data.
[Dovecot] [PATCH] LMTP: Permit user IDs containing recipient_delimiter
The Dovecot 2.1 LMTP server currently always strips the address extension
from a recipient address (if recipient_delimiter is set), meaning user IDs
cannot contain the recipient delimiter character, e.g. "user+foo" is not
supported.
This was surprising for me, as Postfix behaves differently in this regard:
It first looks up "user+foo", and only then "user".
The attached patch works for me and brings Dovecot's behavior in line with
Postfix. Please let me know what you think about it.
diff -r 1ab8e0e699f7 src/lmtp/commands.c
--- a/src/lmtp/commands.c Wed Jan 09 07:01:41 2013 +0200
+++ b/src/lmtp/commands.c Sat Jan 12 17:25:27 2013 +0100
@@ -480,12 +480,39 @@
return ret;
}
+static int rcpt_user_lookup(struct client *client, struct mail_recipient *rcpt,
+ const char *username)
+{
+ struct mail_storage_service_input input;
+ const char *prefix;
+ const char *error = NULL;
+ int ret;
+
+ memset(&input, 0, sizeof(input));
+ input.module = input.service = "lmtp";
+ input.username = username;
+ input.local_ip = client->local_ip;
+ input.remote_ip = client->remote_ip;
+ input.local_port = client->local_port;
+ input.remote_port = client->remote_port;
+
+ ret = mail_storage_service_lookup(storage_service, &input,
+ &rcpt->service_user, &error);
+
+ if (ret < 0) {
+ prefix = t_strdup_printf(ERRSTR_TEMP_USERDB_FAIL_PREFIX,
+ username);
+ client_send_line(client, "%s%s", prefix, error);
+ return -1;
+ }
+
+ return ret;
+}
+
int cmd_rcpt(struct client *client, const char *args)
{
struct mail_recipient rcpt;
- struct mail_storage_service_input input;
- const char *address, *username, *detail, *prefix;
- const char *error = NULL;
+ const char *address, *username, *detail;
int ret = 0;
client->state.name = "RCPT TO";
@@ -508,36 +535,34 @@
client_send_line(client, "501 5.5.4 Unsupported options");
return 0;
}
- rcpt_address_parse(client, address, &username, &detail);
if (client->lmtp_set->lmtp_proxy) {
+ rcpt_address_parse(client, address, &username, &detail);
if (client_proxy_rcpt(client, address, username, detail))
return 0;
}
- memset(&input, 0, sizeof(input));
- input.module = input.service = "lmtp";
- input.username = username;
- input.local_ip = client->local_ip;
- input.remote_ip = client->remote_ip;
- input.local_port = client->local_port;
- input.remote_port = client->remote_port;
-
- ret = mail_storage_service_lookup(storage_service, &input,
- &rcpt.service_user, &error);
-
- if (ret < 0) {
- prefix = t_strdup_printf(ERRSTR_TEMP_USERDB_FAIL_PREFIX,
- username);
- client_send_line(client, "%s%s", prefix, error);
+ /* First, try looking up with the whole address as the username */
+ ret = rcpt_user_lookup(client, &rcpt, address);
+ if (ret > 0) {
+ username = address;
+ detail = "";
+ }
+ /* User not found, remove extension and try again */
+ if (ret == 0) {
+ rcpt_address_parse(client, address, &username, &detail);
+ /* Avoid second userdb lookup when unneccessary */
+ if (*detail != '\0')
+ ret = rcpt_user_lookup(client, &rcpt, username);
+ if (ret == 0) {
+ client_send_line(client,
+ "550 5.1.1 <%s> User doesn't exist: %s",
+ address, username);
+ return 0;
+ }
+ }
+ if (ret == -1)
return 0;
- }
- if (ret == 0) {
- client_send_line(client,
- "550 5.1.1 <%s> User doesn't exist: %s",
- address, username);
- return 0;
- }
if (client->proxy != NULL) {
/* NOTE: if this restriction is ever removed, we'll also need
to send different message bodies to local and proxy
Re: [Dovecot] [PATCH] LMTP: Permit user IDs containing recipient_delimiter
On Tue, Jan 15, 2013 at 08:44:57AM +0200, Timo Sirainen wrote: > On Sat, 2013-01-12 at 17:45 +0100, Ulrich Zehl wrote: > > The Dovecot 2.1 LMTP server currently always strips the address extension > > from a recipient address (if recipient_delimiter is set), meaning user IDs > > cannot contain the recipient delimiter character, e.g. "user+foo" is not > > supported. > > > > This was surprising for me, as Postfix behaves differently in this regard: > > It first looks up "user+foo", and only then "user". > > > > The attached patch works for me and brings Dovecot's behavior in line with > > Postfix. Please let me know what you think about it. > > What do you need this for? Do you really have usernames that have '+' > characters? Sounds like a bad idea. In Postfix it makes more sense, > because you might want user+ext to be delivered differently, but with > Dovecot you can do this in Sieve. My recipient_delimiter is actually '-', and I have a number of usernames that contain this character. I'd like to avoid having to change them. Additionally, I currently seperate address-lists@ and address-spam@ into two different mailboxes (accounts), something I'd also like to keep. But if you don't find these changes useful, I'll just go back to using LDA where this was/is possible. Easier to maintain than patching.
Re: [Dovecot] Dovecot Postfix Quota Policy Service
On Sat, May 04, 2013 at 06:02:36PM +0200, Robert Schetterer wrote: > Am 04.05.2013 09:12, schrieb Noel Butler: > > The quota-grace I think was only bit not backported, is that right? > > Did Timo do or announce plan to do this, or not happening for 2.1 only > > 2.2? > > at my last knowledge it wasnt backported and it will never done, so with > most setups, quota policy service in 2.1 is more or less useless in > reality, cause lda or lmtp will do the bounce job, so mailboxes mostly > may go never "over quota" I haven't actually tried it yet, so this is just from looking at the source code: The policy service will reject (most) messages that would put a mailbox over the quota limit in both 2.1 and 2.2, won't it? That still seems very useful, compared to bouncing it later.
Re: [Dovecot] Dovecot Postfix Quota Policy Service
On Sat, May 04, 2013 at 07:34:44PM +0200, Robert Schetterer wrote: > > > > I haven't actually tried it yet, so this is just from looking at the source > > code: The policy service will reject (most) messages that would put a > > mailbox over the quota limit in both 2.1 and 2.2, won't it? > > That still seems very useful, compared to bouncing it later. > > > > my understanding > you need quota-grace to make sure mailbox get overquota for setup > percent, if there ist no quota-grace ( like in 2.1.x ), most mail will > be bounced by normal lda/lmtp quota rules , so policy quota always will > seen some free space in the mailbox, unless the rare case that one ( > last ) mail fits the mailbox quota in exact 100 percent I just tested it, and now I can verify: As long as the size= attribute is present in the policy request, Dovecot will correctly reject messages that are too large to fit in whatever quota you have left.
Re: [Dovecot] Dovecot Postfix Quota Policy Service
I was able to replicate your problem with 2.1.16 rev 0fa68f3a8f6c (from
Stephan's auto-built packages).
I have the following configuration in 10-master.conf, and no special
configuration for the service in 90-quota.conf.
| service quota-status {
| executable = quota-status -p postfix
| inet_listener {
| port = 12340
| }
| client_limit = 1
| user = root
| }
When I first query the quota-status service, I get the correct response:
| $ printf "recipient=t...@example.org\nsize=1234\n\n" | nc 127.0.0.1 12340
| action=OK
|
But on every subsequent try, I always receive a response like this:
| $ printf "recipient=t...@example.org\nsize=1234\n\n" | nc 127.0.0.1 12340
| action=DEFER_IF_PERMIT Internal error occurred. Refer to server log for more
information.
|
Where the server log only says
| May 7 11:59:45 minna dovecot: quota-status(t...@example.org): Error: user
t...@example.org: Error reading configuration:
net_connect_unix(/var/run/dovecot/config) failed: Permission denied
Looking at the quota-status process, I notice it is not running as root,
but rather as $mail_uid. It seems the service drops / changes its
privileges at some point, which would explains the permission error on
subsequent requests.
Setting service_count=1 for the service is not a viable workaround, as
Postfix sends all policy requests for one SMTP session via one TCP
connection.
Re: [Dovecot] 2.2.4 - quota-status changing the user it is running as
On Thu, Aug 01, 2013 at 02:31:31PM +0200, Axel Luttgens wrote: > > http://hg.dovecot.org/dovecot-2.2/rev/2470bb9106b0 > http://hg.dovecot.org/dovecot-2.2/rev/51b8020b29f6 > http://hg.dovecot.org/dovecot-2.2/rev/eb63eca74471 > http://hg.dovecot.org/dovecot-2.2/rev/43488e1044c9 Is there any chance to get these backported to 2.1 as well? After all, it has the same issues without these patches. The individual patches seem to apply just fine to current 2.1 tip without any code changes: $ patch -p1 < p/2470bb9106b0 patching file src/lib-storage/mail-storage-service.c Hunk #1 succeeded at 946 (offset -21 lines). Hunk #2 succeeded at 966 (offset -21 lines). $ patch -p1 < p/51b8020b29f6 patching file src/plugins/quota/quota-status.c Hunk #1 succeeded at 218 (offset 3 lines). $ patch -p1 < p/eb63eca74471 patching file src/lib-master/master-service-settings.c Hunk #1 succeeded at 323 (offset -13 lines). patching file src/lib-master/master-service-settings.h Hunk #1 succeeded at 53 (offset -3 lines). patching file src/lib-master/master-service.c Hunk #1 succeeded at 229 (offset -7 lines). patching file src/lib-master/master-service.h Hunk #1 succeeded at 12 (offset -3 lines). $ patch -p1 < p/43488e1044c9 patching file src/lib-storage/mail-storage-service.c Hunk #1 succeeded at 728 with fuzz 2 (offset -20 lines). After that, everything works just fine and as expected, as Axel described for 2.2: - Running under non-root user is possible - Querying for more than one user under the same connection is possible $ printf "recipient=m.musterm...@example.net\nsize=1234\n\nrecipient=m.musterm...@example.net\nsize=1234\n\nrecipient=t...@example.net\nsize=1234\n\n" | nc 127.0.0.1 12340 action=OK action=REJECT Over quota action=REJECT Over quota So it seems to me that I can apply them well enough on my own. I'd just like them to come with the default source, so I cannot forget them when upgrading. :-) Ulrich
Re: [Dovecot] Postfix aliases with quota-status service
On Tue, Jul 30, 2013 at 03:20:47PM +0200, Thomas Leuxner wrote: > This is probably intended behaviour, just want to make sure that I'm not > missing a point here. For now the only fix that comes to my mind to create > "quota aware" aliases - is creating 'dummy' users in Dovecot which point to > the same mailbox rather than performing aliasing on the Postfix end. Open > to suggestions... It depends on your user and/or alias database (i.e. how and where they're stored). I use LDAP, and configured Dovecot's userdb lookup to handle both "main" and "alias" addresses, like so: user_filter = (|(mail=%u)(mailAlternateAddress=%u)) (As far as Dovecot is concerned, there's actually no difference between "main" and "alias".) If you store your mailbox and alias information in the same data source (LDAP, SQL, ...), you should be able to do the same.
[Dovecot] [PATCH] quota-status: remove unnecessary newlines
Here's a trivial patch for quota-status.c that removes unecessary newlines
present in the default overquota message. It applies to 2.2 and 2.1.
Ulrich
--- a/src/plugins/quota/quota-status.c 2013-08-01 14:35:35.0 +0200
+++ b/src/plugins/quota/quota-status.c 2013-08-01 17:58:26.0 +0200
@@ -107,7 +107,7 @@
/* over quota */
value = mail_user_plugin_getenv(user,
"quota_status_overquota");
if (value == NULL)
- value = t_strdup_printf("554 5.2.2 %s\n\n",
error);
+ value = t_strdup_printf("554 5.2.2 %s", error);
}
value = t_strdup(value); /* user's pool is being freed */
mail_user_unref(&user);
[Dovecot] [PATCH] quota-status: allow different action for messages that are too large (over maximum quota)
This proposed patch to quota-status allows the administrator to return
different actions for messages that (a) cannot currently fit in the quota
(because there's not enough free space) and (b) are over the maximum quota
limit (and could not even be delivered if the mailbox was empty).
Configuration works like this:
plugin {
quota_status_overquota = DEFER_IF_PERMIT 4.2.2 Mailbox full
quota_status_toolarge = REJECT 5.2.3 Message length exceeds administrative
limit
}
The patch is backwards compatible: If quota_status_toolarge is not set,
quota_status_overquota is returned. No changes are necessary in setups
where the new feature is not desired.
The reason I developed this patch is: In my environment, I have mostly POP3
mailboxes, where condition (a) changes frequently (usually after each
login), but condition (b) will almost never change (users do not get
additional mailbox space often). Thus, I'd like to avoid bouncing mails
that could be delivered half an hour later.
This patch works for 2.2 and 2.1. (This assumes you have already applied
the trivial patch I sent earlier; otherwise patch(1) will report some
fuzz.)
What do you think of this?
Ulrich
--- a/src/plugins/quota/quota-status.c 2013-08-01 18:05:24.0 +0200
+++ b/src/plugins/quota/quota-status.c 2013-08-01 18:03:30.0 +0200
@@ -46,13 +46,12 @@
}
static int
-quota_check(struct mail_user *user, uoff_t mail_size, const char **error_r)
+quota_check(struct mail_user *user, uoff_t mail_size, const char **error_r,
bool *too_large)
{
struct quota_user *quser = QUOTA_USER_CONTEXT(user);
struct mail_namespace *ns;
struct mailbox *box;
struct quota_transaction_context *ctx;
- bool too_large;
int ret;
if (quser == NULL) {
@@ -64,7 +63,7 @@
box = mailbox_alloc(ns->list, "INBOX", 0);
ctx = quota_transaction_begin(box);
- ret = quota_test_alloc(ctx, I_MAX(1, mail_size), &too_large);
+ ret = quota_test_alloc(ctx, I_MAX(1, mail_size), too_large);
quota_transaction_rollback(&ctx);
mailbox_free(&box);
@@ -82,6 +81,7 @@
struct mail_storage_service_user *service_user;
struct mail_user *user;
const char *value = NULL, *error;
+ bool too_large;
int ret;
if (client->recipient == NULL) {
@@ -98,14 +98,21 @@
if (ret == 0) {
value = nouser_reply;
} else if (ret > 0) {
- if ((ret = quota_check(user, client->size, &error)) > 0) {
+ if ((ret = quota_check(user, client->size, &error, &too_large))
> 0) {
/* under quota */
value = mail_user_plugin_getenv(user,
"quota_status_success");
if (value == NULL)
value = "OK";
} else if (ret == 0) {
- /* over quota */
- value = mail_user_plugin_getenv(user,
"quota_status_overquota");
+ if (too_large == TRUE) {
+ /* even over maximum quota */
+ value = mail_user_plugin_getenv(user,
"quota_status_toolarge");
+ if (value == NULL) /* backwards compatibility */
+ value = mail_user_plugin_getenv(user,
"quota_status_overquota");
+ } else {
+ /* over quota */
+ value = mail_user_plugin_getenv(user,
"quota_status_overquota");
+ }
if (value == NULL)
value = t_strdup_printf("554 5.2.2 %s", error);
}
Re: [Dovecot] Postfix aliases with quota-status service
On Tue, Aug 06, 2013 at 09:20:13PM +0200, Thomas Leuxner wrote: > Now everything in between seems to create SMTPD rejections in some cases > _or_ queue the mail and let it hit the quota in other cases. That's my > whole point... I'm sorry, I don't get your point. Are you saying that quota-status does not eliminate all over-quota bounces? That's to be expected. quota-status does not reserve quota when questioned, it only tells Postfix whether enough space is currently free or not. This is not free of race conditions, and therefore cannot eliminate all late bounces, even when quota-status has all the information, including size. For example, if a mailbox has 3000 bytes of free space, and two 2000 byte messages arrive close enough together so that the first one has not been delivered by the time the second one sends "RCPT TO ... SIZE=...", quota-status will allow both messages, even though the second one will be rejected when Postfix actually tries local delivery (LTMP/LDA). Are you saying that rejects depend on SIZE= being sent during the RCPT TO stage (i.e., messages that announce their size correctly are rejected during the SMTP transaction, while those without size inidcation are passed)? If so, follow Rob's suggestion, and run quota-status (again) as part of smtpd_end_of_data_restrictions. By then, Postfix will know the size of a message and pass it to quota-status. (This only works for single-recipient messages; multi-recipient messages will always pass at this stage, because Postfix' policy protocol will not send "recipient=..." in this case, and thus quota-status replies with "DUNNO". In my environment, most messages are single-recipient, so it works well enough for me.) Are you saying something else that I missed? Please tell me, because I'd like to understand your point.
[Dovecot] Override master service settigs with spaces
I'd like to override one setting for a master service in
conf.d/10-master.conf. Unfortunately, said setting contains spaces, and I
do not know how to escape them properly.
Here's what I've tried so far. (Note: This is just the easiest/silliest
test case I could come up with; not the actual setting or service I want to
overwrite.)
conf.d/10-master.conf:
service quota-status {
executable = quota-status -p postfix -o plugin/quota_status_success=Testing 1
2 3
inet_listener {
port = 12340
}
}
Reaction:
$ printf "recipient=t...@example.org\nsize=0\n\n" | nc 127.0.0.1 12340
action=Testing
conf.d/10-master.conf:
service quota-status {
executable = quota-status -p postfix -o "plugin/quota_status_success=Testing
1 2 3"
inet_listener {
port = 12340
}
}
Reaction:
$ printf "recipient=t...@example.org\nsize=0\n\n" | nc 127.0.0.1 12340
action=DUNNO [<-- This is the default setting]
conf.d/10-master.conf:
service quota-status {
executable = quota-status -p postfix -o 'plugin/quota_status_success=Testing
1 2 3'
inet_listener {
port = 12340
}
}
Reaction:
$ printf "recipient=t...@example.org\nsize=0\n\n" | nc 127.0.0.1 12340
action=DUNNO [<-- This is the default setting]
conf.d/10-master.conf:
service quota-status {
executable = quota-status -p postfix -o plugin/quota_status_success="Testing
1 2 3"
inet_listener {
port = 12340
}
}
Reaction:
$ printf "recipient=t...@example.org\nsize=0\n\n" | nc 127.0.0.1 12340
action="Testing [<-- Note the " after = ]
conf.d/10-master.conf:
service quota-status {
executable = quota-status -p postfix -o plugin/quota_status_success=Testing\
1\ 2\ 3
inet_listener {
port = 12340
}
}
Reaction:
$ printf "recipient=t...@example.org\nsize=0\n\n" | nc 127.0.0.1 12340
action=Testing\ [<-- Note the \ ]
I'm out of ideas now. What's the correct way to quote / escape these
options?
Re: [Dovecot] Override master service settigs with spaces
On Wed, Aug 07, 2013 at 10:49:28PM +0200, Axel Luttgens wrote:
>
> What makes you think quota-status would accept a -o option?
Every service running under master accepts -o, as far as I can tell from
the code. (Additionally verified by my tests; for settings without spaces
it works well.)
> And what's that "quota_status_success" plugin supposed to do?
It's just a setting that you'd normally set in conf.d/90-quota.conf (or
similar) like this:
plugin {
quota_status_success = DUNNO
}
It tells quota-status what action to return to Postfix when the message
will fit into quota, just like quota_status_overquota (and
quota_status_toolarge since 2.2.5) do for messages that will not fit.
