[OT] Extdata / Extprograms Plugins on CentOS 7?
Hello, I want to install Dovecot Pigeonhole and use the Extdata and Extprograms plugins on CentOS 7. I prefer to install software via yum, and a reasonably new version of Dovecot is available in the CentOS repo. But according to the dovecot documentation, these plugins need to be compiled, so I don't know if yum is out of the question? * Does anyone know if either of these plugins built into the Pigeonhole that is available in the CentOS 7 extras repo? OR if there is a version of Pigeonhole available in a different 3rd party repo with what I want? (like EPEL, DAG, etc) * Is it possible to use these plugins with Pigeonhole installed from yum? Does the compilation requirement mean that I have to remove not only Pigeonhole, but also Dovecot itself from my system and build everything by hand? :-( THANK YOU!!
Sieve can't find Extprograms or Extdata
Hi, On a new install-from-source with Dovecot 2.2.16rc1, Piegeonhole 0.4.6 and a grab of the newest Extdata code, I confirmed basic Sieve functionality is working (made a simple sieve script with a test on message subject resulting in a fileinto action). But I cannot get Sieve to see Extdata or Extprograms. sievec reports "Warning: sieve: ignored unknown extension 'vnd.dovecot.filter' while configuring available extensions" and the same for vnd.dovecot.extdata. And of course for the user script, "error: require command: unknown Sieve capability `vnd.dovecot.filter'" (same with extdata) I'm pretty sure this is a library path problem, but I've tried symlinking and restarting dovecot with as many iterations of library paths as I can think of but still same problem. The lib90_sieve_extdata_plugin.so and lib90_sieve_extprograms_plugin.so are in /usr/local/lib/dovecot/sieve and all the rest of the normal dovecot libraries are one level above in /usr/local/lib/dovecot. I tried creating symlinks for a "modules" directory and a "modules/sieve" directory, I tried symlinking these two librarie files themselves into the top level dovecot library directory, and a few other ideas, but no luck. Is there a way I can tell where Sieve expects its libraries to reside? To see where it is looking? I didn't provide any arguments to the configure command for pigeonhole as well as extdata and it seems like the libs got placed somewhere sensible, but I don't know what else to do.
Re: Sieve can't find Extprograms or Extdata
Also, of course I have entered this in 90-plugins.conf
plugin {
seive_plugins = sieve_extdata sieve_extprograms
sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.extdata
}
And enabled sieve for lmtp (as I noted, I have tested that simple
sieve scripting is working OK)
protocol lmtp {
mail_plugins = " sieve"
}
> On a new install-from-source with Dovecot 2.2.16rc1,
> Piegeonhole 0.4.6 and a grab of the newest Extdata code, I
> confirmed basic Sieve functionality is working (made a
> simple sieve script with a test on message subject resulting
> in a fileinto action).
>
> But I cannot get Sieve to see Extdata or Extprograms. sievec
> reports "Warning: sieve: ignored unknown extension
> 'vnd.dovecot.filter' while configuring available extensions"
> and the same for vnd.dovecot.extdata. And of course
> for the user script, "error: require command: unknown Sieve
> capability `vnd.dovecot.filter'" (same with extdata)
>
> I'm pretty sure this is a library path problem, but I've
> tried symlinking and restarting dovecot with as many
> iterations of library paths as I can think of but still same
> problem.
>
> The lib90_sieve_extdata_plugin.so and
> lib90_sieve_extprograms_plugin.so are in
> /usr/local/lib/dovecot/sieve and all the rest of the normal
> dovecot libraries are one level above in
> /usr/local/lib/dovecot.
>
> I tried creating symlinks for a "modules" directory and a
> "modules/sieve" directory, I tried symlinking these two
> librarie files themselves into the top level dovecot library
> directory, and a few other ideas, but no luck.
>
> Is there a way I can tell where Sieve expects its libraries
> to reside? To see where it is looking? I didn't provide any
> arguments to the configure command for pigeonhole as well as
> extdata and it seems like the libs got placed somewhere
> sensible, but I don't know what else to do.
Re: Sieve can't find Extprograms or Extdata
> > Also, of course I have entered this in 90-plugins.conf
> >
> >
> > plugin {
> > seive_plugins = sieve_extdata sieve_extprograms
>
> maybe you have a typo...
> seive != sieve
Oh yes that was problem!! Thank you!! Tricky because it's the plugins
section, dovecot cannot lint it so typos won't be detected.
Thank you one time more!
And to Stephan Bosch, consider this a vote to keep Extdata alive.
It's better in many ways than spawning a script to do data lookups.
I hope the dovecot dict mechanism can be enhanced some day to
allow more elaborate controls and customization of lookups.
Overriding dovecot.conf from Userdb Extras
Hi, I thought I read that anything from dovecot.conf can be overridden in a userdb lookup. Or a passdb lookup with "userdb_" prefix. But I tried for fun change log_path but it never worked. Is that because logging is special, already started logging before it comes to the passdb/userdb lookups? So are there some dovecot.conf settings that cannot be overridden? Thanks!
Re: Overriding dovecot.conf from Userdb Extras
> I thought I read that anything from dovecot.conf can be > overridden in a > userdb lookup. Or a passdb lookup with "userdb_" prefix. > > But I tried for fun change log_path but it never worked. Is > that because > logging is special, already started logging before it comes > to the > passdb/userdb lookups? So are there some dovecot.conf > settings > that cannot be overridden? Any takers?
Re: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
> Last time I had a few stupid problems in the releases, so > I'll follow > Timo's example and I release an RC first. > > The highlights include the implementation of the index and > metadata > extensions. Quite a few bugs are fixed as well. When I compiled and installed this, Sieve scripts were being ignored. Not sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it worked fine. No configuration changes, only make install on the different sources and restart dovecot.
Why is Sieve trying to re-compile global scripts?
I have some global scripts that were running nicely. Then I opened one in an editor and (probably, but not 100% sure) mindlessly saved the file, even though I hadn't made any changes. Shortly after, Sieve errors started showing in the log: Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create temporary file: open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: Permission denied... Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/usr/local/var/dovecot/sieve/script2.sieve' need to be pre-compiled using the sievec tool Well, OK, is it going by the timestamp on the files? Fine. I recompiled it by hand. Yet, I STILL got these errors! I triple and quadruple checked that the timestamp on the svbin files was more recent. And Sieve was only complaining about one of the two scripts in the directory. I restarted dovecot. No change. So I removed read permission on the .sieve files and only left read permission on the .svbin files. THIS WORKED. No more error. I can live with that, but why was it not complaining before, why was it only complaining about one of my scripts and why would it complain at all when the timestamps on the svbin should have indicated on compilation is needed?
Re: Why is Sieve trying to re-compile global scripts?
> > I have some global scripts that were running nicely. > > > > Then I opened one in an editor and (probably, but not 100% sure) > > mindlessly saved the file, even though I hadn't made any changes. > > > > Shortly after, Sieve errors started showing in the log: > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create > > temporary file: > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: > Permission denied... > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have > > permission to save global > Sieve script binaries; global Sieve scripts like > `/usr/local/var/dovecot/sieve/script2.sieve' > need to be pre-compiled using the sievec tool > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > it by hand. Yet, I STILL got these errors! > > > > I triple and quadruple checked that the timestamp on the svbin files was > > more recent. And Sieve was only complaining about one of the two > > scripts in the directory. > > > > I restarted dovecot. No change. > > > > So I removed read permission on the .sieve files and only left read > > permission on the .svbin files. THIS WORKED. No more error. > > I can live with that, but why was it not complaining before, why was it > > only complaining about one of my scripts and why would it complain > > at all when the timestamps on the svbin should have indicated on > > compilation is needed? > > I've heard about this problem before. Do you have the opportunity to > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > (shown when mail_debug=yes) that would indicate why Sieve is thinking > the global script is not up-to-date. Yes, I do as a matter of fact. I was just going to put in the RC in order to answer your email on the thread about the RC. Don't have the full answers yet, but when I installed the RC and restarted, I now get an error where Sieve doesn't like that I won't give it read permission on the .sieve file, so now I'm back to square one with this particular issue. OTOH, regarding my earlier post about the RC ignoring seive files, at least it is seeing global scripts (or trying to). Not sure about personal scripts yet. Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve script: open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... I will do some more testing and report what I find.
Re: Why is Sieve trying to re-compile global scripts?
> > > I have some global scripts that were running nicely. > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create > > > temporary file: > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) failed: > > Permission denied... > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not have > > > permission to save global > > Sieve script binaries; global Sieve scripts like > > `/usr/local/var/dovecot/sieve/script2.sieve' > > need to be pre-compiled using the sievec tool > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I recompiled > > > it by hand. Yet, I STILL got these errors! > > > > > > I triple and quadruple checked that the timestamp on the svbin files was > > > more recent. And Sieve was only complaining about one of the two > > > scripts in the directory. > > > > > > I restarted dovecot. No change. > > > > > > So I removed read permission on the .sieve files and only left read > > > permission on the .svbin files. THIS WORKED. No more error. > > > I can live with that, but why was it not complaining before, why was it > > > only complaining about one of my scripts and why would it complain > > > at all when the timestamps on the svbin should have indicated on > > > compilation is needed? > > > > I've heard about this problem before. Do you have the opportunity to > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > the global script is not up-to-date. > > Yes, I do as a matter of fact. I was just going to put in the RC in > order to answer your email on the thread about the RC. Don't have the > full answers yet, but when I installed the RC and restarted, I now get > an error where Sieve doesn't like that I won't give it read permission > on the .sieve file, so now I'm back to square one with this particular > issue. > > OTOH, regarding my earlier post about the RC ignoring seive files, at > least it is seeing global scripts (or trying to). Not sure about > personal scripts yet. > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve > script: > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission denied... > > I will do some more testing and report what I find. I gave read permission to the .sieve files and the same original error happens as with .0.4.6. Now it's complaining about both scripts in my global directory. That it was working without these errors for a while and then complained only about one of the scripts, now both scripts seems to say something but I'm not sure what. Maybe I'll try to recreate the files for fun. I'll test personal scripts again too...
Re: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
> >> Last time I had a few stupid problems in the releases, so
> >> I'll follow
> >> Timo's example and I release an RC first.
> >>
> >> The highlights include the implementation of the index and
> >> metadata
> >> extensions. Quite a few bugs are fixed as well.
> > When I compiled and installed this, Sieve scripts were being ignored. Not
> > sure if it's my own stupid mistake, but when I put v0.4.6 back in place, it
> > worked fine. No configuration changes, only make install on the different
> > sources and restart dovecot.
>
> Could you show your dovecot -n output?
>
> Also, if you enable mail_debug, what sieve-related debug lines are shown?
OK, I re-tested and it's still ignoring personal scripts (but not
global ones). No .svbin gets generated, no errors, just nothing.
However, I do see that Sieve was accessing the user home directory
because for some reason now it just created a ".pki" directory
therein, which inside of it has an empty "nssdb" directory. That
never happened before...? Not a big problem, but I'd prefer not
to have that there.
Re: mail_debug, this relates to another post I made that didn't get any
replies - can I not override settings such as that (and log_path) from
a userdb lookup? Hmm, I WAS able to override mail_debug from userdb,
but not log_path?
Sieve-related mail_debug, then?
This looks like the relevant log info:
dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: file
storage: Storage path `/vmail/example.com/testuser/sieve' not found
dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: No
default script configured for user
dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: User
has no personal script
I'll check on 0.4.6 and report if I see anything interesting,
but I will assume for the moment that since personal scripts
work in 0.4.6 that this log info won't be there. It is correct that
there is no "sieve" file or directory in the user's home dir. This
wasn't a problem in 0.4.6. Is it a requirement?
Also, if you didn't see my post a few days ago, while I have your
attention, I thank you for the extdata plugin and vote to keep it
alive. Only caveat is I hope to see the dict mechanism in Dovecot
become more flexible... at a minimum, would like to be able to pass
at least one parameter (beside the implicit username) and indicate
what field to test it against (I would like at least a simply-configured
WHERE clause) if not full query customization. :) Perhaps not easy
stuff, but extdata as is still helps me out.
doveconf -n
# 2.2.16.rc1: /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.7.rc1 (c74220e16e0f+)
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406
(Core) xfs
dict {
sieve = mysql:/usr/local/etc/dovecot/pigeonhole-sql.dict.ext
}
mail_location = mdbox:/vmail/%d/%1Mn/%1.1Mn/%n
mail_plugins = " zlib"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_before = /usr/local/var/dovecot/sieve/
sieve_execute_bin_dir = /usr/local/var/dovecot/sieve-extscripts/
sieve_extdata_dict_uri = proxy::sieve
sieve_global_extensions = +vnd.dovecot.extdata +vnd.dovecot.execute
sieve_plugins = sieve_extdata sieve_extprograms
zlib_save = gz
zlib_save_level = 9
}
service auth-worker {
user = $default_internal_user
}
service dict {
unix_listener dict {
group = vmail
mode = 0660
}
}
service imap-login {
inet_listener imap {
address = 127.0.0.1
}
}
service lmtp {
inet_listener lmtp {
address = 127.0.0.1
port = 24
}
}
ssl_cert =
Re: Why is Sieve trying to re-compile global scripts?
> > > > I have some global scripts that were running nicely. > > > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create > > > > temporary file: > > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) > > > failed: Permission denied... > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not > > > > have permission to save global > > > Sieve script binaries; global Sieve scripts like > > > `/usr/local/var/dovecot/sieve/script2.sieve' > > > need to be pre-compiled using the sievec tool > > > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I > > > > recompiled > > > > it by hand. Yet, I STILL got these errors! > > > > > > > > I triple and quadruple checked that the timestamp on the svbin files was > > > > more recent. And Sieve was only complaining about one of the two > > > > scripts in the directory. > > > > > > > > I restarted dovecot. No change. > > > > > > > > So I removed read permission on the .sieve files and only left read > > > > permission on the .svbin files. THIS WORKED. No more error. > > > > I can live with that, but why was it not complaining before, why was it > > > > only complaining about one of my scripts and why would it complain > > > > at all when the timestamps on the svbin should have indicated on > > > > compilation is needed? > > > > > > I've heard about this problem before. Do you have the opportunity to > > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > > the global script is not up-to-date. > > > > Yes, I do as a matter of fact. I was just going to put in the RC in > > order to answer your email on the thread about the RC. Don't have the > > full answers yet, but when I installed the RC and restarted, I now get > > an error where Sieve doesn't like that I won't give it read permission > > on the .sieve file, so now I'm back to square one with this particular > > issue. > > > > OTOH, regarding my earlier post about the RC ignoring seive files, at > > least it is seeing global scripts (or trying to). Not sure about > > personal scripts yet. > > > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve > > script: > > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission > > denied... > > > > I will do some more testing and report what I find. > > I gave read permission to the .sieve files and the same > original error happens as with .0.4.6. Now it's complaining > about both scripts in my global directory. That it was > working without these errors for a while and then complained > only about one of the scripts, now both scripts seems to say > something but I'm not sure what. Maybe I'll try to recreate the > files for fun. The relevant mail_debug lines seem to be these: dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Opening script 1 of 2 from `/usr/local/var/dovecot/sieve/script1.sieve' dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Loading script /usr/local/var/dovecot/sieve/script1.sieve dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: binary open: binary /usr/local/var/dovecot/sieve/script1.svbin stored with different binary version 1.2 (!= 1.3; automatically fixed when re-compiled) dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: Script `script1' from /usr/local/var/dovecot/sieve/script1.sieve successfully compiled Is this possibly due to a mixing of 0.4.6 and 0.4.7 sievec command? Well, I'm not sure that would be it because when I started getting ther error, I recompiled the sieve scrips and restarted dovecot which presumably would have made software versions match up. On the other hand, I don't know exactly what's happening: I downgraded to 0.4.6 again, intentionally triggered the error by updating the timestamp on the .sieve file, recompiled the script and now the error went away.
Extending dict lookups (SQL)?
Hi, How difficult would it be to try to hack some extension to the Dovecot dict mechanism for someone unfamiliar with the code? I'm using SQL as a backend and am looking for, at a minimum, the ability to specify a WHERE clause in addition to the built-in one that feeds the current username. The field name to test the additional value against would be necessary as well of course. Naturally, full query customization would be great to have. Perhaps providing a mechanism for the caller to pass data for the extra WHERE clause is the hardest part, not sure... but it can't hurt to ask. Don't take this as lack of gratitude - I can work with what's there and greatly appreciate it.
Re: Why is Sieve trying to re-compile global scripts?
> > > > > I have some global scripts that were running nicely. > > > > > > > > > > Then I opened one in an editor and (probably, but not 100% sure) > > > > > mindlessly saved the file, even though I hadn't made any changes. > > > > > > > > > > Shortly after, Sieve errors started showing in the log: > > > > > > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: binary save: failed to create > > > > > temporary file: > > > > open(/usr/local/var/dovecot/sieve/script2.svbin.example.com.4139.) > > > > failed: Permission denied... > > > > > Error: 4k5JA74R/1TlIwABG/SpMA: sieve: The LDA Sieve plugin does not > > > > > have permission to save global > > > > Sieve script binaries; global Sieve scripts like > > > > `/usr/local/var/dovecot/sieve/script2.sieve' > > > > need to be pre-compiled using the sievec tool > > > > > > > > > > Well, OK, is it going by the timestamp on the files? Fine. I > > > > > recompiled > > > > > it by hand. Yet, I STILL got these errors! > > > > > > > > > > I triple and quadruple checked that the timestamp on the svbin files > > > > > was > > > > > more recent. And Sieve was only complaining about one of the two > > > > > scripts in the directory. > > > > > > > > > > I restarted dovecot. No change. > > > > > > > > > > So I removed read permission on the .sieve files and only left read > > > > > permission on the .svbin files. THIS WORKED. No more error. > > > > > I can live with that, but why was it not complaining before, why was > > > > > it > > > > > only complaining about one of my scripts and why would it complain > > > > > at all when the timestamps on the svbin should have indicated on > > > > > compilation is needed? > > > > > > > > I've heard about this problem before. Do you have the opportunity to > > > > test this with the 0.4.7.rc1 release? That adds a few extra debug lines > > > > (shown when mail_debug=yes) that would indicate why Sieve is thinking > > > > the global script is not up-to-date. > > > > > > Yes, I do as a matter of fact. I was just going to put in the RC in > > > order to answer your email on the thread about the RC. Don't have the > > > full answers yet, but when I installed the RC and restarted, I now get > > > an error where Sieve doesn't like that I won't give it read permission > > > on the .sieve file, so now I'm back to square one with this particular > > > issue. > > > > > > OTOH, regarding my earlier post about the RC ignoring seive files, at > > > least it is seeing global scripts (or trying to). Not sure about > > > personal scripts yet. > > > > > > Error: TiQJHH2X/1S5UuAAM/SpMA: sieve: file script: Failed to open sieve > > > script: > > > open(/usr/local/var/dovecot/sieve/script1.sieve) failed: Permission > > > denied... > > > > > > I will do some more testing and report what I find. > > > > I gave read permission to the .sieve files and the same > > original error happens as with .0.4.6. Now it's complaining > > about both scripts in my global directory. That it was > > working without these errors for a while and then complained > > only about one of the scripts, now both scripts seems to say > > something but I'm not sure what. Maybe I'll try to recreate the > > files for fun. > > The relevant mail_debug lines seem to be these: > > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: > sieve: Opening script 1 of 2 > from `/usr/local/var/dovecot/sieve/script1.sieve' > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: > sieve: Loading script /usr/local/var/dovecot/sieve/script1.sieve > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: > sieve: binary open: binary > /usr/local/var/dovecot/sieve/script1.svbin stored with different binary > version 1.2 (!= 1.3; > automatically fixed when re-compiled) > dovecot: lmtp(testuser example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: > sieve: Script `script1' from > /usr/local/var/dovecot/sieve/script1.sieve successfully compiled > > Is this possibly due to a mixing of 0.4.6 and 0.4.7 sievec command? > Well, I'm not sure that would be it because when I started getting > ther error, I recompiled the sieve scrips and restarted dovecot > which presumably would have made software versions match up. > > On the other hand, I don't know exactly what's happening: I downgraded > to 0.4.6 again, intentionally triggered the error by updating the > timestamp on the .sieve file, recompiled the script and now the > error went away. After editing one of the global scripts (and compiling it), I am able to get the error back again (and not able to get it to go away). The previous log info I found may have been unrelated and more to do with haivng switched to 0.4.7 without recompiling the scripts. I'm back with 0.4.6 and the only thing I see in the log now is this: Debug: lRgL3tO1/1RvOyA6M/SpMA: sieve: Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date Debug: lRgL3tO1/1RvOyA6M/SpMA: sieve: Script `script2' from /usr/local/var
Sieve extprograms socket vs. direct execution
Hi, I'm hoping to get some clarification of the differences between calling a script using the Sieve extprograms plugin execute method via direct execution or using the socket feature. Being naive, I see the socket option and think that way you tell Dovecot to spawn a daemon and I think that's going to be far superior in performance. But if that really was the difference, why would direct execution even be an option? (And doesn't the daemon still have to spawn a new shell every time if the target is a shell script? Does that defeat the purpose of having a daemon?) So I don't really know what the difference is, and under what circumstances you'd want to use one or the other. Can someone please help clarify?
Sieve reject with ORIG-TO vs TO
Hi, The bounce message generated by the reject extension has what looks like a hard coded message prefix that comes before the configurable reason text: "Your message to was automatically rejected:" In some cases, the is NOT the original-to address, which can cause confusion to the sender or expose private aliasing data that some people might want to hide. Is there a way to make the reject extension use the original-to address in that hard-coded message prefix? Is there a way to completely customize the FULL message? Is there a way to customize the headers for such messages?
Sieve security: Any way to protect credentials used in extprograms?
I need to connect to a database in a script called using Sieve extprograms plugin. When delivering mail, Sieve is running as the mail recipient user, which means any files, either the sieve script or the extprograms it invokes, are run under that user's permissions. What would be a way to hide the database credentials in a more restricted file? I can think of... * Store the credentials in the database itself, return them from a DICT lookup that is fed to the sieve script. The credentials would have to be duplicated for every user record in the database. * Somehow put the value in the Sieve environment, but I'm not sure how to get a value into the environment (just return it in a userdb lookup?) and then how to get at that value once I'm executing the sieve script (can I retrieve it with the "environment" extension?) Are there better ways? [Of course, if I could do more elaborate things like call a stored procedure from Dovecot DICT then I could avoid this situation]
Re: Why is Sieve trying to re-compile global scripts?
> Not sure how I got it to go away last time. Might have gotten it to go away by deleting the scripts, causing an email delivery, THEN creating the scripts again. Although I think my ideas are all flawed: I can delete the scripts and recreate and recompile all in the same minute and I don't get errors. I can cause the error to happen again by editing and recompiling one of the files, *whether or not in the same minute* and I do get the error. This time around, deleting the files and recreating/recompiling them even without an email delivery in between seems to fix the error. Might be unpredictable caching. Might be the error didn't go away last time I recreated due to different methods of creating the files. Who knows, I think I should give up and stop spamming the list with uneducated guesswork.
Re: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
> >>> When I compiled and installed this, Sieve scripts were being ignored. Not > >>> sure if it's my own stupid mistake, but when I put v0.4.6 back in place, > >>> it > >>> worked fine. No configuration changes, only make install on the different > >>> sources and restart dovecot. > >> Could you show your dovecot -n output? > >> > >> Also, if you enable mail_debug, what sieve-related debug lines are shown? > > OK, I re-tested and it's still ignoring personal scripts (but not > > global ones). No .svbin gets generated, no errors, just nothing. > > However, I do see that Sieve was accessing the user home directory > > because for some reason now it just created a ".pki" directory > > therein, which inside of it has an empty "nssdb" directory. That > > never happened before...? Not a big problem, but I'd prefer not > > to have that there. > > Sieve doesn't do that. I don't think Dovecot does that either, but I am > not sure. Odd. Some lib Sieve uses? These directories do not appear in user home directories unless I install the newest Sieve (and not until a delivery via LMTP happens). No other changes. No other software is currently accessing user home locations at all. > > Re: mail_debug, this relates to another post I made that didn't get any > > replies - can I not override settings such as that (and log_path) from > > a userdb lookup? Hmm, I WAS able to override mail_debug from userdb, > > but not log_path? > > > > Sieve-related mail_debug, then? > > > > This looks like the relevant log info: > > > > dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: > > file storage: Storage path `/vmail/example.com/testuser/sieve' not found > > dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: > > No default script configured for user > > dovecot: lmtp(testu...@example.com): Debug: Be3h7iRf/1TnUw2PM/SpMA: sieve: > > User has no personal script > > > > I'll check on 0.4.6 and report if I see anything interesting, > > but I will assume for the moment that since personal scripts > > work in 0.4.6 that this log info won't be there. It is correct that > > there is no "sieve" file or directory in the user's home dir. This > > wasn't a problem in 0.4.6. Is it a requirement? > > Well, since your config says the following: > > sieve = file:~/sieve;active=~/.dovecot.sieve > > It expects a sieve storage directory at ~/sieve (created when > ManageSieve is used to upload a script). > Also, a symbolic link pointing to the active script will be located at > ~/.dovecot.sieve once a script is activated (i.e. through ManageSieve or > doveadm sieve). > > I wonder how this would have worked before with 0.4.6. Is the > ~/.dovecot.sieve a normal script file perhaps (rather than a symlink)? > This would mean that the following config would work (e.g. if you don't > use ManageSieve): > > sieve = file:~/.dovecot.sieve The configuration for that was not of my doing (doesn't that mean it shouldn't have shown up in doveconf -n?). Yes, the .sieve scripts in user home are regular files. Strange 0.4.6 didn't mind this situation, but seems easy to put the configuration right and move on.
Re: Why is Sieve trying to re-compile global scripts?
> Since E.B. still got an obscure debug message about metadata not being > up to date, I added debug lines to the remaining places where this could > emerge (currently only available from hg). Using hg from just now - first line looks like what you want: dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: file script: Binary reports different script location (`script2.sieve' rather than `/usr/local/var/dovecot/sieve/script2.sieve') dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: binary up-to-date: script metadata indicates that binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date
Re: Why is Sieve trying to re-compile global scripts?
> Using hg from just now - first line looks like what you want: > > dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: > file script: Binary reports different script location (`script2.sieve' rather > than `/usr/local/var/dovecot/sieve/script2.sieve') > dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: > binary up-to-date: script metadata indicates that binary > /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date > dovecot: lmtp(testu...@example.com): Debug: U5ZtLH8IAVXydgNAM/SpMA: sieve: > Script binary /usr/local/var/dovecot/sieve/script2.svbin is not up-to-date > Also, it does appear that blowing away *everything* in my global script location (is removing the svbin file the key?) and re-creating it all seems to fix the problem.
Re: Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
> > > However, I do see that Sieve was accessing the user home directory > > > because for some reason now it just created a ".pki" directory > > > therein, which inside of it has an empty "nssdb" directory. That > > > never happened before...? Not a big problem, but I'd prefer not > > > to have that there. > > > > Sieve doesn't do that. I don't think Dovecot does that either, but I am > > not sure. > > Odd. Some lib Sieve uses? These directories do not appear in user > home directories unless I install the newest Sieve (and not until > a delivery via LMTP happens). No other changes. No other software > is currently accessing user home locations at all. Using today's hg version, these .pki directories aren't created. I wonder if Timo could shine a light on this.
Re: Overriding dovecot.conf from Userdb Extras
> > I thought I read that anything from dovecot.conf can be overridden in a > > userdb lookup. Or a passdb lookup with "userdb_" prefix. > > > > But I tried for fun change log_path but it never worked. Is that because > > logging is special, already started logging before it comes to the > > passdb/userdb lookups? So are there some dovecot.conf settings > > that cannot be overridden? > > > To my understanding only these extra parameters can be tweaked through the > userdb/passdb: > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields > http://wiki2.dovecot.org/UserDatabase/ExtraFields (+ mail and quota_rule) Quoting from your second link: "It's possible to override settings from dovecot.conf" (quota rules being a common example, yes). I've successfully overridden a few different dovecot.conf settings, mostly for lda and sieve, but also mail_debug from 10-logging.conf. So it does work to override *some* settings from dovecot.conf -- but there seem to be a few like log_path that are immune to overrides I guess.
quota_over_flag examples?
I can't find any posts on this list for peoples using quota_over_flag http://wiki2.dovecot.org/Quota/Configuration#Overquota-flag_.28v2.2.16.2B-.29 If my userdb is sql what would be best script to use in terms of performance? (I mean if over-quota-flag triggers script every time it changes and the script calls CLI mysql client isn't all this so expensive to spawn a new shell session which spawns a mysql client?) Anyone knows how to use this flag with postfix *making postfix send special reject* "user over quota" note instead of plain SMTP reject?? Is an additional database lookup (restriction class?) unavoidable? :( TIA1 PS Looks like it is tricky almost impossible to make postfix do rejects based on this flag for aliases. (Special query would be a little messy for our schema but i dunno at what point postfix resolves aliases?)
Thunderbird sync time? (reverse quota warning)
For reverse quota warning, it looks like webmail works nice. quota_warning3 = -storage=100%% quota-warning below %u # user is no longer over quota But using thunderbird I deleted enough messages to go below quota and I deleted them from trash folder too. But mail log does not show thunderbird connecting to imap to actually delete mails and get the trigger below quota warning msg. It's i thinks a thundrbrd behavior but i wonder if anyone on this list knows how it works? Do i just keep waiting for it to sync eventually? or??? TIA!
Re: Thunderbird sync time? (reverse quota warning)
> Try to right-click a folder and click "compact". That > should trigger an expunge. > There are additional configuration items under account > settings. You > can make TB expunge on exit, you can turn off local > cache to make it > more "live" a la webmail... > > See more: > http://kb.mozillazine.org/Deleting_messages_in_IMAP_accounts Thank you for the link
Re: quota_over_flag examples?
Thanks you so much for your reply-- > > Anyone knows how to use this flag with postfix *making postfix send > > special reject* "user over quota" note instead of plain SMTP reject?? > > Is an additional database lookup (restriction class?) unavoidable? :( > > I don't actually use this, but try perhaps: > https://sys4.de/en/blog/2013/04/08/postfix-dovecot-mailbox-quota/ > And perhaps search the mailing list for "quota-status" for more info. That's not the same. Strange but I never found quota-status docs on Dovecot wiki nowhere!Anyway, I think quota_over_flag is new and possibly Timo replacing quota-status with this flag now? It's easier I think to use this flag as a smtpd restriction in postfix and not have to do a policy lookup. However, it would be more nice on the postfix side to do only one lookup including the user lookup and the quota_over_flag but I don't know if I can do that and be able to give reject message that is particular to accounts over quota. > > PS Looks like it is tricky almost impossible to make postfix do rejects > > based on this flag for aliases. (Special query would be a little messy > > for our schema but i dunno at what point postfix resolves aliases?) > > Tough one. It gets more complicated: What about aliases expanding to > multiple recipients? Good point!! Maybe best to let aliases cause bounces like in years before. SMTP reject for real accounts only is still a improvemtn. > I figure the options are: > * Reject (or defer) the RCPT TO because of the one offender who's over quota > * Accept, and deliver only to within-quota recipients, silently drop out > the over-quota ones > * Let a bounce message go out in this case, as necessary > > I don't know how it's done with postfix, anyway...
Performance impace of spawning shell processes from Dovecot [was: quota_over_flag examples?]
> > I can't find any posts on this list for peoples using quota_over_flag > > > > http://wiki2.dovecot.org/Quota/Configuration#Overquota-flag_.28v2.2.16.2B-.29 > > > > If my userdb is sql what would be best script to use in terms of > > performance? > > (I mean if over-quota-flag triggers script every time it changes and the > > script > > calls CLI mysql client isn't all this so expensive to spawn a new shell > > session > > which spawns a mysql client?) > > I have a post-login script updating a "lastlogin" timestamp every time a > user logs in. This can happen many times per second in busy hours. The > only noticeable load is on the mysql _server_ (namely, some I/O). The > shell + mysql client load is not noticeable at all. Thank you. Is this common for most people === repeatedly spawning shell scripts from Dovecot processes is not impact performance? I thought it's why apps are written as daemons especially for many times a second as you say! > Don't use bash, of course! Hmm well I didn't not know about this. On CentOS-- lrwxrwxrwx. 1 root root 4 Apr 5 10:31 /bin/sh -> bash* Can you state the reasons you say do not use bash so I can google about them?
Sieve extprograms ?not exexuting?
Hello, I was testing the extprograms plugin. I think I had it working in the past, but many things have changed since then, so no use trying to figure out where it broke - starting over again... Debug-enabled log give me: Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: qOGyA0DePHVaOyHEM/SpMA: sieve: action execute: running program: test.sh Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: waiting for program `/usr/local/etc/dovecot/sieve_globals/test.sh' to finish after 0 seconds So I guess it thinks it is running my script? But simple test script does nothing. Here it is: #!/bin/sh read INPUT INPUT="Hello world: $INPUT" echo "$INPUT" >> /tmp/hello echo "---" >> /tmp/hello Permissions on this script file for now are rwxrwxrwx But nothing goes to /tmp/hello at all. Script works when I run it manually. I also tried without the "read" but I think that's required isn't it? Anyway, what else can I do to debug this? Dovecot 2.2.16 Pigeonhole 0.4.7
Re: Sieve extprograms ?not exexuting?
> > Debug-enabled log give me: > > Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: > > qOGyA0DePHVaOyHEM/SpMA: sieve: action execute: running > > program: test.sh > > Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: > > waiting for program `/usr/local/etc/dovecot/sieve_globals/test.sh' > > to finish after 0 seconds > > > > So I guess it thinks it is running my script? But simple test script > > does nothing. Here it is: > > > > #!/bin/sh > > read INPUT > > INPUT="Hello world: $INPUT" > > echo "$INPUT" >> /tmp/hello > > echo "---" >> /tmp/hello > > > > Permissions on this script file for now are rwxrwxrwx > > But nothing goes to /tmp/hello at all. Script works when I run it > > manually. I also tried without the "read" but I think that's required > > isn't it? Anyway, what else can I do to debug this? > > Well, first try with a script that cannot fail (well most likely), e.g.: I tried your script for fun, same result. Log showing the script was called, but no output from the script. I also deleted the script and made sure that debug log showed that sieve could not find the script. Is it chrooted or something weird? I have confirmed it is being run by calling "exit 3" and seeing in the log that "program ... terminated with non-zero exit code 3" so problem is in commands accessing the filesystem I guess. I added this: echo "HELLO WORLD" 1>&2 And log shows "Error: HELLO WORLD" So it's working but no filesystem access. Calling from sieve script with: execute :input "myinput" "test.sh"; Also tried execute "test.sh";
Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
I switched from running my extprograms execute script directly
to running with dovecot socket. Log shows only this
dovecot: lmtp(t...@example.com): Debug: wdi0Tb5VPlGfPnEAM/SpMA: sieve: action
execute: running program: test
dovecot: lmtp(t...@example.com): Debug: Namespace : Using permissions from
/vmail/example.com/test: mode=0770 gid=default
dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected
For testing I opened up the script and socket with
permissions 777 but the error seems to indicate
less about permissions more about some kind of
protocol problem i guessing.
Sieve script calls using this:
execute "test";
Plugin config:
plugin {
sieve_plugins = sieve_extprograms
sieve_global_extensions = +vnd.dovecot.execute
sieve_execute_socket_dir = sieve-execute
sieve_before = /usr/local/etc/dovecot/sieve
}
service test {
executable = script /usr/local/etc/dovecot/sieve_globals/test.sh
unix_listener sieve-execute/test {
mode = 0660
group = vmail
}
}
FYI I have quota-warning sockets configured
identical to this and they work good.
Dovecot 2.2.16
Pigeonhole 0.4.7
Help appreciate a lot.
Re: Sieve extprograms ?not exexuting?
> In another thread you said you are running CentOS. So I strongly guess > it is SELinux interfering. Check your auditd log > > grep -i AVC /var/log/audit/audit.log > > You can test whether your setup works after "setenforce 0". That sets > SELinux into permissive mode, loggging AVCs but not blocking actions. Good idea, but there are no AVC reports so I guess that's not it. In the meantime I switched to calling the script using a dovecot service and now the script isn't run at all -- see my new thread on that.
Re: quota_over_flag examples?
> > > Anyone knows how to use this flag with postfix *making postfix send > > > special reject* "user over quota" note instead of plain SMTP reject?? > > > Is an additional database lookup (restriction class?) unavoidable? :( > > > > I don't actually use this, but try perhaps: > > https://sys4.de/en/blog/2013/04/08/postfix-dovecot-mailbox-quota/ > > And perhaps search the mailing list for "quota-status" for more info. > > That's not the same. Strange but I never found quota-status docs on > Dovecot wiki nowhere!Anyway, I think quota_over_flag is new and > possibly Timo replacing quota-status with this flag now? Can anyone confirm this is true?
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> I switched from running my extprograms execute script directly
> to running with dovecot socket. Log shows only this
>
> dovecot: lmtp(t...@example.com): Debug: wdi0Tb5VPlGfPnEAM/SpMA: sieve: action
> execute: running program: test
> dovecot: lmtp(t...@example.com): Debug: Namespace : Using permissions from
> /vmail/example.com/test: mode=0770 gid=default
> dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected
>
> For testing I opened up the script and socket with
> permissions 777 but the error seems to indicate
> less about permissions more about some kind of
> protocol problem i guessing.
No one can help? Is anybody using the Sieve extprograms
execute via Dovecot socket service? I think my config
is vanilla, no? All other Dovecot and Sieve things
(including quota service scripts configured very
similarly) work fine.
Taking a look at the code, the error seems to indicate
that no input is available on the socket when Dovecot
checks. Does my script need to behave differently?
What exactly to do?
Should I just go back to direct execute? What's the
difference anyway beside the user/permissions will
be different?
Stephan? Anyone?
> Sieve script calls using this:
>
> execute "test";
>
> Plugin config:
>
> plugin {
> sieve_plugins = sieve_extprograms
> sieve_global_extensions = +vnd.dovecot.execute
> sieve_execute_socket_dir = sieve-execute
> sieve_before = /usr/local/etc/dovecot/sieve
> }
> service test {
>executable = script /usr/local/etc/dovecot/sieve_globals/test.sh
>unix_listener sieve-execute/test {
> mode = 0660
> group = vmail
>}
> }
>
> FYI I have quota-warning sockets configured
> identical to this and they work good.
>
> Dovecot 2.2.16
> Pigeonhole 0.4.7
>
> Help appreciate a lot.
Re: Sieve extprograms ?not exexuting?
>>> Debug-enabled log give me: >>> Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: >>> qOGyA0DePHVaOyHEM/SpMA: sieve: action execute: running >>> program: test.sh >>> Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: >>> waiting for program `/usr/local/etc/dovecot/sieve_globals/test.sh' >>> to finish after 0 seconds >>> >>> So I guess it thinks it is running my script? But simple test script >>> does nothing. Here it is: >>> >>> #!/bin/sh >>> read INPUT >>> INPUT="Hello world: $INPUT" >>> echo "$INPUT" >> /tmp/hello >>> echo "---" >> /tmp/hello >>> >>> Permissions on this script file for now are rwxrwxrwx >>> But nothing goes to /tmp/hello at all. Script works when I run it >>> manually. I also tried without the "read" but I think that's required >>> isn't it? Anyway, what else can I do to debug this? >> >> Well, first try with a script that cannot fail (well most likely), e.g.: > > I tried your script for fun, same result. Log showing > the script was called, but no output from the script. > > I also deleted the script and made sure that debug > log showed that sieve could not find the script. Is it > chrooted or something weird? > > I have confirmed it is being run by calling "exit 3" and > seeing in the log that "program ... terminated with > non-zero exit code 3" so problem is in commands > accessing the filesystem I guess. I added this: > > echo "HELLO WORLD" 1>&2 > > And log shows "Error: HELLO WORLD" > > So it's working but no filesystem access. I just tried this too: touch /tmp/hello-world And nothing. I tried to *read* from the filesystem: TEST=$(cat /tmp/test) echo "TEST: $TEST" 1>&2 Nothing. I found that the script can do other things like connect to network or other services, but any way I try to do something with the filesystem come up empty. Dont' know if calling the script via dovecot socket service would make this different because there is a bug that prevents extprograms execute via socket broken (see other thread). Is this on purpose no filesystem access allowed? why? > Calling from sieve script with: > > execute :input "myinput" "test.sh"; > > Also tried > > execute "test.sh";
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> Will look at this later this week. Thank you.
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> >> I switched from running my extprograms execute script directly > >> to running with dovecot socket. Log shows only this > >> > >> dovecot: lmtp(t...@example.com): Debug: wdi0Tb5VPlGfPnEAM/SpMA: sieve: > >> action execute: running program: test > >> dovecot: lmtp(t...@example.com): Debug: Namespace : Using permissions from > >> /vmail/example.com/test: mode=0770 gid=default > >> dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > >> > >> For testing I opened up the script and socket with > >> permissions 777 but the error seems to indicate > >> less about permissions more about some kind of > >> protocol problem i guessing. > > No one can help? Is anybody using the Sieve extprograms > > execute via Dovecot socket service? I think my config > > is vanilla, no? All other Dovecot and Sieve things > > (including quota service scripts configured very > > similarly) work fine. > > > > Taking a look at the code, the error seems to indicate > > that no input is available on the socket when Dovecot > > checks. Does my script need to behave differently? > > What exactly to do? > > > > Should I just go back to direct execute? What's the > > difference anyway beside the user/permissions will > > be different? > > > > Stephan? Anyone? > > Last two changes should fix this: Not yet -- this may be unrelated(?) but here is what I have after installing the newest source package: Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: mail_deliver_ctx_get_log_var_expand_table Help?
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> > Last two changes should fix this: > > > Not yet -- this may be unrelated(?) but here is what I have > after installing the newest source package: > > Error: Couldn't load required plugin > /usr/local/lib/dovecot/lib90_sieve_plugin.so: > dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined > symbol: > mail_deliver_ctx_get_log_var_expand_table Oh i guessing this requires the newest Dovecot source too (maybe you should put notice on the wiki that extprograms run-via-socket is broken until a new release?) After installed newest Dovecot and Sieve, I getting this: dovecot: lmtp(t...@example.com): Debug: auth input: dovecot: lmtp(te...@example.com): Fatal: master: service(lmtp): child 7033 killed with signal 11 (core dumps disabled) With set mail_debug=yes doesn't give anything else of interesting for this.
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> > > Last two changes should fix this: > > > > > > Not yet -- this may be unrelated(?) but here is what I have > > after installing the newest source package: > > > > Error: Couldn't load required plugin > > /usr/local/lib/dovecot/lib90_sieve_plugin.so: > > dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined > > symbol: > > mail_deliver_ctx_get_log_var_expand_table > > Oh i guessing this requires the newest Dovecot source too (maybe you > should put notice on the wiki that extprograms run-via-socket is broken until > a new release?) > > After installed newest Dovecot and Sieve, I getting this: > > dovecot: lmtp(t...@example.com): Debug: auth input: > dovecot: lmtp( > te...@example.com): Fatal: master: service(lmtp): child > 7033 killed with signal 11 (core dumps disabled) > > With set mail_debug=yes doesn't give anything else of interesting for this. Sorrys. I decided to "make clean" and do new "./configure" "make" and "make install" for dovecot first, then pigeonhole to make sure nothing is off. After I do that I get same original error: dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected I went to go verifying in dovecot-pigeonhole that the two new patches are there by manual inspect the source files you patched. I found only the first one, not http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this because I downloaded a snap shot link at the top that wasn't built with the 2nd change? I put 2nd change in manually (its only one liner) rebuilded and after install. same error with recv(MSG_PEEK) Thanks for ongoing help
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> Last two changes should fix this: > >>> > >>> Not yet -- this may be unrelated(?) but here is what I have > >>> after installing the newest source package: > >>> > >>> Error: Couldn't load required plugin > >>> /usr/local/lib/dovecot/lib90_sieve_plugin.so: > >>> dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined > >>> symbol: > >>> mail_deliver_ctx_get_log_var_expand_table > >> Oh i guessing this requires the newest Dovecot source too (maybe you > >> should put notice on the wiki that extprograms run-via-socket is broken > >> until > >> a new release?) > >> > >> After installed newest Dovecot and Sieve, I getting this: > >> > >> dovecot: lmtp(t...@example.com): Debug: auth input: > >> dovecot: lmtp( > >> te...@example.com): Fatal: master: service(lmtp): child > >> 7033 killed with signal 11 (core dumps disabled) > >> > >> With set mail_debug=yes doesn't give anything else of interesting for this. > > Sorrys. I decided to "make clean" and do new "./configure" "make" and "make > > install" for dovecot first, then pigeonhole to make sure nothing is off. > > After > > I do that I get same original error: > > > > dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > > > > I went to go verifying in dovecot-pigeonhole that the two new patches are > > there > > by manual inspect the source files you patched. I found only the first one, > > not > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this > > because I downloaded a snap shot link at the top that wasn't built with the > > 2nd change? > > > > I put 2nd change in manually (its only one liner) rebuilded and after > > install. > > same error with recv(MSG_PEEK) > > Did you restart Dovecot? Yes. One more restart now to make sure. > Also, can you try running the script through `sieve-test -D -t - > -Tlevel=matching` (see man page). I cannot make this working. I'm logg in as unix user "adm" and the sieve-test thinks that is the user it's delivering to?? I tried to tell the recipient with both -a and -r and I put hard-coded -l mail location but output still showing it wants to deliver to mailbox for "adm" user. Also mail file has To: header with the correct recipient but doesn't help. Users are virtual I cant logg in as the correct recipient to run sieve-test. What do I do?
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> > I went to go verifying in dovecot-pigeonhole that the two new patches are
> > there
> > by manual inspect the source files you patched. I found only the first one,
> > not
> > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this
> > because I downloaded a snap shot link at the top that wasn't built with the
> > 2nd change?
> >
> > I put 2nd change in manually (its only one liner) rebuilded and after
> > install.
> > same error with recv(MSG_PEEK)
> >
> > Thanks for ongoing help
>
> Argh! Ok, that fix was a bit incomplete still:
>
> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/89e0cef5b264
Sorry I over-looking this. I just tested with this new fix. IT WORKS.
> I wonder why my test didn't fail.
>
> The extent of this bug is limited to using execute as a command (as
> opposed to using it as a test in an `if' statement) and not providing it
> with any input.
That's not matching to my environment. My script is calling execute
within a if block but not as a test but it IS providing input (but not get
any output if that matters)
if ...
{
execute :input "test input" "test";
}
Only thing still unsolved is my other thread don't know why the exectued
script has no filesystem access? (like "touch /tmp/test" ignored no error)
Also is the only different between direct and socket execute that with
socket I can run the script with more restricted owner and permissions?
Are there other differences?
Re: Sieve extprograms ?not exexuting?
> Hello, I was testing the extprograms plugin. I think I had it working > in the past, but many things have changed since then, so no use > trying to figure out where it broke - starting over again... > > Debug-enabled log give me: > Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: > qOGyA0DePHVaOyHEM/SpMA: sieve: action execute: running > program: test.sh > Apr 27 04:11:36 mail dovecot: lmtp(t...@example.com): Debug: > waiting for program `/usr/local/etc/dovecot/sieve_globals/test.sh' > to finish after 0 seconds > > So I guess it thinks it is running my script? But simple test script > does nothing. Here it is: > > #!/bin/sh > read INPUT > INPUT="Hello world: $INPUT" > echo "$INPUT" >> /tmp/hello > echo "---" >> /tmp/hello > > Permissions on this script file for now are rwxrwxrwx > But nothing goes to /tmp/hello at all. Script works when I run it > manually. I also tried without the "read" but I think that's required > isn't it? Anyway, what else can I do to debug this? Turns out this is a problem with systemd. I have PrivateTmp=true in the dovecot.service file so anything written to /tmp goes to lala land (is it anywhere I can see outside of the dovecot process?). Problem solved.
Re: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected
> Only thing still unsolved is my other thread don't know why the exectued > script has no filesystem access? (like "touch /tmp/test" ignored no error) See my other post. Problem is systemd PrivateTmp=true hides anything you do with /tmp from view by anyone else. Wondering if its in memory or stashed away somewhere i can see it on the CLI
[Dovecot] LDA without lookup as non-root?
Hello, I'm having some problems getting LDA to work without userdb lookups and have a few related questions. This system has all users in MySQL, each user with unique UID/GID, no local users at all. Installation is from apt-get. 1) If LDA is invoked without lookups, is it correct to assume that the "service auth" and "service auth-worker" can be completely removed from dovecot master configuration? (I have tried commenting them out and logging into IMAP, which seems to work, not sure if anyone else needs the auth service) 2) If LDA is invoked without lookups, will I be unable to use Dovecot quota plugin? Does it need to have a user lookup to get quota info? (haven't added quota support, need to take this one step at a time) 3) The interesting part -- I am invoking LDA from Maildrop. See: http://thread.gmane.org/gmane.mail.imap.dovecot/65473 So when invoked, Maildrop has already dropped to the destination UID/GID and the needed paths are available in the environment. However, using as many permutations of calling LDA as I can think of (based on http://wiki2.dovecot.org/LDA ), I always get this: (command line usage error. Command output: lda: Fatal: Couldn't lookup our username (uid=2500) ) The UID is correct for the target user. If I add "-d $LOGNAME" to my LDA callout, I get permission denied on the userdb lookup, which I guess is another issue to work out if I want to go with lookups. But right now I am trying not to. Why does LDA seem to try for a lookup even when I follow the wiki instructions how to call it without a lookup? 3.5) Related question, my users have separate homedir and maildir, both paths are looked up by Maildrop. I think I need to call LDA with "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct?
Re: [Dovecot] LDA without lookup as non-root?
> 1) If LDA is invoked without > lookups, is it correct to assume that the "service auth" and > "service > auth-worker" can be completely removed from dovecot master > configuration? (I have tried commenting them out and logging into IMAP, > which seems to work, not sure if anyone else needs the auth service) Any confirmation on this? > 2) > If LDA is invoked without lookups, will I be unable to use Dovecot > quota plugin? Does it need to have a user lookup to get quota info? > (haven't added quota support, need to take this one step at a time) I'm especially interested if someone can comment on this, since maybe it makes my efforts here wasted > 3) The interesting part -- I am invoking LDA from Maildrop. See: > http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So > when invoked, Maildrop has already dropped to the destination UID/GID > and the needed paths are available in the environment. However, using > as many permutations of calling LDA as I can think of (based on > http://wiki2.dovecot.org/LDA ), I always get this: > > (command line usage error. Command output: lda: Fatal: Couldn't lookup our > username (uid=2500) ) I could not find anything in the mailing list archives to help me, but I googled and found a link to a source file: http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 I read the source, it looks like it's not exactly a userdb lookup - LDA is trying to get the unix username for the given UID. In my case, UIDs are "virtual" so there isn't a unix username. The source doesn't really use the username that it looks up except in a call "open_logfile." Is it possible to avoid this problem? It looks like the answer is no, I have to use -d which also forces a userdb lookup. Maybe this limitation can be removed in the future? Now I suppose I have to go understand the problems of userdb lookup permissions, but I think there are solutions for that. Am I on the right understanding ? > The > UID is correct for the target user. If I add "-d $LOGNAME" to my LDA > callout, I get permission denied on the userdb lookup, which I guess is > another issue to work out if I want to go with lookups. But right now I > am trying not to. Why does LDA seem to try for a lookup even when I > follow the wiki instructions how to call it without a lookup? > > 3.5) > Related question, my users have separate homedir and maildir, both > paths are looked up by Maildrop. I think I need to call LDA with > "HOME=$DEFAULT dovecot-lda -f $FROM". Is this correct? >
Re: [Dovecot] LDA without lookup as non-root?
>> 3) The interesting part -- I am invoking LDA from Maildrop. See: >> http://thread.gmane.org/gmane.mail.imap.dovecot/65473 >> So >> when invoked, Maildrop has already dropped to the destination UID/GID >> and the needed paths are available in the environment. However, using >> as many permutations of calling LDA as I can think of (based on >> http://wiki2.dovecot.org/LDA ), I always get this: >> >> (command line usage error. Command output: lda: Fatal: Couldn't lookup > our >> username (uid=2500) ) > > I could not find anything in the mailing list archives to help me, but I > googled > and found a link to a source file: > > http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96 > > I read the source, it looks like it's not exactly a userdb lookup - LDA is > trying to get the unix username for the given UID. In my case, UIDs are > "virtual" so there isn't a unix username. The source doesn't > really use the username that it looks up except in a call > "open_logfile." > > Is it possible to avoid this problem? It looks like the answer is no, I have > to > use -d which also forces a userdb lookup. Maybe this limitation can be > removed > in the future? Now I suppose I have to go understand the problems of userdb > lookup permissions, but I think there are solutions for that. FWIW, in this scenario, "service auth" in master config has to have its mode relaxed to 0606 to make userdb lookups work. So ANYONE on the machine can see all userdb lookups. I don't have local users here, so it's probably safe anyway(?). Can anyone explain if there are other security risks of running the auth service at 0606?
Re: [Dovecot] LDA without lookup as non-root?
Timo, Sorry I didn't see your response until now >> 3) The interesting part -- I am invoking LDA from Maildrop. See: >> http://thread.gmane.org/gmane.mail.imap.dovecot/65473 > So >> when invoked, Maildrop has already dropped to the destination UID/GID >> and the needed paths are available in the environment. However, using >> as many permutations of calling LDA as I can think of (based on >> http://wiki2.dovecot.org/LDA ), I always get this: >> >> (command line usage error. Command output: lda: Fatal: Couldn't lookup >> our username (uid=2500) ) > > Set USER environment. Sorry, would you mind being more specific? If you see my follow-up posts on this thread, I found a source file with this error message in it (link below) and reading that code, there is no way to avoid this error for non-system users (uid's) if you don't use -d. (looking at the "destination" variable) http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/7d85833eff96
