Re: [Dovecot] mail spool filesystem

[Daniel L. Miller]

On 8/19/2011 10:49 AM, Seth Mattinen wrote:

On 8/17/11 7:42 AM, Adrian Ulrich wrote:

I read that XFS is a good choice, but is not
too reliable...

Are you using Maildir or MBOX?

In any case: XFS would be my last choice:

XFS is nice if you are working with large files (>  2GB), but
for E-Mail i'd stick with ext3 (or maybe even reiser3)
as it works very well with small files.


I'd have to disagree. This is completely anecdotal, but I originally
deployed ext3 on all of my mail servers (Dovecot maildir) and spools
(Postfix) until they started exhibiting loading issues when busy.
Reformatting into XFS resolved the problem with no other changes. I
didn't have time to do any comparisons or gather statistics since it was
an emergency situation and this was before ext4, but XFS has performed
flawlessly for me.

~Seth


+1. :)

--
Daniel

[Dovecot] OT - small hd recommendation

[Daniel L. Miller]

A little OT - but I've seen a few opinions voiced here by various admins 
and I'd like to benefit.


Currently running a single combined server for multiple operations - 
fileserver, mailserver, webserver, virtual server, and whatever else 
pops up.  Current incarnation of the machine, after the last rebuild, is 
an AMD Opteron 4180 with a Supermicro MB using ATI SB700 chipset - which 
means it supports SATA 3.0.


Current storage is one o/s drive, and a 4-drive RAID10 array.  The 
RAID10 is using the Linux softraid via mdadm.  The drives are Seagates - 
ST3160811AS.  So it's 320GB of storage, and SATA 1.5.


I'm seeing some warnings & errors in my logs & dmesg - and Google tells 
me this can the result of several factors, including the hard drives.  I 
haven't seen any SMART warnings as yet - but I am getting a little 
nervous and thinking about upgrading the storage.


At this time, I'm just interested in recommendations for hard drives & 
partitioning strategy.  At the time I created the RAID10 array - I was 
still in the early stages of learning these technologies (not that I 
know that much more now!) so I probably didn't take advantage of any of 
the advanced settings such as stripe size, cluster size, etc.  I am 
using XFS.


As part of the potential upgrade, I'm considering changing to RAID6 - 
seems a bit more efficient use of space.  I see no reason for SSD - I 
think a set of reasonable 7200rpm drives should be just fine.  What I 
don't know is, compared to my current 4 drive RAID10 with SATA 1.5, 
would even a single SATA 3 drive be comparable in terms of seek 
performance?  Should I stick with the RAID-10?

--
Daniel

Re: [Dovecot] OT - small hd recommendation

[Daniel L. Miller]

On 8/31/2011 12:41 PM, Thomas Harold wrote:

On 8/30/2011 5:43 PM, Daniel L. Miller wrote:

A little OT - but I've seen a few opinions voiced here by various admins
and I'd like to benefit.


RAID-10 is fine (note that the default mdadm RAID10 isn't actually 
RAID10, but it works well enough).  RAID-6 won't be faster (and will 
probably be worse) although RAID-6 does do a bit better in a 
double-drive failure over RAID-10.  The only way to get more 
performance out of (4) drives is to switch to 10k or 15k SAS (or SSDs).


[...]

(Take a look at /dev/disk/by-id, /dev/disk/by-uuid, etc. Export a copy 
of that information on a daily/weekly basis off of the machine.  In a 
software RAID environment, it gives you better information about which 
drive serial # failed rather then relying on lights.)


Our mail server is 3-way RAID1 (triple mirror) for the OS and mail 
queue with a 5-disk RAID-10 (4+spare) for mail storage.


Given my extensive requirements - I haven't yet filled my existing 320GB 
- size isn't a big deal.  Am I actually deriving much benefit from 
4-disk RAID10 using 160GB discs - vs a 2-4 disc 1TB RAID1 array?


--
Daniel

[Dovecot] Group ACL

[Daniel L. Miller]

Using Dovecot 2.0, LDAP userdb & passdb, with prefetch:

hosts = myhost.mydomain
dn = cn=x,cn=y
dnpass = xyz123
auth_bind = yes
auth_bind_userdn = uid=%n,ou=users,dc=x
ldap_version = 3
base = ou=Users, dc=x
user_attrs = =home=/var/mail/%d/%n, \
=mail=mdbox:/var/mail/%d/%n/mdbox, \
=uid=vmail, \
=gid=mail

user_filter = (&(objectClass=inetOrgPerson)(mail=%u))

pass_attrs = mail=user, \
userPassword=password, \
=userdb_home=/var/mail/%d/%n, \
=userdb_mail=mdbox:/var/mail/%d/%n/mdbox, \
=userdb_uid=vmail, \
=userdb_gid=mail, \
=userdb_acl_groups='allshared'

pass_filter = (&(objectClass=inetOrgPerson)(mail=%u))

iterate_attrs = mail=user
iterate_filter = (&(objectClass=inetOrgPerson)(mail=*))

default_pass_scheme = plain

I have added permissions for "$allshared" to a mailbox's Inbox.  It does 
not appear in the shared folders list.  Other mailboxes, with explicit 
permission for a given username, work fine.  What am I doing wrong?


--
Daniel

Re: [Dovecot] Group ACL

[Daniel L. Miller]

On 10/6/2011 8:38 AM, Daniel L. Miller wrote:

Using Dovecot 2.0, LDAP userdb & passdb, with prefetch:

[..]

I have added permissions for "$allshared" to a mailbox's Inbox.  It 
does not appear in the shared folders list.  Other mailboxes, with 
explicit permission for a given username, work fine.  What am I doing 
wrong?



Am I, as usual, doing something wrong?

--
Daniel

Re: [Dovecot] Group ACL

[Daniel L. Miller]

On 10/7/2011 11:08 PM, Daniel L. Miller wrote:

On 10/6/2011 8:38 AM, Daniel L. Miller wrote:

Using Dovecot 2.0, LDAP userdb & passdb, with prefetch:

[..]

I have added permissions for "$allshared" to a mailbox's Inbox.  It 
does not appear in the shared folders list.  Other mailboxes, with 
explicit permission for a given username, work fine.  What am I doing 
wrong?



Am I, as usual, doing something wrong?


Is there anything else I should supply to help diagnose the issue?

--
Daniel

[Dovecot] doveadm index

[Daniel L. Miller]

Does "doveadm index -A INBOX" reindex everything?  What about users with 
multiple folders, both subbed & not subbed under INBOX?


--
Daniel

Re: [Dovecot] doveadm index

[Daniel L. Miller]

On 12/20/2011 2:18 AM, Timo Sirainen wrote:

On Wed, 2011-12-14 at 19:27 -0800, Daniel L. Miller wrote:

Does "doveadm index -A INBOX" reindex everything?  What about users with
multiple folders, both subbed&  not subbed under INBOX?

It indexes INBOX (only) for all users. And it doesn't "reindex"
anything, it just adds any missing stuff.


Ok - IS there a command that will ... fully update ... all folders for 
all users?  Will something like "doveadm search -A text zyxabcxyz > 
/dev/null" work?


--
Daniel

[Dovecot] Lucene vs Solr with 2.1 single server

[Daniel L. Miller]

For a single server, is the Lucene backend now "better" than running a 
Solr server?


--
Daniel

[Dovecot] fts_solr break-imap-search 2.1

[Daniel L. Miller]

Is "break-imap-search" still required using 2.1 with Solr to index TEXT 
& BODY searches?


--
Daniel

[Dovecot] Possible mdbox corruption

[Daniel L. Miller]

I thought I had cleared out the corruption I had before - perhaps I was 
mistaken.  What steps should I take to help locate these issues?  
Currently using 2.1rc1.  I see the following errors in my logs, 
including out of memory and message size issues (at 15:30):


Jan  4 05:17:17 bubba dovecot: indexer-worker(us...@domain.com): Error: 
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a) 
[0x7f6e17cbfd0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56) 
[0x7f6e17cbfd56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7f6e17c98d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310) 
[0x7f6e17cd0310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965) 
[0x7f6e17cbc965] -> 
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c) 
[0x7f6e17cbd0ec] -> 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292) 
[0x7f6e164b7292] -> 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97) 
[0x7f6e164b7a97] -> 
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c) 
[0x7f6e166c4abc] -> 
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1) 
[0x7f6e166c5561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630) 
[0x7f6e166ca630] -> dovecot/indexer-worker [us...@domain.com Sent - 
5500/7510]() [0x40245f] -> dovecot/indexer-worker [us...@domain.com Sent 
- 5500/7510]() [0x4027dd] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f6e17ccc0f6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f6e17ccd17f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f6e17ccc098] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f6e17cb9123] -> dovecot/indexer-worker [us...@domain.com Sent - 
5500/7510](main+0x109) [0x401f29] -> 
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f6e1791cd8e] -> 
dovecot/indexer-worker [us...@domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 05:17:17 bubba dovecot: indexer: Error: Indexer worker 
disconnected, discarding 1 requests for us...@domain.com
Jan  4 05:17:17 bubba dovecot: master: Error: service(indexer-worker): 
child 10896 returned error 83 (Out of memory (vsz_limit=256 MB, you may 
need to increase it))
Jan  4 06:17:17 bubba dovecot: indexer-worker(us...@domain.com): Fatal: 
pool_system_realloc(134217728): Out of memory
Jan  4 06:17:17 bubba dovecot: indexer-worker(us...@domain.com): Error: 
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a) 
[0x7f0ec1a57d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56) 
[0x7f0ec1a57d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7f0ec1a30d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310) 
[0x7f0ec1a68310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965) 
[0x7f0ec1a54965] -> 
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c) 
[0x7f0ec1a550ec] -> 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292) 
[0x7f0ec024f292] -> 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97) 
[0x7f0ec024fa97] -> 
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c) 
[0x7f0ec045cabc] -> 
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1) 
[0x7f0ec045d561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630) 
[0x7f0ec0462630] -> dovecot/indexer-worker [us...@domain.com Sent - 
5500/7510]() [0x40245f] -> dovecot/indexer-worker [us...@domain.com Sent 
- 5500/7510]() [0x4027dd] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f0ec1a640f6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f0ec1a6517f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f0ec1a64098] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f0ec1a51123] -> dovecot/indexer-worker [us...@domain.com Sent - 
5500/7510](main+0x109) [0x401f29] -> 
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f0ec16b4d8e] -> 
dovecot/indexer-worker [us...@domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 06:17:17 bubba dovecot: indexer: Error: Indexer worker 
disconnected, discarding 1 requests for us...@domain.com
Jan  4 06:17:17 bubba dovecot: master: Error: service(indexer-worker): 
child 11941 returned error 83 (Out of memory (vsz_limit=256 MB, you may 
need to increase it))
Jan  4 07:17:18 bubba dovecot: indexer-worker(us...@domain.com): Fatal: 
pool_system_realloc(134217728): Out of memory
Jan  4 07:17:18 bubba dovecot: indexer-worker(us...@domain.com): Error: 
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a) 
[0x7faed4e56d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56) 
[0x7faed4e56d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7faed4e2fd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310) 
[0x7faed4e67310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965) 
[0x7faed4e53965] -> 
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c) 
[0x7faed4e540ec] -> 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292) 
[0x7faed364e292] -> 
/usr/local/lib/dovecot/lib2

Re: [Dovecot] Possible mdbox corruption

[Daniel L. Miller]

On 1/6/2012 2:57 AM, Timo Sirainen wrote:

On 6.1.2012, at 12.55, Timo Sirainen wrote:


Jan  4 05:17:17 bubba dovecot: master: Error: service(indexer-worker): child 
10896 returned error 83 (Out of memory (vsz_limit=256 MB, you may need to 
increase it))
Jan  4 06:17:17 bubba dovecot: indexer-worker(us...@domain.com): Fatal: 
pool_system_realloc(134217728): Out of memory

The problem is clearly that index-worker's vsz_limit is too low. Increase it 
(or default_vsz_limit).

Although the source of the out-of-memory

/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c) [0x7f0ec1a550ec] ->  
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292) [0x7f0ec024f292] ->

is something that shouldn't really be happening. I guess the Solr plugin wastes 
memory unnecessarily, I'll see what I can do about it. But for now just 
increase vsz limit.

I set default_vsz_limit = 1024M.  Those errors appear gone - but I do 
have messages like:


Jan  6 09:22:42 bubba dovecot: indexer-worker(us...@domain.com): Error: 
fts_solr: Indexing failed: 400 Illegal character ((CTRL-CHAR, code 18))  
at [row,col {unknown-source}]: [482765,16]

Jan  6 09:22:42 bubba dovecot: indexer-worker: Error:

Google seems to indicate that Solr cannot handle "invalid" characters - 
and that it is the responsibility of the calling program to strip out 
such.  A quick search shows me a both an individual character comparison 
in Java and a regex used for the purpose.  Is there any "illegal 
character protection" in the Dovecot Solr plugin?


--
Daniel

[Dovecot] FTS-Solr plugin

[Daniel L. Miller]

Solr plugin appears to break when mailbox names have an ampersand in the 
name.  The messages appear to indicate '&' gets translated to '&--'.

--
Daniel

Re: [Dovecot] Possible mdbox corruption

[Daniel L. Miller]

On 1/6/2012 9:36 AM, Timo Sirainen wrote:

On 6.1.2012, at 19.30, Daniel L. Miller wrote:


Jan  6 09:22:42 bubba dovecot: indexer-worker(us...@domain.com): Error: 
fts_solr: Indexing failed: 400 Illegal character ((CTRL-CHAR, code 18))  at 
[row,col {unknown-source}]: [482765,16]
Jan  6 09:22:42 bubba dovecot: indexer-worker: Error:

Google seems to indicate that Solr cannot handle "invalid" characters - and that it is 
the responsibility of the calling program to strip out such.  A quick search shows me a both an 
individual character comparison in Java and a regex used for the purpose.  Is there any 
"illegal character protection" in the Dovecot Solr plugin?

Yes, there is. So I'm not really sure what it's complaining about. Are you using the 
"solr" or "solr_old" backend?



"Solr".

plugin {
  fts = solr
  fts_solr = url=http://localhost:8983/solr/
}

--
Daniel

Re: [Dovecot] Possible mdbox corruption

[Daniel L. Miller]

On 1/6/2012 12:32 PM, Daniel L. Miller wrote:

On 1/6/2012 9:36 AM, Timo Sirainen wrote:

On 6.1.2012, at 19.30, Daniel L. Miller wrote:

Jan  6 09:22:42 bubba dovecot: indexer-worker(us...@domain.com): 
Error: fts_solr: Indexing failed: 400 Illegal character ((CTRL-CHAR, 
code 18))  at [row,col {unknown-source}]: [482765,16]

Jan  6 09:22:42 bubba dovecot: indexer-worker: Error:

Google seems to indicate that Solr cannot handle "invalid" 
characters - and that it is the responsibility of the calling 
program to strip out such.  A quick search shows me a both an 
individual character comparison in Java and a regex used for the 
purpose.  Is there any "illegal character protection" in the Dovecot 
Solr plugin?
Yes, there is. So I'm not really sure what it's complaining about. 
Are you using the "solr" or "solr_old" backend?




"Solr".

plugin {
  fts = solr
  fts_solr = url=http://localhost:8983/solr/
}


Now seeing:

Jan  8 15:40:09 bubba dovecot: imap(us...@domain.com): Error: fts_solr: 
Lookup failed: 400 undefined field CC

Jan  8 15:40:09 bubba dovecot: imap: Error:


--
Daniel

[Dovecot] Solr plugin

[Daniel L. Miller]

On 1/8/2012 3:40 PM, Daniel L. Miller wrote:

On 1/6/2012 12:32 PM, Daniel L. Miller wrote:

On 1/6/2012 9:36 AM, Timo Sirainen wrote:

On 6.1.2012, at 19.30, Daniel L. Miller wrote:



Jan  8 15:40:09 bubba dovecot: imap(us...@domain.com): Error: 
fts_solr: Lookup failed: 400 undefined field CC

Jan  8 15:40:09 bubba dovecot: imap: Error:




Looking at the Solr output - looks like the CC parameter is being 
capitalized while all the other fieldnames are lowercase.


--
Daniel

Re: [Dovecot] Solr plugin

[Daniel L. Miller]

On 1/9/2012 7:00 AM, Timo Sirainen wrote:

On 9.1.2012, at 1.48, Daniel L. Miller wrote:


On 1/8/2012 3:40 PM, Daniel L. Miller wrote:

On 1/6/2012 12:32 PM, Daniel L. Miller wrote:

On 1/6/2012 9:36 AM, Timo Sirainen wrote:

On 6.1.2012, at 19.30, Daniel L. Miller wrote:


Jan  8 15:40:09 bubba dovecot: imap(us...@domain.com): Error: fts_solr: Lookup 
failed: 400 undefined field CC
Jan  8 15:40:09 bubba dovecot: imap: Error:



Looking at the Solr output - looks like the CC parameter is being capitalized 
while all the other fieldnames are lowercase.

Did you look at the input? Looking at the code, it should be lowercased. Maybe Solr just 
uppercases it for some reason. Are you using a Solr schema that has "cc" field?



I see the following in a running Solr instance.  This is generated from 
a Windoze Thunderbird 8.0 client:


Jan 9, 2012 4:20:13 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select 
params={fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a3bc41c5f+%2Buser:"dmil...@amfes.com"&q=from:"test"+OR+to:"test"+OR+CC:"test"+OR+subject:"test"+OR+body:"test"&rows=9038} 
status=400 QTime=4



That's where I see the uppercased CC.
--
Daniel

Re: [Dovecot] Imap Quota Exceeded - But Still Receiving Emails?

[Daniel L. Miller]

On 1/24/2012 8:35 AM, Palatine Web Support wrote:


Here is my dovecot config:

plugin {
  quota = maildir:User Quota
  quota_rule2 = Trash:storage=+100M
}

[..]


So it picks up my set quota of 3MB but dovecot is not rejecting emails 
if I am over my quota.


Can anyone help?


Is the quota plugin being loaded?  What is the output of:

doveconf | grep -B 2 plug

--
Daniel

Re: [Dovecot] Imap Quota Exceeded - But Still Receiving Emails?

[Daniel L. Miller]

On 1/25/2012 1:39 AM, Palatine Web Support wrote:

On 2012-01-25 05:38, Daniel L. Miller wrote:

On 1/24/2012 8:35 AM, Palatine Web Support wrote:


Here is my dovecot config:

plugin {
  quota = maildir:User Quota
  quota_rule2 = Trash:storage=+100M
}

[..]


So it picks up my set quota of 3MB but dovecot is not rejecting 
emails if I am over my quota.


Can anyone help?


Is the quota plugin being loaded?  What is the output of:

doveconf | grep -B 2 plug


Hi Daniel

I tried the command and it returned the command was not found. I have 
installed:


apt-get install dovecot-common
apt-get install dovecot-dev
apt-get install dovecot-imapd


Which package does the binary doveconf come from?


You need to make sure to reply to the list - not just to me.

If you don't have doveconf...what version of Dovecot are you using?
--
Daniel

Re: [Dovecot] Imap Quota Exceeded - But Still Receiving Emails?

[Daniel L. Miller]

On 1/25/2012 2:01 AM, Palatine Web Support wrote:

On 2012-01-25 05:38, Daniel L. Miller wrote:

On 1/24/2012 8:35 AM, Palatine Web Support wrote:


Here is my dovecot config:

plugin {
  quota = maildir:User Quota
  quota_rule2 = Trash:storage=+100M
}

[..]


So it picks up my set quota of 3MB but dovecot is not rejecting 
emails if I am over my quota.


Can anyone help?


Is the quota plugin being loaded?  What is the output of:

doveconf | grep -B 2 plug


The modules are being loaded. From the log file with debugging enabled:

Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Loading modules from 
directory: /usr/lib/dovecot/modules/imap
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Module loaded: 
/usr/lib/dovecot/modules/imap/lib10_quota_plugin.so
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Module loaded: 
/usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Effective uid=150, 
gid=8, home=/var/vmail/xxx.com/support
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Quota root: name=User 
Quota backend=dirsize args=
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Quota rule: root=User 
Quota mailbox=* bytes=3145728 messages=0
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Quota rule: root=User 
Quota mailbox=Trash bytes=104857600 messages=0
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): maildir: 
data=/var/vmail/xxx.com/support
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): maildir++: 
root=/var/vmail/xxx.com/support, index=, control=, 
inbox=/var/vmail/xxx.com/support
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Namespace : Using 
permissions from /var/vmail/xxx.com/support: mode=0700 gid=-1
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Disconnected: Logged 
out bytes=82/573
Jan 25 09:59:58 mail dovecot: IMAP(x...@xxx.com): Disconnected: Logged 
out bytes=269/8243




I don't know if it makes any difference, but in your config file, try 
changing:

plugin {
  quota = maildir:User Quota

to

plugin {
  quota = maildir:User quota

(lowercase the "quota")

--
Daniel

[Dovecot] Crash on mail folder delete

[Daniel L. Miller]

Attempting to delete a folder from within the trash folder using 
Thunderbird.  I see the following in the log:


Jan 25 15:36:22 bubba dovecot: imap(dmil...@amfes.com): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
Jan 25 15:36:22 bubba dovecot: imap(dmil...@amfes.com): Error: Raw 
backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3efba) 
[0x7f5fe9f86fba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f006) 
[0x7f5fe9f87006] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x17f5a) 
[0x7f5fe9f5ff5a] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47287) 
[0x7f5fea214287] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) 
[0x7f5fe8b9cc71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) 
[0x7f5fe8b9cd47] -> 
/usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) 
[0x7f5fe8ba061e] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) 
[0x7f5fea2085b2] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) 
[0x7f5fea2073d2] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](cmd_delete+0x72) [0x409922] -> dovecot/imap [dmil...@amfes.com 
192.168.0.91 delete](command_exec+0x3d) [0x4109ad] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40f97e] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40fa5d] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete](client_handle_input+0x135) 
[0x40fc85] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](client_input+0x5f) [0x4105af] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f5fe9f93406] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f5fe9f9448f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f5fe9f933a8] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f5fe9f803b3] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](main+0x301) [0x418a61] -> /lib/libc.so.6(__libc_start_main+0xfe) 
[0x7f5fe9be3d8e] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete]() [0x4083f9]
Jan 25 15:36:23 bubba dovecot: imap(dmil...@amfes.com): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
Jan 25 15:36:23 bubba dovecot: imap(dmil...@amfes.com): Error: Raw 
backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3efba) 
[0x7f33673dafba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f006) 
[0x7f33673db006] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x17f5a) 
[0x7f33673b3f5a] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47287) 
[0x7f3367668287] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) 
[0x7f3365ff0c71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) 
[0x7f3365ff0d47] -> 
/usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) 
[0x7f3365ff461e] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) 
[0x7f336765c5b2] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) 
[0x7f336765b3d2] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](cmd_delete+0x72) [0x409922] -> dovecot/imap [dmil...@amfes.com 
192.168.0.91 delete](command_exec+0x3d) [0x4109ad] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40f97e] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40fa5d] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete](client_handle_input+0x135) 
[0x40fc85] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](client_input+0x5f) [0x4105af] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f33673e7406] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f33673e848f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f33673e73a8] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f33673d43b3] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](main+0x301) [0x418a61] -> /lib/libc.so.6(__libc_start_main+0xfe) 
[0x7f3367037d8e] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete]() [0x4083f9]
Jan 25 15:36:23 bubba dovecot: imap(dmil...@amfes.com): Fatal: master: 
service(imap): child 6074 killed with signal 6 (core dumps disabled)
Jan 25 15:36:23 bubba dovecot: imap(dmil...@amfes.com): Fatal: master: 
service(imap): child 6589 killed with signal 6 (core dumps disabled)


--
Daniel

Re: [Dovecot] Crash on mail folder delete

[Daniel L. Miller]

On 1/25/2012 3:42 PM, Timo Sirainen wrote:

On 26.1.2012, at 1.37, Daniel L. Miller wrote:


Attempting to delete a folder from within the trash folder using Thunderbird.  
I see the following in the log:

Dovecot version?


2.1.rc3.  I'm compiling rc5 now...

--
Daniel

Re: [Dovecot] Crash on mail folder delete

[Daniel L. Miller]

On 1/25/2012 3:43 PM, Daniel L. Miller wrote:

On 1/25/2012 3:42 PM, Timo Sirainen wrote:

On 26.1.2012, at 1.37, Daniel L. Miller wrote:

Attempting to delete a folder from within the trash folder using 
Thunderbird.  I see the following in the log:

Dovecot version?


2.1.rc3.  I'm compiling rc5 now...


Error still there on rc5.

Jan 25 16:03:47 bubba dovecot: imap(dmil...@amfes.com): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
Jan 25 16:03:47 bubba dovecot: imap(dmil...@amfes.com): Error: Raw 
backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3f1ba) 
[0x7f7c3f0331ba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f206) 
[0x7f7c3f033206] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x1804a) 
[0x7f7c3f00c04a] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47317) 
[0x7f7c3f2c0317] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) 
[0x7f7c3dc48c71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) 
[0x7f7c3dc48d47] -> 
/usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) 
[0x7f7c3dc4c61e] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) 
[0x7f7c3f2b4662] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) 
[0x7f7c3f2b3482] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](cmd_delete+0x72) [0x409972] -> dovecot/imap [dmil...@amfes.com 
192.168.0.91 delete](command_exec+0x3d) [0x4109fd] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40f9ce] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40faad] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete](client_handle_input+0x135) 
[0x40fcd5] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](client_input+0x5f) [0x4105ff] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f7c3f03f5d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f7c3f04065f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f7c3f03f578] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f7c3f02c593] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](main+0x2a5) [0x418a55] -> /lib/libc.so.6(__libc_start_main+0xfe) 
[0x7f7c3ec8fd8e] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete]() [0x408449]
Jan 25 16:03:48 bubba dovecot: imap(dmil...@amfes.com): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
Jan 25 16:03:48 bubba dovecot: imap(dmil...@amfes.com): Error: Raw 
backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3f1ba) 
[0x7f9e52e211ba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f206) 
[0x7f9e52e21206] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x1804a) 
[0x7f9e52dfa04a] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47317) 
[0x7f9e530ae317] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) 
[0x7f9e51a36c71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) 
[0x7f9e51a36d47] -> 
/usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) 
[0x7f9e51a3a61e] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) 
[0x7f9e530a2662] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) 
[0x7f9e530a1482] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](cmd_delete+0x72) [0x409972] -> dovecot/imap [dmil...@amfes.com 
192.168.0.91 delete](command_exec+0x3d) [0x4109fd] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40f9ce] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete]() [0x40faad] -> dovecot/imap 
[dmil...@amfes.com 192.168.0.91 delete](client_handle_input+0x135) 
[0x40fcd5] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](client_input+0x5f) [0x4105ff] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f9e52e2d5d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f9e52e2e65f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f9e52e2d578] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f9e52e1a593] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete](main+0x2a5) [0x418a55] -> /lib/libc.so.6(__libc_start_main+0xfe) 
[0x7f9e52a7dd8e] -> dovecot/imap [dmil...@amfes.com 192.168.0.91 
delete]() [0x408449]
Jan 25 16:03:48 bubba dovecot: imap(dmil...@amfes.com): Fatal: master: 
service(imap): child 3300 killed with signal 6 (core dumps disabled)
Jan 25 16:03:48 bubba dovecot: imap(dmil...@amfes.com): Fatal: master: 
service(imap): child 3267 killed with signal 6 (core dumps disabled)

--
Daniel

Re: [Dovecot] Crash on mail folder delete

[Daniel L. Miller]

On 1/25/2012 4:04 PM, Daniel L. Miller wrote:

On 1/25/2012 3:43 PM, Daniel L. Miller wrote:

On 1/25/2012 3:42 PM, Timo Sirainen wrote:

On 26.1.2012, at 1.37, Daniel L. Miller wrote:

Attempting to delete a folder from within the trash folder using 
Thunderbird.  I see the following in the log:

Dovecot version?


2.1.rc3.  I'm compiling rc5 now...


Error still there on rc5.



Can I do anything to help find this?  Folders are still shown in Trash - 
unable to delete.


--
Daniel

Re: [Dovecot] v2.1.rc6 released

[Daniel L. Miller]

On 2/12/2012 1:01 PM, Timo Sirainen wrote:

http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz
http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig

I've finally finished all of my email backlog. If you haven't received an 
answer to some question/bugreport, resend the mail.

This is hopefully the last v2.1 RC. If I don't receive any (serious) bug 
reports about this release in next few days, I'll just change the version 
number to v2.1.0 (and maybe update man pages, some are still missing..)


With this configure line:
./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene

I get this error on make:
make[4]: Entering directory `/opt/dovecot-2.1.rc6/src/plugins/fts-lucene'
/bin/bash ../../../libtool --tag=CC   --mode=link gcc  -std=gnu99 -g -O2 
-Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arh 
-Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime 
-Wstrict-aliasing=2-module -avoid-version   -o 
lib20_doveadm_s_lucene_plugin.la -rpath /usr/local/lib/dovecot/doveadm 
doveadm-fts-lucene.lo  -lrt
/bin/bash ../../../libtool --tag=CXX   --mode=compile g++ 
-DHAVE_CONFIG_H -I. -I../../..  -I../../../src/lib 
-I../../../src/lib-mail -I..//../src/lib-index 
-I../../../src/lib-storage -I../../../src/plugins/fts 
-I../../../src/doveadm   -g -O2 -MT lucene-wrapper.lo -MD -MP -MF 
eps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib 
-I../../../src/lib-mail -I../../../src/lib-index 
-I../../../src/l-storage -I../../../src/plugins/fts 
-I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF 
.deps/lucene-wrapper.Tpo -c lucene-wraer.cc  -fPIC -DPIC -o 
.libs/lucene-wrapper.o

lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory
compilation terminated.
make[4]: *** [lucene-wrapper.lo] Error 1
make[4]: *** Waiting for unfinished jobs

--
Daniel

Re: [Dovecot] v2.1.rc6 released

[Daniel L. Miller]

On 2/13/2012 7:00 PM, Timo Sirainen wrote:

On 14.2.2012, at 4.54, Daniel L. Miller wrote:


With this configure line:
./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene

lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory

Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 
version is somewhere around in http://clucene.sourceforge.net/

Ah!  I would've thought configure would catch that though - before make 
did.  Ok.


--
Daniel

[Dovecot] Boehm

[Daniel L. Miller]

As the Boehm GC is an available option, do I take that to mean it's 
beneficial for some users but not most?


--
Daniel

Re: [Dovecot] 2.1.rc5 Cannot delete folders from trash

[Daniel L. Miller]

On 2/12/2012 9:00 AM, Timo Sirainen wrote:

On 12.2.2012, at 15.27, Mike Constabel wrote:


Another Question to this Option:

  #   tb-extra-mailbox-sep:
  # With mbox storage a mailbox can contain either mails or submailboxes,
  # but not both. Thunderbird separates these two by forcing server to
  # accept '/' suffix in mailbox names in subscriptions list.


I thought this is not needed because I use mdbox. But mbox here
means also mdbox and sdbox?

Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b


Should I use all 3 workarounds if I don't know which clients are
in use?

I don't know if delay-newmail is still relevant.



I'm fighting the same issue.  With the current release (2.1rc6) no 
errors are logged - but I'm having problems deleted folders from trash.  
I've enabled tb-extra-mailbox-sep.  Two folders that were previously in 
Trash were successfully removed.  I deleted four more - after several 
Thunderbird restarts, compacts, refreshes, I have killed three of them - 
there is still one folder that won't "die".  Thunderbird reports the 
mailbox doesn't exist, yet it continues to appear.


--
Daniel

Re: [Dovecot] 2.1.rc5 Cannot delete folders from trash

[Daniel L. Miller]

On 2/14/2012 12:06 PM, Willie Gillespie wrote:

On 2/13/2012 8:34 PM, Daniel L. Miller wrote:

I'm fighting the same issue. With the current release (2.1rc6) no errors
are logged - but I'm having problems deleted folders from trash. I've
enabled tb-extra-mailbox-sep. Two folders that were previously in Trash
were successfully removed. I deleted four more - after several
Thunderbird restarts, compacts, refreshes, I have killed three of them -
there is still one folder that won't "die". Thunderbird reports the
mailbox doesn't exist, yet it continues to appear.


Out of curiosity -- does the mailbox actually exist if you look on the 
Dovecot server?

Secondly, is it listed in your subscription file?


At one time - yes they were.  After further playing, they do not exist 
in the on-disk folder structure, nor in the subscription file.  But 
still show in the client - with errors.

--
Daniel

Re: [Dovecot] 2.1.rc5 Cannot delete folders from trash

[Daniel L. Miller]

On 2/14/2012 4:42 PM, Willie Gillespie wrote:

On 02/14/2012 01:40 PM, Daniel L. Miller wrote:

On 2/14/2012 12:06 PM, Willie Gillespie wrote:

On 2/13/2012 8:34 PM, Daniel L. Miller wrote:
I'm fighting the same issue. With the current release (2.1rc6) no 
errors

are logged - but I'm having problems deleted folders from trash. I've
enabled tb-extra-mailbox-sep. Two folders that were previously in 
Trash

were successfully removed. I deleted four more - after several
Thunderbird restarts, compacts, refreshes, I have killed three of 
them -

there is still one folder that won't "die". Thunderbird reports the
mailbox doesn't exist, yet it continues to appear.


Out of curiosity -- does the mailbox actually exist if you look on the
Dovecot server?
Secondly, is it listed in your subscription file?


At one time - yes they were. After further playing, they do not exist in
the on-disk folder structure, nor in the subscription file. But still
show in the client - with errors.


Sounds like a Thunderbird bug then.  With Thunderbird shut down, you 
could browse into the profile.
~\.thunderbird or 
C:\Users\name\AppData\Roaming\Thunderbird\whatever\ImapMail\server.name


Find the mailbox that is not supposed to be there and move/delete it.
The filename will usually just be the mailbox name.  A .msf file is 
Thunderbird's index file.  It will create .sbd folders for any 
sub-mailboxes.


I tried going in via squirrelmail - saw no subfolders under trash.  Went 
back in with Thunderbird - all gone.



--
Daniel

[Dovecot] Solr not indexing

[Daniel L. Miller]

What did I do wrong to make this happen?  "doveadm(dmil...@amfes.com): 
Debug: fts: Indexes disabled for namespace 'shared/%n/'"


doveadm -D fts rescan -u dmil...@amfes.com
doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot
doveadm(root): Debug: Module loaded: 
/usr/local/lib/dovecot/lib01_acl_plugin.so
doveadm(root): Debug: Module loaded: 
/usr/local/lib/dovecot/lib20_fts_plugin.so
doveadm(root): Debug: Module loaded: 
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so
doveadm(root): Debug: Loading modules from directory: 
/usr/local/lib/dovecot/doveadm
doveadm(root): Debug: Module loaded: 
/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
doveadm(root): Debug: Skipping module doveadm_expire_plugin, because 
dlopen() failed: 
/usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined 
symbol: expire_set_lookup (this is usually intentional, so just ignore 
this message)
doveadm(root): Debug: Skipping module doveadm_quota_plugin, because 
dlopen() failed: 
/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined 
symbol: quota_user_module (this is usually intentional, so just ignore 
this message)
doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because 
dlopen() failed: 
/usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined 
symbol: i_stream_create_deflate (this is usually intentional, so just 
ignore this message)
doveadm(root): Debug: Module loaded: 
/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so
doveadm(dmil...@amfes.com): Debug: Added userdb setting: 
mail=mdbox:/var/mail/amfes.com/dmiller/mdbox
doveadm(dmil...@amfes.com): Debug: Added userdb setting: 
plugin/acl_groups='allshared'
doveadm(dmil...@amfes.com): Debug: Effective uid=5000, gid=8, 
home=/var/mail/amfes.com/dmiller
doveadm(dmil...@amfes.com): Debug: Namespace : type=private, prefix=, 
sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes 
location=mdbox:/var/mail/amfes.com/dmiller/mdbox
doveadm(dmil...@amfes.com): Debug: fs: 
root=/var/mail/amfes.com/dmiller/mdbox, index=, control=, inbox=, alt=
doveadm(dmil...@amfes.com): Debug: Namespace : Using permissions from 
/var/mail/amfes.com/dmiller/mdbox: mode=0700 gid=-1
doveadm(dmil...@amfes.com): Debug: acl: initializing backend with data: 
vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300

doveadm(dmil...@amfes.com): Debug: acl: acl username = dmil...@amfes.com
doveadm(dmil...@amfes.com): Debug: acl: owner = 1
doveadm(dmil...@amfes.com): Debug: acl vfile: Global ACL directory: 
/usr/local/etc/dovecot/global-acls
doveadm(dmil...@amfes.com): Debug: Namespace : type=shared, 
prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, 
subscriptions=no location=mdbox:/var/mail/%d/%n/mdbox
doveadm(dmil...@amfes.com): Debug: shared: 
root=/usr/local/var/run/dovecot, index=, control=, inbox=, alt=
doveadm(dmil...@amfes.com): Debug: fts: Indexes disabled for namespace 
'shared/%n/'
doveadm(dmil...@amfes.com): Debug: acl: initializing backend with data: 
vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300

doveadm(dmil...@amfes.com): Debug: acl: acl username = dmil...@amfes.com
doveadm(dmil...@amfes.com): Debug: acl: owner = 0
doveadm(dmil...@amfes.com): Debug: acl vfile: Global ACL directory: 
/usr/local/etc/dovecot/global-acls


--
Daniel

Re: [Dovecot] Solr not indexing

[Daniel L. Miller]

On 2/24/2012 10:24 PM, Timo Sirainen wrote:

On 25.2.2012, at 8.09, Daniel L. Miller wrote:


What did I do wrong to make this happen?  "doveadm(dmil...@amfes.com): Debug: fts: 
Indexes disabled for namespace 'shared/%n/'"

That's intentional. "shared/%n/" namespace has no mails.

Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently..

Solr is currently not indexing at all.  I've tried a few different 
settings and commands - nothing is re-generating Solr indexes.  I have 
deleted the Solr index completely - it gets generated empty, and is not 
filling.  Does Dovecot think everything is indexed so nothing needs to 
be added?  It SEEMS new messages, after a search, WILL be indexed - but 
not my existing mail store.  How can I reset the internal Dovecot fts log?


--
Daniel

Re: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header

[Daniel L. Miller]

On 4/5/2012 5:59 AM, Charles Marcus wrote:

On 2012-04-05 4:18 AM, Thomas Leuxner  wrote:
> Also with 2.x you may want to use LMTP rather than the LDA Piping.
>
> http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP

I am preparing to convert my main client's postfix_courier-imap setup 
to dovecot 2.1, which currently just uses the postfix virtual delivery 
agent...


Does anyone know if the use of LMTP (or even the dovecot LDA) still 
loses the x-original-to header that the postfix vda adds and that I 
rely heavily on (since I use a lot of aliases), and if it does, is 
there any solution to get the original recipient added back in before 
final delivery?


Everything I'm reading says that LMTP is better, but I really do need 
this header (or one like it) to be there so I know who the original 
recipient was (for filtering and other purposes).


I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA.  I 
have a pure virtual user environment stored in LDAP.  My messages 
include X-Original-To and Delivered-To headers.


I had difficulty getting the LMTP transport to work previously - I may 
revisit that.

--
Daniel

Re: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header

[Daniel L. Miller]

On 4/6/2012 1:00 PM, Charles Marcus wrote:

On 2012-04-06 2:53 PM, Daniel L. Miller  wrote:

I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I
have a pure virtual user environment stored in LDAP. My messages include
X-Original-To and Delivered-To headers.


Well that is great news... at least I'll be able to use the LDA, if 
not LMTP...


Thanks! :)


I had difficulty getting the LMTP transport to work previously - I may
revisit that.


If you do, by all means reply back on whether or not the headers are 
still there...


Thanks again,



From the documentation...
http://www.postfix.org/virtual.8.html

The*virtual*(8)  <http://www.postfix.org/virtual.8.html>   delivery  agent  prepends 
 a "*From*  /sender/
   /time/*_*/stamp/" envelope header to each  message,  prepends  a
   *Delivered-To:*   message  header with the envelope recipient
   address, prepends an*X-Original-To:*  header with the recip-
   ient  address as given to Postfix, prepends a*Return-Path:*
   message header with the envelope sender address,  prepends
   a>  character to lines beginning with "*From*  ", and appends
   an empty line.

Using the Postfix pipe agent, which is what is used with the Dovecot LDA,
http://www.postfix.org/pipe.8.html

*flags=BDFORXhqu.*>  (optional)
  Optional  message  processing  flags. By default, a
  message is copied unchanged.

  *B*   Append a blank line at the end of each  mes-
 sage.  This  is  required  by some mail user
 agents that recognize  "*From*   "  lines  only
 when preceded by a blank line.

  *D*   Prepend  a "*Delivered-To:*  /recipient/" message
 header with the envelope recipient  address.
 Note: for this to work, the/transport/*_desti-*
 *nation_recipient_limit*  must be 1  (see  SIN-
 GLE-RECIPIENT DELIVERY above for details).

 The*D*   flag  also  enforces  loop detection
 (Postfix  2.5  and  later):  if  a   message
 already contains a*Delivered-To:*  header with
 the same recipient address, then the message
 is  returned  as  undeliverable. The address
 comparison is case insensitive.

 This feature is available as of Postfix 2.0.

  *F*   Prepend  a "*From*  /sender time/*_*/stamp/" envelope
 header to  the  message  content.   This  is
 expected by, for example,*UUCP*  software.

  *O*   Prepend  an  "*X-Original-To:*  /recipient/" mes-
 sage header with the  recipient  address  as
 given  to  Postfix.  Note: for this to work,
 the*/transport/_destination_recipient_limit  
<http://www.postfix.org/postconf.5.html#transport_destination_recipient_limit>*
 must  be  1  (see  SINGLE-RECIPIENT DELIVERY
 above for details).


Unfortunately, the docs for the ltmp agent 
http://www.postfix.org/lmtp.8.html don't say anything about adding these 
headers.  I tried asking on the Postfix list - didn't get much of an 
answer.

--
Daniel

Re: [Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header

[Daniel L. Miller]

  

On Sat, 7 Apr 2012 14:30:38 -0400, Jerry wrote: 

> On Sat, 07 Apr
2012 11:06:48 -0700
> Daniel L. Miller articulated:
> 
>> Unfortunately,
the docs for the ltmp agent http://www.postfix.org/lmtp.8.html [1] don't
say anything about adding these headers. I tried asking on the Postfix
list - didn't get much of an answer.
> 
> I may be wrong; however, from
what I have been able to understand in
> regards to the Postfix
documentation, if it does not explicitly claim to
> have a feature, then
that feature is not available. In other words, if
> it doesn't state it
can do it, it can't.

As I just stated on that list - even though a
given feature may be documented, the possible uses of that feature may
not be immediately apparent. And while the Postfix lda & virtual
transports have the "flag" parameters, and the lmtp transport does not -
the lmtp transport DOES have a whole slew of other parameters not
available in the lda. So I was simply inquiring if there was a way to
achieve my goal - given that my understanding of smtp handling in
general, and Postfix in particular, are extremely limited. 

For some
reason, I seem to irritate people with my polite questions - while
others who are (in my opinion) downright rude and aggressive get
assistance and acceptance. Maybe I need to start being more of a jerk on
purpose... 

-- 
Daniel
  

Links:
--
[1]
http://www.postfix.org/lmtp.8.html

Re: [Dovecot] Folder Aliases - Revisited Yet Again Four Years Later

[Daniel L. Miller]

On 8/13/2010 6:07 AM, Timo Sirainen wrote:

On Thu, 2010-08-12 at 17:04 -0700, Daniel L. Miller wrote:

If we can't get clients to agree on a "standard" set of folders
(probably reasonably safe assumption)

[...]


I'm kind of annoyed with listescape though. Hopefully for v2.1 I can
figure out some API changes to make it much easier to modify mailbox
names, and I guess also make such aliasing plugin much easier to
implement.



Did you happen to look into this in your spare time?

--
Daniel

[Dovecot] Single Instance Storage management

[Daniel L. Miller]

Was there a doveadm command that checks the SIS files and removes any 
not referenced?  Will the "doveadm purge -A" catch that?


--
Daniel

[Dovecot] mdbox packing

[Daniel L. Miller]

Is there a way to manually force an mdbox storage to be rebuilt into new 
files?  Particularly files of the maximum size?


--
Daniel

[Dovecot] Dovecot/doveadm crash

[Daniel L. Miller]

Having a problem with a mailbox.  I've been trying to rebuild - but 
doveadm force-resync crashes.  This is mdbox with sis.


doveadm purge -u dmil...@amfes.com
doveadm(dmil...@amfes.com): Panic: file istream.c: line 466 
(i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0)
doveadm(dmil...@amfes.com): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7fb73465467a] -> 
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) 
[0x7fb734654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7fb73462c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) 
[0x7fb73465c49b] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) 
[0x7fb73465c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) 
[0x7fb73465e1c8] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) 
[0x7fb73465bb69] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) 
[0x7fb73465c04d] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_purge+0x921) 
[0x7fb7348d7d71] -> doveadm() [0x40e1e6] -> doveadm() [0x40e018] -> 
doveadm() [0x40ea83] -> doveadm(doveadm_mail_try_run+0x141) [0x40eef1] 
-> doveadm(main+0x3c1) [0x415911] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fb73429deff] 
-> doveadm() [0x40de39]


--
Daniel

Re: [Dovecot] Dovecot/doveadm crash

[Daniel L. Miller]

On 4/30/2012 1:16 PM, Daniel L. Miller wrote:
Having a problem with a mailbox.  I've been trying to rebuild - but 
doveadm force-resync crashes.  This is mdbox with sis.



Here's a backtrace -

gdb --args doveadm force-resync -u msie...@amfes.com Sent
GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/bin/doveadm...done.
(gdb) run
Starting program: /usr/local/bin/doveadm force-resync -u 
msie...@amfes.com Sent

[Thread debugging using libthread_db enabled]
doveadm(msie...@amfes.com): Warning: mdbox 
/var/mail/amfes.com/msiegel/mdbox/storage: rebuilding indexes
doveadm(msie...@amfes.com): Panic: file istream.c: line 466 
(i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0)
doveadm(msie...@amfes.com): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7765467a] -> 
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) 
[0x77654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7762c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) 
[0x7765c49b] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) 
[0x7765c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) 
[0x7765e1c8] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) 
[0x7765bb69] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) 
[0x7765c04d] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(dbox_file_metadata_read+0xd0) 
[0x778dd870] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x3b958) 
[0x778da958] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x2a9) 
[0x778daff9] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x6b4) 
[0x778d97a4] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x34) 
[0x778d98d4] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x87) 
[0x778d99b7] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) 
[0x77918871] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) 
[0x77918987] -> /usr/local/bin/doveadm() [0x40e2ff] -> 
/usr/local/bin/doveadm() [0x40e018] -> /usr/local/bin/doveadm() 
[0x40ea83] -> /usr/local/bin/doveadm(doveadm_mail_try_run+0x141) 
[0x40eef1] -> /usr/local/bin/doveadm(main+0x3c1) [0x415911] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7729deff] 
-> /usr/local/bin/doveadm() [0x40de39]


Program received signal SIGABRT, Aborted.
0x772b2d05 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt full
#0  0x772b2d05 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x772b6ab6 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x77654688 in default_fatal_finish (type=out>, status=0) at failures.c:191
backtrace = 0x648140 
"/usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7765467a] -> 
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) 
[0x77654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fat"...
#3  0x77654762 in default_fatal_handler (ctx=0x7fffdc80, 
format=, args=)

at failures.c:205
status = 0
#4  0x7762c2a0 in i_panic (format=0x62b6 bounds>) at failures.c:263

ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0}
args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 
0x7fffdd50, reg_save_area = 0x7fffdc90}}
#5  0x7765c49b in i_stream_grow_buffer (stream=0x6c7590, 
bytes=) at istream.c:466

old_size = 
__FUNCTION__ = "i_stream_grow_buffer"
#6  0x7765c522 in i_stream_get_buffer_space (stream=0x6c7590, 
wanted_size=, size_r=)

at istream.c:500
__FUNCTION__ = "i_stream_get_buffer_space"
#7  0x7765e1c8 in i_stream_file_read (stream=0x6c7590) at 
istream-file.c:58

fstream = 0x6c7590
size = 
ret = 
__FUNCTION__ = "i_stream_file_read"
#8  0x7765bb69 in i_stream_read (stream=0x6c75f0) at istream.c:130
_stream = 0x6c7590
old_size = 8192
ret = 
__FUNCTION__ = "i_stream_read"
#9  0x7765c04d in i_stream_read_next_line (stream=0x6c75f0) at 
istream.c:382

line = 
#10 0x778d

[Dovecot] mdbox corruption

[Daniel L. Miller]

I have isolated the corruption in a mailbox to 4 mdbox files.  Two of 
these (smaller) files cause the force-resync to report a breakage and 
save the file as m.XX.broken.  The other two (larger) files cause a crash.


Is there a way I can fix these files and recover the mails?
--
Daniel

Re: [Dovecot] Dovecot/doveadm crash

[Daniel L. Miller]

On 5/4/2012 10:14 AM, Timo Sirainen wrote:

On 30.4.2012, at 23.16, Daniel L. Miller wrote:


Having a problem with a mailbox.  I've been trying to rebuild - but doveadm 
force-resync crashes.  This is mdbox with sis.

doveadm purge -u dmil...@amfes.com
doveadm(dmil...@amfes.com): Panic: file istream.c: line 466 (i_stream_grow_buffer): 
assertion failed: (stream->max_buffer_size>  0)

http://hg.dovecot.org/dovecot-2.1/rev/fa6662ab4df3 should fix this.



Thank you - that worked.

--
Daniel

[Dovecot] Corrupted mdbox file

[Daniel L. Miller]

With an error like this:

doveadm(dmil...@amfes.com): Error: Corrupted dbox file 
/var/mail/amfes.com/dmiller/mdbox/storage/m.20 (around offset=74408): 
Unexpected EOF while reading metadata header


What can be done?
--
Daniel

Re: [Dovecot] mdbox packing

[Daniel L. Miller]

On 4/30/2012 5:23 PM, Timo Sirainen wrote:

On Mon, 2012-04-30 at 12:43 -0700, Daniel L. Miller wrote:

Is there a way to manually force an mdbox storage to be rebuilt into new
files?  Particularly files of the maximum size?

You could do it with dsync.



dsync seems to do a marvelous job - new question.?

Given a mailbox with a larger number of older mails - assuming any new 
mails will have later dates - will the new mail storage files be 
identical?  I'm probably not saying the right - let me try this:


1.  "dsync backup mdbox" (with appropriate args) is run for a given user.
2.  "dsync mirror mdbox" is done just to catch up.
3.  Old mailstore moved off.
4.  New mailstore moved to active location.

I'm sure there's a safer way to do the above - but I've got a 
low-traffic site and I can just shut down mail service altogether for a 
few minutes if I want during this.


Now - repeat the above four steps.  Will mail files m.1 through m.(n-1) 
be identical to the last run?  Is this a valid packing strategy prior to 
performing an rsync type backup - assuming no changes are being made to 
the archived mails between pack runs?

--
Daniel

Re: [Dovecot] mdbox packing

[Daniel L. Miller]

On 5/6/2012 8:09 PM, Daniel L. Miller wrote:

On 4/30/2012 5:23 PM, Timo Sirainen wrote:

On Mon, 2012-04-30 at 12:43 -0700, Daniel L. Miller wrote:
Is there a way to manually force an mdbox storage to be rebuilt into 
new

files?  Particularly files of the maximum size?

You could do it with dsync.



dsync seems to do a marvelous job - new question.?

Given a mailbox with a larger number of older mails - assuming any new 
mails will have later dates - will the new mail storage files be 
identical?  I'm probably not saying the right - let me try this:


1.  "dsync backup mdbox" (with appropriate args) is run for a given user.
2.  "dsync mirror mdbox" is done just to catch up.
3.  Old mailstore moved off.
4.  New mailstore moved to active location.

I'm sure there's a safer way to do the above - but I've got a 
low-traffic site and I can just shut down mail service altogether for 
a few minutes if I want during this.


Now - repeat the above four steps.  Will mail files m.1 through 
m.(n-1) be identical to the last run?  Is this a valid packing 
strategy prior to performing an rsync type backup - assuming no 
changes are being made to the archived mails between pack runs?


Given the above, and SIS - if the backup is performed in the same spool, 
i.e. original is /var/mail/domain/user, backup is 
/var/mail/domain/user-new - will this result in any problems with SIS?  
If after the pack and rename operation, the old mail tree is simply 
deleted - will that leave SIS files unreferenced if the user later 
deletes the messages from their active store?


--
Daniel

Re: [Dovecot] mdbox packing

[Daniel L. Miller]

On 5/6/2012 8:12 PM, Daniel L. Miller wrote:

On 5/6/2012 8:09 PM, Daniel L. Miller wrote:

On 4/30/2012 5:23 PM, Timo Sirainen wrote:

On Mon, 2012-04-30 at 12:43 -0700, Daniel L. Miller wrote:
Is there a way to manually force an mdbox storage to be rebuilt 
into new

files?  Particularly files of the maximum size?

You could do it with dsync.



dsync seems to do a marvelous job - new question.?

Given a mailbox with a larger number of older mails - assuming any 
new mails will have later dates - will the new mail storage files be 
identical?  I'm probably not saying the right - let me try this:


1.  "dsync backup mdbox" (with appropriate args) is run for a given 
user.

2.  "dsync mirror mdbox" is done just to catch up.
3.  Old mailstore moved off.
4.  New mailstore moved to active location.

I'm sure there's a safer way to do the above - but I've got a 
low-traffic site and I can just shut down mail service altogether for 
a few minutes if I want during this.


Now - repeat the above four steps.  Will mail files m.1 through 
m.(n-1) be identical to the last run?  Is this a valid packing 
strategy prior to performing an rsync type backup - assuming no 
changes are being made to the archived mails between pack runs?


Given the above, and SIS - if the backup is performed in the same 
spool, i.e. original is /var/mail/domain/user, backup is 
/var/mail/domain/user-new - will this result in any problems with 
SIS?  If after the pack and rename operation, the old mail tree is 
simply deleted - will that leave SIS files unreferenced if the user 
later deletes the messages from their active store?




And yet another one - dsync does not APPEAR to be copying ACL's in this 
process.


--
Daniel

[Dovecot] Shared namespace

[Daniel L. Miller]

I'm not sure what's triggered it - I THINK it has something to do with a 
force-resync, but not sure.


I'm no longer viewing my full list of shared mailboxes.  I used to have 
a number of users shown - they had all had their ACL's set individually 
and were shown in the list.  "doveadm acl debug" shows they SHOULD be 
shown - at least I think so, as the output for the mailboxes that DO 
appear is identical for those that don't.


Where should I look for breakage?
--
Daniel

Re: [Dovecot] Shared namespace

[Daniel L. Miller]

On 5/6/2012 8:34 PM, Daniel L. Miller wrote:
I'm not sure what's triggered it - I THINK it has something to do with 
a force-resync, but not sure.


I'm no longer viewing my full list of shared mailboxes.  I used to 
have a number of users shown - they had all had their ACL's set 
individually and were shown in the list.  "doveadm acl debug" shows 
they SHOULD be shown - at least I think so, as the output for the 
mailboxes that DO appear is identical for those that don't.


Where should I look for breakage?


After upgrading to 2.1.6 - mailboxes are back.  Not sure I want to TRY 
to break it again...


--
Daniel

Re: [Dovecot] Shared namespace

[Daniel L. Miller]

On 5/7/2012 6:46 AM, Daniel L. Miller wrote:

On 5/6/2012 8:34 PM, Daniel L. Miller wrote:
I'm not sure what's triggered it - I THINK it has something to do 
with a force-resync, but not sure.


I'm no longer viewing my full list of shared mailboxes.  I used to 
have a number of users shown - they had all had their ACL's set 
individually and were shown in the list.  "doveadm acl debug" shows 
they SHOULD be shown - at least I think so, as the output for the 
mailboxes that DO appear is identical for those that don't.


Where should I look for breakage?


After upgrading to 2.1.6 - mailboxes are back.  Not sure I want to TRY 
to break it again...


Ok - I broke it again.  Tried changing mail_shared_explicit_inbox to no 
- problem mailboxes disappeared again.  Changed back to yes - they came 
back.


Problem with both Thunderbird and Roundcube as clients.
--
Daniel

Re: [Dovecot] Shared namespace

[Daniel L. Miller]

On 5/7/2012 11:02 AM, Timo Sirainen wrote:

On 7.5.2012, at 16.52, Daniel L. Miller wrote:


Ok - I broke it again.  Tried changing mail_shared_explicit_inbox to no - 
problem mailboxes disappeared again.  Changed back to yes - they came back.

Problem with both Thunderbird and Roundcube as clients.

Try talking IMAP protocol manually:

a login user pass
b list "" *
c lsub "" *

Are you only talking about shared INBOXes or also other shared mailboxes? 
mail_shared_explicit_inbox=yes is the default and the same behavior as in previous 
versions. I don't think that change should be breaking anything.. Changing it to 
"no" could break your INBOX subscriptions, but other mailboxes should be 
visible.

Ok - that gives us a clue.  With "yes", the output from list & lsub is 
basically the same.  With "no" - I see a truncated list (missing 
mailboxes), but lsub still shows them all.


--
Daniel

Re: [Dovecot] Shared namespace

[Daniel L. Miller]

On 5/7/2012 12:24 PM, Timo Sirainen wrote:

On 7.5.2012, at 22.13, Daniel L. Miller wrote:


On 5/7/2012 11:02 AM, Timo Sirainen wrote:

On 7.5.2012, at 16.52, Daniel L. Miller wrote:


Ok - I broke it again.  Tried changing mail_shared_explicit_inbox to no - 
problem mailboxes disappeared again.  Changed back to yes - they came back.

Problem with both Thunderbird and Roundcube as clients.

Try talking IMAP protocol manually:

a login user pass
b list "" *
c lsub "" *

Are you only talking about shared INBOXes or also other shared mailboxes? 
mail_shared_explicit_inbox=yes is the default and the same behavior as in previous 
versions. I don't think that change should be breaking anything.. Changing it to 
"no" could break your INBOX subscriptions, but other mailboxes should be 
visible.


Ok - that gives us a clue.  With "yes", the output from list&  lsub is basically the 
same.  With "no" - I see a truncated list (missing mailboxes), but lsub still shows them all.

By missing mailboxes I guess you mean the INBOXes aren't anymore in the LIST 
output, which is exactly what the setting is supposed to do?


Not exactly.  Using the old style, I'd see something like:

shared\
  user1\
inbox
  user2\
inbox
  user3\
inbox
  user4\
inbox

When it breaks, for whatever reason (whether it's the new setting or 
something I do), I get:


shared\
  user2\
  user4\

So user1 & user3 are missing entirely.  If it "breaks" using the old 
style, then I would see user 2 & 4 inboxes - but not user 1 & 3.  I also 
have other folders shared from each user.


The breakage is that for some reason certain users' mailboxes are simply 
invisible.  As I said, using list vs lsub (and I don't know how those 
commands are used by clients), lsub will show mailboxes where list doesn't.


--
Daniel

Re: [Dovecot] Shared namespace

[Daniel L. Miller]

On 5/7/2012 1:42 PM, Timo Sirainen wrote:

On 7.5.2012, at 23.27, Timo Sirainen wrote:


On 7.5.2012, at 23.15, Timo Sirainen wrote:


So user1&  user3 are missing entirely.  If it "breaks" using the old style, then I 
would see user 2&  4 inboxes - but not user 1&  3.  I also have other folders shared from 
each user.

Ah, I see. Could you try if the attached patch fixes it?

No, it doesn't. I'll do some more tests.

OK, these should do it:

http://hg.dovecot.org/dovecot-2.1/rev/20c9446e537e
http://hg.dovecot.org/dovecot-2.1/rev/41f2bcb43dad



Ok - new problem.  Now the user mailboxes DO appear - however that's ALL 
I see.  The Sent/Trash/whatever folders, that were previously shared & 
visible - gone.


Changing back to "yes" brings them back.
--
Daniel

Re: [Dovecot] dovecot wants to access my music directory

[Daniel L. Miller]

On 5/10/2012 10:02 PM, Ian Pilcher wrote:

I am getting an SELinux error every time dovecot starts, because it is
trying to access my music directory (/srv/music).  I've read the
doveadm-mount man page, and tried:

   doveadm mount add /srv/music ignore

but it didn't make any difference.

Now, I certainly didn't tell dovecot to access this directory, so how
can I tell it *not* to do so.

This is dovecot-2.1.6-2.fc17.x86_64 on Fedora 17 Beta, BTW.

Thanks!


Is your /srv/music shown in the output from "doveadm mount list"?

Is /srv/music referenced via a symlink from another folder - possibly a 
home folder?


--
Daniel

Re: [Dovecot] gnutls support

[Daniel L. Miller]

On 9/16/2010 5:41 AM, Timo Sirainen wrote:

On Wed, 2010-09-15 at 20:25 -0700, Daniel L. Miller wrote:

Other than license issues, is there an advantage to using gnutls vs
openssl?  Or is openssl superior - at least in the current implementations?

Dovecot's GNUTLS support was written long time ago and its API has
changed since. It doesn't work. But a working GNUTLS support would still
be nice some day. I don't much like OpenSSL.



With 2.1.7 - is GNUTLS supported?

--
Daniel

[Dovecot] Indexes

[Daniel L. Miller]

When something "bad" happens to the indexes, my e-mail client 
(Thunderbird) reports an "unable to succeed" error on opening a 
mailbox.  Leaving that mailbox and coming back works fine.  Is this 
expected behaviour?


--
Daniel

[Dovecot] Indexes

[Daniel L. Miller]

To be more specific:

Alert
The current command did not succeed.  The mail server responded: 
Internal error occurred. Refer to server log for more information. 
[timestamp]

--
Daniel

Re: [Dovecot] Indexes

[Daniel L. Miller]

Scott Silva wrote:

on 12/31/2007 10:54 AM Daniel L. Miller spake the following:
When something "bad" happens to the indexes, my e-mail client 
(Thunderbird) reports an "unable to succeed" error on opening a 
mailbox.  Leaving that mailbox and coming back works fine.  Is this 
expected behaviour?


You need to at least include which version of dovecot you are using, 
as there are some very old versions in use on some linux distros.

It could be thunderbird timing out as the indexes get rebuilt.


Ubuntu, Dovecot 1.0.5

--
Daniel

Re: [Dovecot] Indexes

[Daniel L. Miller]

Timo Sirainen wrote:

On Mon, 2007-12-31 at 10:54 -0800, Daniel L. Miller wrote:
  
When something "bad" happens to the indexes, my e-mail client 
(Thunderbird) reports an "unable to succeed" error on opening a 
mailbox.  Leaving that mailbox and coming back works fine.  Is this 
expected behaviour?



It's expected, but it's not expected that anything bad happens to your
indexes normally. What do the logs show as the reason?

  

I see a variety, but a sample:

Dec 31 11:09:26 bubba dovecot: IMAP([EMAIL PROTECTED]): Corrupted 
transaction log file 
/var/mail/amfes.com/mgillespie/.Trash/dovecot.index.log: start_offset 
(20200) > file size (20184)
Dec 31 11:09:30 bubba dovecot: IMAP([EMAIL PROTECTED]): Transaction 
log file /var/mail/amfes.com/mgillespie/.Trash/dovecot.index.log: marked 
corrupted
Dec 31 11:09:34 bubba dovecot: IMAP([EMAIL PROTECTED]): Unexpected 
transaction log desync with index 
/var/mail/amfes.com/mgillespie/.Trash/dovecot.index


--
Daniel

Re: [Dovecot] Indexes

[Daniel L. Miller]

Timo Sirainen wrote:

On Wed, 2008-01-02 at 09:55 -0800, Daniel L. Miller wrote:
  

Timo Sirainen wrote:


On Mon, 2007-12-31 at 10:54 -0800, Daniel L. Miller wrote:
  
  
When something "bad" happens to the indexes, my e-mail client 
(Thunderbird) reports an "unable to succeed" error on opening a 
mailbox.  Leaving that mailbox and coming back works fine.  Is this 
expected behaviour?



It's expected, but it's not expected that anything bad happens to your
indexes normally. What do the logs show as the reason?

  
  

I see a variety, but a sample:

Dec 31 11:09:26 bubba dovecot: IMAP([EMAIL PROTECTED]): Corrupted 
transaction log file 
/var/mail/amfes.com/mgillespie/.Trash/dovecot.index.log: start_offset 
(20200) > file size (20184)



What filesystem is this with? Looks like NFS caching issues.
  
XFS, on a RAID-10, local to the Dovecot server.  No NFS used anywhere in 
the network - Samba used for filesharing. 


--
Daniel

[Dovecot] Multiple SSL certs

[Daniel L. Miller]

Does 1.1 support multiple certs?

--
Daniel

Re: [Dovecot] Disallow folder delete

[Daniel L. Miller]

Timo Sirainen wrote:

On May 20, 2008, at 2:07 AM, Kenneth Porter wrote:

Is there a straightforward way to disallow the deletion of all IMAP 
mailboxes?


I have a user who's deleted an important IMAP mailbox and I'm now 
recovering a recent copy from the backup. But I'd rather just blanket 
disallow all folder deletions.


The user is using Thunderbird and this has happened more than once so 
I suspect Tbird is willing to let a folder get deleted too easily. 
Perhaps there was a delay in the confirmation dialog and the user 
clicked ahead and confirmed something he shouldn't have.


I'm looking at  and it looks like I 
should be able to use a group override to disallow "x" (mailbox 
delete) but the page says that groups aren't implemented, so I don't 
know how one says that nobody can do this for any mailbox.


Using global ACLs gets you closer at least. You can define:

acl = vfile:/etc/dovecot/acls

Then having /etc/dovecot/acls/.DEFAULT probably does something.. But I 
don't remember if it applies to all mailboxes or just those on the 
root level or what. I should look into this some day and make all of 
them possible. :)
This actually may have a real benefit to some specialty folders - such 
as junk/nojunk folders that are looked for by a scanning re-classifier.  
Can you give an example of how to make...Inbox/Junk non-deletable (or 
even better, update the wiki)?


--
Daniel

Re: [Dovecot] Antispam plugin custom behavior?

[Daniel L. Miller]

Kyle Wheeler wrote:

On Wednesday, June 11 at 05:01 PM, quoth Hugo Monteiro:

Have you tried the plugin using the mailtrain backend?


The antispam plugin? No, I haven't... mostly because it looks like no 
matter which backend I use, I'd have to alter the user-visible 
interface to my training system (which I don't really want to do), and 
it still doesn't handle the "altered message" problem.


Basically it will forward the message, as attachment, to spam/notspam 
addresses that you define. That includes the use of a %u variable 
expansion, if you choose to use retrain addresses like like 
[EMAIL PROTECTED] or something. I've been pretty happy with 
with it and it scales a lot better than piping the message into a 
retrain command, since the mail system itself will handle the load in 
a more intelligent way.


Hmmm, load is something I hadn't thought about... (the system I'm 
working with at the moment has plenty of capacity to spare). That's a 
good point.

Just thinking out loud here...

Couple possibilities:
1.  Have an SMTP server dedicated to spam re-processing.  This dedicated 
function SMTP server could use your existing scripts to strip the 
unwanted headers and then pass it back to your standard server.  This 
would be trivial with Postfix, either using domain mapping or dedicated 
IP/port listeners.
2.  If you're not using an SMTP server with Postfix's flexibility, use 
something like smtpprox to add it.


Bottom line is use a tool that works within the IMAP server (the 
anti-spam plugin) to detect mail changes and then pass the message on to 
a separate process for treatment.

--
Daniel

Re: [Dovecot] 9GB of messages in /tmp

[Daniel L. Miller]

Charles Marcus wrote:

Hello,

Well, I have a brain-dead user (using Thunderbird 2.0.0.14) who just 
won't listen when I tell them not to try to move 5000+ messages at one 
time (or if deleting, bypass the Trash), and they managed to royally 
screw up their maildirs...
It's never bothered me - though I'm typically on a local IMAP 
connection.  Is yours different?


I have two problems...

1. The /tmp directory has 9+GB in it... can I just rm tmp/* on that 
directory without making this problem worse?
That'll make 'em go away...shouldn't compromise anything, unless they 
moved good stuff they wanted to keep


and

2. One of the directories they were moving now has 5 copies of a whole 
bunch of messages...


I'm very dangerous when it comes to command-line work beyond really 
basic stuff, and my scripting skills are less than inspiring...


Is there a relatively safe command-line way of finding all messages 
that have more than one instance, and deleting all but the last instance?
The only way to determine if the messages are the same would be to do a 
full-text comparison, per message.  Not fun.  Tell the user they get to 
enjoy processing their mail by hand - an object lesson in why they 
should listen to their sysadmin.


--
Daniel

Re: [Dovecot] v1.1.rc12 released

[Daniel L. Miller]

Timo Sirainen wrote:

http://dovecot.org/releases/1.1/rc/dovecot-1.1.rc12.tar.gz
http://dovecot.org/releases/1.1/rc/dovecot-1.1.rc12.tar.gz.sig

Sorry, one more RC because of the mbox+quota bug. v1.1.0 still planned
for tomorrow.

- mbox: Don't give "Can't find next message offset" warnings when
  plugin (e.g. quota) accesses the message being saved.
- deliver: Settings inside protocol imap {} weren't ignored.

  
Oh wow.  I can't wait to start nagging the Debian/Ubuntu project 
managers to update their packages!


--
Daniel

Re: [Dovecot] Ubuntu packages [Was: Re: v1.1.1 released]

[Daniel L. Miller]

Wolfram Schlich wrote:

Has anyone already built 1.1.1 packages for Ubuntu 8.04?!

Thanks,
Wolfram
  
The Debian maintainers are currently reviewing 1.1.x - they're looking 
to determine if 1.1.x should be considered stable enough for inclusion 
in their next release ("Lenny") or wait'll the next one 
(psstTimothis is your cue to voice your opinion).  I haven't 
gotten word back from the Ubuntu maintainers yet, but certainly if 
Debian makes the move Ubuntu will follow close behind.


--
Daniel

Re: [Dovecot] fd limit 1024 is lower in dovecot-1.1.1

[Daniel L. Miller]

Zhang Huangbin wrote:

Hi, all.

I just upgrade from 1.0.15 to 1.1.1 in a test box(RHEL 5.2, x86_64).

after upgrade, i got this warning msg:

8< 
# /etc/init.d/dovecot restart
Stopping Dovecot Imap: [  OK  ]
Starting Dovecot Imap: Warning: fd limit 1024 is lower than what 
Dovecot can use under full load (more than 1280). Either grow the 
limit or change login_max_processes_count and max_mail_processes settings

  [  OK  ]
8< 

but i changed either login_max_processes_count and max_mail_processes
to 2048, it raised the same msg. How can i solove this issue?

I'm just guessing - but reading that warning it appears to me that 
Dovecot is saying that as it is configured, it can consume more O/S 
resources (I assume fd is "file descriptors") than the O/S is currently 
configured for.  So you need to DECREASE your dovecot max processes to 
decrease the (potential) system demands - or increase your O/S settings.


Daniel

Re: [Dovecot] ManageSieve v0.10.3 released for Dovecot 1.1.1

[Daniel L. Miller]

Stephan Bosch wrote:

Hello Dovecot users,

I have formulated a new release that addresses an important 
ManageSieve compilation issue that arises when compiling with GCC 4.3 
(as used by the new Debian Lenny).
Sorry for being slightly off-topic - but if you change your code to be 
compliant with GCC-4.3, would it still compile under older versions?


--
Daniel

[Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

I know I could test this - but I'd rather ask first.  To my knowledge, 
we haven't come up with a good server-side implementation for savings 
copies of sent messages (by all means correct my ignorance in this 
regard).  So the typical way is to enable it in clients like Thunderbird 
(this prompted a whole discussion of how to save these messages without 
sending them to the server multiple times).


My understanding is that sending a message from a client (use 
Thunderbird for simplicity of this conversation) is performed via SMTP.  
Saving a copy into a sent folder is performed via IMAP (hence the 
multiple transfers to the server).  Now that I've laid a background - 
let's make it Dovecot specific.  I don't know how "behind-the-scenes" 
Dovecot performs the act of saving mail messages that it receives from 
IMAP, instead of SMTP.  Specifically, if I'm using sieve filters via 
deliver - can I setup a filter that will place mail copies to specific 
recipients into specific subfolders?

--
Daniel

Re: [Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

Patrick Nagel wrote:

Hi,

On Mon, 28 Jul 2008 09:38:19 -0700, "Daniel L. Miller" <[EMAIL PROTECTED]>
wrote:
  
My understanding is that sending a message from a client (use 
Thunderbird for simplicity of this conversation) is performed via SMTP.  
Saving a copy into a sent folder is performed via IMAP (hence the 
multiple transfers to the server).  Now that I've laid a background - 
let's make it Dovecot specific.  I don't know how "behind-the-scenes" 
Dovecot performs the act of saving mail messages that it receives from 
IMAP, instead of SMTP.  Specifically, if I'm using sieve filters via 
deliver - can I setup a filter that will place mail copies to specific 
recipients into specific subfolders?



It seems to me, that you are assuming that Dovecot also speaks SMTP and is
able to actually handle sending of mails to other hosts. To clarify:
Dovecot is an IMAP/POP3 mail server (with integrated Mail Delivery Agent
(MDA) or Local Delivery Agent (LDA) (synonyms)). SMTP is being handled by a
Mail Transfer Agent (MTA) such as Postfix.
While it's usually appropriate to assume I'm ignorant, naive, or 
generally speaking out of my @ss, I'm afraid the responses I've received 
don't answer the question I asked.  I do understand how the MUA talks to 
the MTA and the whole SMTP/IMAP thing.  I rambled a little bit just in 
case someone came up with a better solution for the double transmit 
scenario - but that wasn't the question.


The question:  can Dovecot be configured to support sieve filters on 
IMAP (not just SMTP) operations (Eduardo helped me here - so I'll 
specifically ask about the APPEND command).


If not , I'll make that a feature-request - "Timo, can Dovecot be 
modified to support sieve-filters on APPEND operations?".  Failing that, 
can some specific operation subsets of sieve be supported on a cron-like 
basis?  My understanding is that operations involving indexes are 
extremely fast - and that things like sender/recipient are included in 
the indexes.  So filters based on sender/recipient might be possible?


--
Daniel

Re: [Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

Eduardo M KALINOWSKI wrote:

Daniel L. Miller escreveu:
The question:  can Dovecot be configured to support sieve filters on 
IMAP (not just SMTP) operations (Eduardo helped me here - so I'll 
specifically ask about the APPEND command).


If not , I'll make that a feature-request - "Timo, can Dovecot be 
modified to support sieve-filters on APPEND operations?".  Failing 
that, can some specific operation subsets of sieve be supported on a 
cron-like basis?  My understanding is that operations involving 
indexes are extremely fast - and that things like sender/recipient 
are included in the indexes.  So filters based on sender/recipient 
might be possible?




At this moment it can't. And as a matter of fact, I doubt it ever 
will, because it doesn't really make sense.


I'm no expert in the IMAP protocol, but as far as I understand, the 
APPEND command is used to store a new mail in a given mailbox. It 
would not make sense to run a filter to store the message somewhere 
other than where the client asked when using that command (just to use 
the specific example).

I never said it was a GOOD ideabut, actually, what's wrong with it?

Once again, an opportunity for me to use my lower mouth.

IMAP is based on supporting relatively high-speed links (not INTENDED 
for dial-up use...).  So mail is (normally) not stored on the 
client-side - and changes made on the server are quickly reflected by 
the client (for LAN use, I normally set my update freq to 1 minute).  
And this is the kind of thing that would be implemented with the user's 
knowledge (presumably) since it's implemented using the user-configured 
sieve filter.  So, without changing the MUA/MTA/IMAP interaction, the 
IMAP server will simply file new messages according to user-set rules.  
Doesn't address the multiple-transfer issue at all, but does provide an 
option for centralized control of message filing.


--
Daniel

Re: [Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

Eduardo M KALINOWSKI wrote:

Daniel L. Miller escreveu:
So, without changing the MUA/MTA/IMAP interaction, the IMAP server 
will simply file new messages according to user-set rules.  Doesn't 
address the multiple-transfer issue at all, but does provide an 
option for centralized control of message filing.


With the APPEND command, storing the mail somewhere that is not the 
default location would be a violation of the protocol:


6.3.11. APPEND Command

  Arguments:  mailbox name
  OPTIONAL flag parenthesized list
  OPTIONAL date/time string
  message literal

  Responses:  no specific responses for this command

  Result: OK - append completed
  NO - append error: can't append to that mailbox, error
   in flags or date/time or message text
  BAD - command unknown or arguments invalid

 The APPEND command appends the literal argument as a new message
 to the end of the specified destination mailbox.  This argument
 SHOULD be in the format of an [RFC-2822 
<http://www.faqs.org/rfcs/rfc2822.html>] message.



The simplest solution would be, as already mentioned, configure the 
client to BCC yourself, and filter that message. (And disable the 
'Store copy of sent mails' option.) I do not think running filters for 
APPEND'ed messages is an option (even if one not active by default).
First - I can't argue the point about BCC'ing - you're certainly correct 
that that would be the "simplest" solution in terms of immediate 
implementation - if we assume users are willing to change their habits.  
Since I don't - I'm exploring server-side options


I disagree about "violating" the protocol.  Nothing about the mail 
server/client communication would change - the client would still 
receive an "OK" at the end.  The difference would be after the message 
appeared in the "Sent" folder (or never show at all), a moment later it 
would disappear and be placed in the sieve-directed folder.  I don't see 
how this violates the protocol - though I do agree it would be quite 
confusing for someone who wasn't aware of this behavior.

--
Daniel

Re: [Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

Anders Melchiorsen wrote:

Daniel L. Miller wrote:
First - I can't argue the point about BCC'ing - you're certainly 
correct that that would be the "simplest" solution in terms of 
immediate implementation - if we assume users are willing to change 
their habits.  Since I don't - I'm exploring server-side options



If you are able to identify the sender, you can use something like the 
always_bcc setting in Postfix, and have it bcc to a filter that stores 
the message back in the right Sent folder.
Now that's not fair!  Providing a reasonable alternative, instead of 
agreeing with my feature request, totally blows my argument!  Especially 
since I'm using Postfix!


But...then I still need to get my users to not have their "Sent" enabled 
or I'll have a lot of duplication.  Including the users who are using 
remote clients.  Hah!  I still want my APPEND-Sieve support!


--
Daniel

Re: [Dovecot] Server-side sieve for client-side copies

[Daniel L. Miller]

Stephan Bosch wrote:

Daniel L. Miller wrote:
  

The question:  can Dovecot be configured to support sieve filters on
IMAP (not just SMTP) operations (Eduardo helped me here - so I'll
specifically ask about the APPEND command).


It is quite funny that you bring this up right now, because as a matter
of fact there is a new IMAP extension in the works that should support
just that:

http://www.ietf.org/internet-drafts/draft-ietf-lemonade-imap-sieve-05.txt

However, as it is now, it is still cause for much debate in the Sieve
community, so I don't think this will yield an RFC any time soon.
  

Drooling with anticipation...%$#! I almost shorted out my keyboard!
--
Daniel

[Dovecot] NFS vs CIFS - and Dovecot

[Daniel L. Miller]

I've seen a number of posts discussing NFS based storage and issues with 
Dovecot.  When the term "NFS" is used - does this exclusively refer to 
the Linux NFS network file system, or is it a generic reference to any 
network file system?  I don't recall seeing any posts referencing issues 
with CIFS or SMBFS - is this because no one is using it, or because it 
just works instead of fiddling with NFS?

--
Daniel

Re: [Dovecot] Procmail versus Dovecot LDA

[Daniel L. Miller]

Jordan Hayes wrote:

I'll note that I'm using procmail because of the ability to filter
and run SpamAssassin from it.


Typically you'd run SpamAssassin first and Dovecot deliver after that.


Is anyone here doing this for virtual users who could describe what 
they did?


Bonus points for actual configuration files :-)

Thanks,

/jordan

What SMTP server are you running?

--
Daniel

Re: [Dovecot] problems with sieve and lda

[Daniel L. Miller]

Harondel J. Sibble wrote:

On 11 Aug 2008 at 8:24, Eduardo M KALINOWSKI wrote:

  

How are you calling deliver in postfix? Do you see in postfix's logs the
messages being handled to deliver?




Hmm, oddly enough, adding a .forward in the users home dir with 


| "/usr/libexec/dovecot/deliver"

Then everything works. Any ideas why?  According to everything I've read 
including the wiki, that shouldn't be necessary.
  
I'll give it a shot.  My Postfix/Dovecot is under Debian/Ubuntu - but 
that shouldn't matter too much.  My configuration is based on all 
virtual users/mailboxes - and all mail is owned by the mail user/group.  
May have to adjust for yours.


First of all, Dovecot authorization has to be setup.  I went whole hog 
and am using Dovecot for client authorization with Postfix - I recommend 
it.  So, in your dovecot.conf file:

auth default {
   [...]
 socket listen {
   master {
 # Master socket provides access to userdb information. It's typically
 # used to give Dovecot's local delivery agent access to userdb so it
 # can find mailbox locations.
 path = /var/run/dovecot/auth-master
 mode = 0600
 # Default user/group is the one who started dovecot-auth (root)
 user = vmail
 group = mail
   }
   client {
 # The client socket is generally safe to export to everyone. 
Typical use

 # is to export it to your SMTP server so it can do SMTP AUTH lookups
 # using it.
 path = /var/spool/postfix/var/dovecot
 mode = 0666
 user = vmail
 group = mail
   }
 }
   [...]
}

Note the two paths - the auth server and client.  These paths must 
exist.  If not ... you're going to have problems.  Also - the client 
path (in my case /var/spool/postfix/var/dovecot) must be read/writeable 
by whatever user deliver is running as.  In particular, 
/var/spool/postfix/var must be readable by the deliver user.


Also - based on the error message you show, you need to verify 
/var/run/dovecot/auth-master.  Same permission issues apply.  Based on 
your reported error messages - this is probably something you need to 
look hard at.


Now in Postfix, ONE of the important settings is smtpd_sasl_path.  In my 
case, it's set to "var/dovecot".  Notice, when added to the chroot that 
Postfix operates in (/var/spool/postfix, for my own server) it matches 
the path defined in the client auth section in dovecot.conf.


Now for me, I wanted even more flexibility than the built-in behaviors 
of Postfix and Dovecot provide - however thanks to the defined 
flexibility of both programs it was easy to get what I wanted.  In 
master.cf, I've defined two transports:


dovecot unix-   n   n   -   -   pipe
   flags=ODRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -f 
${sender} -d ${recipient}


dsieve  unix-   n   n   -   -   pipe
   flags=ODRhu user=vmail:mail 
argv=/usr/local/bin/do_postfix_deliver.sh ${sender} ${recipient}


The "dovecot" transport is for users not using sieve.  I've made that 
the default for users by setting "virtual_transport = dovecot" in main.cf.


The dsieve (short for deliver + sieve) transport enables sieve.  How?  
The magic of /usr/local/bin/do_postfix_deliver.sh:

#!/bin/sh
#
# The following parameters are passed to this script
#
# Sending address
# Destination address
# Message (piped)
#
PD_SENDER=$1
PD_DESTINATION=$2

PD_DOMAIN=`echo $PD_DESTINATION|sed s/@.*$//`
PD_USER=`echo $PD_DESTINATION|sed s/^.*@//`

HOME=/var/mail/${PD_DOMAIN}/${PD_USER}
MAIL=$HOME
export HOME
export MAIL

/usr/lib/dovecot/deliver -f ${PD_SENDER} -d ${PD_DESTINATION}
#EOF

I used this method because I wanted to minimize the information 
necessary in my LDAP database.  For everything mail related, the only 
items necessary are the complete email address and a password - 
everything else is generated from there.  I've been advised I should 
change my MAIL parameter to be something like $HOME/Maildir - I'll 
probably do that when I upgrade from 1.0.


Hope this helps.
--
Daniel

Re: [Dovecot] Webmail app ... again.

[Daniel L. Miller]

Geert Hendrickx wrote:

On Wed, Aug 13, 2008 at 04:37:11PM -0400, Timo Sirainen wrote:
  

One thing that would be nice, that pretty much no webmail does, is to
keep a stateful connection open all the time (or at least some of the
time) instead of creating tons of short-lived connections that ask the
same stuff over and over again. With a stateful connection you could
basically run IDLE and wait for changes there instead of asking all  the
time "is there new mail?" "is there new mail now?" "what about  now?".

I'm sure "native" support would be better, but how is this different 
from using Squirrelmail with IMAPProxy?


--
Daniel

Re: [Dovecot] dovecot: Fatal: Time just moved backwards by 3603 seconds.

[Daniel L. Miller]

Stephen Feyrer wrote:

 Hi Everyone. 

Ooops I just broke my dovecot install.

I've looked through the mailing list archives and not found anything 
relevant.



When I try to connect to my mail server I get this message.
"Could not connect to mail server chen.home.org; connection was refused"


The last message in /var/log/mail.err is:-

Aug 13 21:02:27 chen dovecot: Fatal: Time just moved backwards by 3603 
seconds. This might cause a lot of problems, so I'll just kill myself 
now. http://wiki.dovecot.org/TimeMovedBackwards



It seems this is my own fault.  Time on my system is a little wonky so 
I reset the time and then I found out I'd broken Dovecot.  I didn't 
realise that doing this would have an impact on dovecot.  I read the 
web page and to be honest I'm not confident about hacking source code.


I've reinstalled dovecot which worked except it hasn't solved my time 
travel problem.


By the way since my first install of dovecot the config file seems to 
have changed a little but since I've tried to migrate my settings over 
the new file and been met with the same problem I don't think it's that.


Is it possible to get dovecot working again?
What happens when you try to start Dovecot again?  Not re-install - just 
start it.


--
Daniel

Re: [Dovecot] dovecot: Fatal: Time just moved backwards by 3603 seconds.

[Daniel L. Miller]

Stephen Feyrer wrote:

Daniel L. Miller wrote:

Stephen Feyrer wrote:

 Hi Everyone. 

Ooops I just broke my dovecot install.

I've looked through the mailing list archives and not found anything 
relevant.



When I try to connect to my mail server I get this message.
"Could not connect to mail server chen.home.org; connection was 
refused"



The last message in /var/log/mail.err is:-

Aug 13 21:02:27 chen dovecot: Fatal: Time just moved backwards by 
3603 seconds. This might cause a lot of problems, so I'll just kill 
myself now. http://wiki.dovecot.org/TimeMovedBackwards



It seems this is my own fault.  Time on my system is a little wonky 
so I reset the time and then I found out I'd broken Dovecot.  I 
didn't realise that doing this would have an impact on dovecot.  I 
read the web page and to be honest I'm not confident about hacking 
source code.


I've reinstalled dovecot which worked except it hasn't solved my 
time travel problem.


By the way since my first install of dovecot the config file seems 
to have changed a little but since I've tried to migrate my settings 
over the new file and been met with the same problem I don't think 
it's that.


Is it possible to get dovecot working again?
What happens when you try to start Dovecot again?  Not re-install - 
just start it.




I just did this to show you.

# /etc/init.d/dovecot restart
 * Stopping dovecot ...[ ok ]
 * Starting dovecot ...[ ok ]

And then this...


And then this again...
 # /etc/init.d/dovecot restart
 * Stopping dovecot ... [ ok ]
 * Starting dovecot ...
Warning: Last died with error (see error log for more information): 
Time just moved backwards by 106 seconds. This might cause a lot of 
problems, so I'll just kill myself now. 
http://wiki.dovecot.org/TimeMovedBackwards[ ok ]


I've not seen this before in a restart.

That was followed by...
# /etc/init.d/dovecot restart
 * Stopping dovecot ...[ ok ]
 * Starting dovecot ...[ ok ]


--
Regards

Stephen.
Whoa!  OK - so I'm seeing sometimes it works fine, and sometimes it 
reports this problem?  You started off with a 3306 second jump (an hour 
off) to a 106 second jump (almost two minutes).  Something is REALLY 
whacked with your clock!


Umdo you have a UPS?  If not - get one!
Are you running running an ntp server?  I'm assuming not.  It's time to 
start.


--
Daniel

Re: [Dovecot] Multiple network interface question

[Daniel L. Miller]

Mike Henry wrote:

Hi all,

Sorry if this question has been posted before. I have dug through the 
archives but haven't found any relevant answers to this question.


I am running Dovecot on a server that serves a number of (virtual) 
domains each with a number of (virtual) users.


To read mail I am providing a webmail client and have configured HTTPS 
connections using reverse DNS so that each domain has its own IP 
address and that each IP address gets mapped to a different virtual 
host. Each virtual host presents and HTTPS connection particular to 
their own domain using the SSL certificate for that domain so that 
each user only sees his or her own domain throughout.


Is it possible to configure Dovecot so that mail-clients like 
Thunderbird or Outlook can connect via the default port using their 
own domain (maybe also using reverse DNS) and be presented with an SSL 
connection particular to that domain (i.e. using that domains 
certificate)?


Any comments welcome.

Thanks in advance,

-Mike
At this time, Dovecot does not support multiple SSL certificates.  The 
only way to implement it (currently) would be to install multiple 
instances of Dovecot, listening on different IPs/ports.


--
Daniel

[Dovecot] Search results as virtual folders

[Daniel L. Miller]

This may have been asked before - if so I apologize.

Some mail clients - like Thunderbird - can provide a virtual folder or 
"saved search" to provide a list of mails that match certain criteria.  
Obviously, unless special client-side caching is implemented - which 
Thunderbird (my own client of choice) - does not, generation of these 
folders take time.  Does Dovecot have any provision for generating such 
virtual folders?  Possibly as the result of a sieve script?

--
Daniel

Re: [Dovecot] Released v1.1.5 and v1.2.alpha3

[Daniel L. Miller]

Timo Sirainen wrote:

http://dovecot.org/releases/1.1/dovecot-1.1.5.tar.gz
http://dovecot.org/releases/1.1/dovecot-1.1.5.tar.gz.sig

http://dovecot.org/releases/1.2/alpha/dovecot-1.2.alpha3.tar.gz
http://dovecot.org/releases/1.2/alpha/dovecot-1.2.alpha3.tar.gz.sig

A combined release announcement for a change. :)

Largest changes since v1.1.4:

* Dovecot prints an informational message about authentication problems
  at startup. The message goes away after the first successful
  authentication. This hopefully reduces the number of "Why doesn't
  my authentication work?" questions.

+ Maildir/dbox: Try harder to assign unique UIDVALIDITY values to
  mailboxes to avoid potential problems when recreating or renaming
  mailboxes. The UIDVALIDITY is tracked using dovecot-uidvalidity*
  files in the mail root directory.
+ Many logging improvements
- In some conditions Dovecot could have stopped using existing cache
  file and never used it again until it was deleted.
- pop3 + Maildir: Make sure virtual sizes are always written to
  dovecot-uidlist. This way if the indexes are lost Dovecot will never
  do a huge amount of work to recalculate them.
- mbox: Fixed listing mailboxes in namespaces with prefix beginning
  with '~' or '/' (i.e. UW-IMAP compatibility namespaces didn't work).
- dict quota: Don't crash when recalculating quota (when quota warnings
  enabled).
- Fixes to handling "out of disk space/quota" failures.
- Blocking passdbs/userdbs (e.g. PAM, MySQL) could have failed lookups
  sometimes when auth_worker_max_request_count was non-zero.
- Fixed compiling with OpenBSD

Largest changes since v1.2.alpha2:

+ Added userdb checkpassword (by Sascha Wilde)
+ Autocreate plugin: http://wiki.dovecot.org/Plugins/Autocreate
+ Listescape plugin: http://wiki.dovecot.org/Plugins/Listescape
- All the same fixes as for v1.1.5

I was thinking about waiting for Kolab people's work for shared mailbox
support to get done, get that code to v1.2 tree and then release
v1.2.beta1 with feature freeze.
  
If 1.2 has other features beyond 1.1.5, and is otherwise ready, I would 
go ahead and release 1.2 now (so we can all beat up on it) and add the 
shared support to 1.3.


--
Daniel

Re: [Dovecot] Backing Up

[Daniel L. Miller]

Stewart Dean wrote:

Dave McGuire wrote:

On Oct 29, 2008, at 3:42 PM, Scott Silva wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The 
inital sync
can take a while, but it gets faster after there is a base to work 
from.


  ...and it's much less painful if you're using maildir instead of mbox!

   -Dave

I have to wonder.  I have a mailserver that I do a bootable complete 
image copy of with all files and O/S in two hours to an Ultrium-2 
tape, 95 GB.  When I switch to maildir, I will go from some 25,000 
mbox files to 2.5 to 3 million files...I can't believe that isn't 
going to hurt and will force me into incrementals.
Well, I can't talk about an installation of that size - but I CAN say 
I've been using an rsync based backup and been thrilled with the 
results.  I'm supporting a measly dozen users, but I'm incrementally 
backing up our complete server on a nightly basis.  I think the script 
now takes less than an hour to run - for a complete backup.


Since I'm doing this via a VPN to a server at my house, it makes it 
rather convenient.  I love having an immediately available backup I can 
ssh to - that now represents a daily snapshot for over a year - so when 
one of my users says, "Um...if I deleted a file that I've been working 
on for the past two weeks...can you get it back?"


--
Daniel

[Dovecot] Folder aliases

[Daniel L. Miller]

I don't if this has been discussed before - or already implemented - but 
it would be great if I could define one or more "aliases" for a folder.  
A particular example might be "Sent" where different (badly-written) 
clients might have hard-coded "Sent" folder locations like "Sent", "Sent 
Items", "Sent Mail", "Mail Delivered", etc. - and I'd love to get those 
consolidated.


It would also be nice to show a different set of folders based on the 
client - or if that's not possible (if the client doesn't identify 
itself as part of the login sequence) then somehow via login name - 
maybe by an extension?  So where my usual login might be 
"[EMAIL PROTECTED]", and that would continue to be supported as the 
default login, I could now have "[EMAIL PROTECTED]", 
"[EMAIL PROTECTED]", etc.?

--
Daniel

Re: [Dovecot] Folder aliases

[Daniel L. Miller]

Steffen Kaiser wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 12 Nov 2008, Daniel L. Miller wrote:

I don't if this has been discussed before - or already implemented - 
but it would be great if I could define one or more "aliases" for a 
folder.  A particular example might be "Sent" where different 
(badly-written) clients might have hard-coded "Sent" folder locations 
like "Sent", "Sent Items", "Sent Mail", "Mail Delivered", etc. - and 
I'd love to get those consolidated.


It would also be nice to show a different set of folders based on the 
client - or if that's not possible (if the client doesn't identify 
itself as part of the login sequence) then somehow via login name - 
maybe by an extension?  So where my usual login might be 
"[EMAIL PROTECTED]", and that would continue to be supported as the 
default login, I could now have "[EMAIL PROTECTED]", 
"[EMAIL PROTECTED]", etc.?


Because the IMAP client does not identify itself, the server does not 
know, which alias to return to the client. I'm not convinced that the 
extension method you describe will be widely accepted.
"Widely accepted"?  Not sure what you meant - or I guess I didn't 
express myself well.  In my own case, for a particular reason that 
seemed justified at the time, my mail user names are the complete mail 
[EMAIL PROTECTED] (even though I'm really only doing one domain).  My 
intent was that whatever the user name as defined in the database used 
by Dovecot - could then have appended or inserted or whatever some 
identifier flag that Dovecot would use in selecting the folders to share 
with the client.  So I'm asking for a Dovecot-specific modification to 
be made to Dovecot - followed by implementation instructions for the 
clients.  I wasn't aware of any particular limitations on mail usernames 
imposed by clients - but if there are, we should certainly that into 
account.


I definitely would like a solution for this problem as well.

But if the server would return all aliases, most recent clients will 
at least download all headers from all of them (not knowing about that 
they are all equal); if the server would return just one alias, the 
client would assume that its folder has been deleted and redownload 
the sent message as soon as the folder re-appears.


At a very simple level - so it's system wide, not per-user, would 
include a database similar to the following:


Default Extension:  TBird
TBird: -Sent-Items, -Composing, -Posted
Mobile: Sent-Items=Sent, Composing=Drafts
Mutt:  Posted=Sent, Writing=Drafts
PMail:  Flown=Sent, Droppings=Trash

Now when a client connects, Dovecot parses the username.  If no 
extension is present, Dovecot applies the filters defined as the 
default.  Otherwise, based on the extension, perform the appropriate 
folder name conversion.  The first line I showed goes beyond translation 
and would allow for specific folder "hiding" based on the client - 
perhaps if there was a dedicated-purpose folder for a particular client 
that there is no translation for in other clients (perhaps a 
special-function junk folder or something).



--
Daniel

[Dovecot] Remote storage

[Daniel L. Miller]

What options are available for remote storage, i.e. storage devices 
directly mounted on the machine running Dovecot?  I've seen discussions 
of NFS usage - but my experiences with NFS in times past have been 
poor.  CIFS has worked well for me for non-mail use - would this be 
suitable?  Or is there another method I've not considered?

--
Daniel

Re: [Dovecot] Remote storage

[Daniel L. Miller]

Daniel L. Miller wrote:
What options are available for remote storage, i.e. storage devices 
directly mounted on the machine running Dovecot?  I've seen 
discussions of NFS usage - but my experiences with NFS in times past 
have been poor.  CIFS has worked well for me for non-mail use - would 
this be suitable?  Or is there another method I've not considered?
I'm actually trying it - and at first glance it's functional.  I'm using 
the following mount parameters:


-o user=vmail%vmail,rw,serverino,directio

I'm using Dovecot 1.1.11 (from Ubuntu Jaunty).  Also Postfix, LDAP, and 
a few other odds & ends.  While message delivery is working (using 
deliver), I do see the following in my logs:


Mar  1 13:35:09 mailserver postfix/pickup[21464]: 5FAFC2A002: uid=0 
from=
Mar  1 13:35:09 mailserver postfix/cleanup[21526]: 5FAFC2A002: 
message-id=<20090301213509.5fafc2a...@mailserver.amfeslan.local>
Mar  1 13:35:09 mailserver postfix/qmgr[21429]: 5FAFC2A002: 
from=, size=367, nrcpt=1 (queue active)
Mar  1 13:35:09 mailserver dovecot: auth(default): master in: 
user^i1^idmil...@amfes.com^iservice=deliver
Mar  1 13:35:09 mailserver dovecot: auth(default): master out: 
user^i1^idmil...@amfes.com^iuid=5000^igid=8^ihome=/var/mail/amfes.com/dmiller^Imail=dbox:/var/mail/amfes.com/dmiller/Maildir
Mar  1 13:35:10 mailserver deliver(dmil...@amfes.com): 
fdatasync_path(/var/mail/amfes.com/dmiller/Maildir/INBOX/dbox-Mails) 
failed: Invalid argument
Mar  1 13:35:10 mailserver deliver(dmil...@amfes.com): 
msgid=<20090301213509.5fafc2a...@mailserver.amfeslan.local>: saved mail 
to INBOX
Mar  1 13:35:10 mailserver postfix/pipe[21528]: 5FAFC2A002: 
to=, relay=dovecot, delay=1.1, delays=0.07/0.01/0/1, 
dsn=2.0.0, status=sent (delivered via dovecot service)

Mar  1 13:35:10 mailserver postfix/qmgr[21429]: 5FAFC2A002: removed

That error from deliver with "fdatasync_path" bothers me - what does 
that indicate, and where should I go to fix it?  Based on mmap errors, I 
already set "mmap_disable = yes"

--
Daniel

[Dovecot] Deliver error - fdatasync_path

[Daniel L. Miller]

I see the following history in my log during message delivery:

Mar  1 13:35:09 mailserver postfix/pickup[21464]: 5FAFC2A002: uid=0 
from=
Mar  1 13:35:09 mailserver postfix/cleanup[21526]: 5FAFC2A002: 
message-id=<20090301213509.5fafc2a...@mailserver.amfeslan.local>
Mar  1 13:35:09 mailserver postfix/qmgr[21429]: 5FAFC2A002: 
from=, size=367, nrcpt=1 (queue active)
Mar  1 13:35:09 mailserver dovecot: auth(default): master in: 
user^i1^idmil...@amfes.com^iservice=deliver
Mar  1 13:35:09 mailserver dovecot: auth(default): master out: 
user^i1^idmil...@amfes.com^iuid=5000^igid=8^ihome=/var/mail/amfes.com/dmiller^Imail=dbox:/var/mail/amfes.com/dmiller/Maildir 

Mar  1 13:35:10 mailserver deliver(dmil...@amfes.com): 
fdatasync_path(/var/mail/amfes.com/dmiller/Maildir/INBOX/dbox-Mails) 
failed: Invalid argument
Mar  1 13:35:10 mailserver deliver(dmil...@amfes.com): 
msgid=<20090301213509.5fafc2a...@mailserver.amfeslan.local>: saved mail 
to INBOX
Mar  1 13:35:10 mailserver postfix/pipe[21528]: 5FAFC2A002: 
to=, relay=dovecot, delay=1.1, delays=0.07/0.01/0/1, 
dsn=2.0.0, status=sent (delivered via dovecot service)

Mar  1 13:35:10 mailserver postfix/qmgr[21429]: 5FAFC2A002: removed

--
Daniel

Re: [Dovecot] Deliver error - fdatasync_path

[Daniel L. Miller]

Timo Sirainen wrote:

On Mon, 2009-03-02 at 07:31 -0800, Daniel L. Miller wrote:
  
Mar  1 13:35:10 mailserver deliver(dmil...@amfes.com): 
fdatasync_path(/var/mail/amfes.com/dmiller/Maildir/INBOX/dbox-Mails) 
failed: Invalid argument



Some OSes/filesystems don't like that. This should help:
http://hg.dovecot.org/dovecot-1.1/rev/71b7c7f8d7a2

I'm not sure if CIFS has other problems though. And I hope you're not
trying to access the same user's mails in two different servers at the
same time?

BTW. Is it intentional that you're using dbox format under Maildir?
  
2nd question first - I am intentionally storing the mail in the /Maildir 
directory in the user's home directory.  I setup the server originally 
for Maildir format and then thought I'd try out dbox for fun.  Unless 
there's a problem with this...


I am NOT trying to access from multiple servers simultaneously.

1st question...hmm...wow.  Is that a patch you just wrote?!  Cool!

Hmm...since I'm running on a pre-compiled package...is this an error I 
can simply ignore?  Your patch seems to indicate you're just ignoring 
the result of the fsync call.

--
Daniel

Re: [Dovecot] Deliver error - fdatasync_path

[Daniel L. Miller]

Timo Sirainen wrote:

On Mon, 2009-03-02 at 08:59 -0800, Daniel L. Miller wrote:
  

Some OSes/filesystems don't like that. This should help:
http://hg.dovecot.org/dovecot-1.1/rev/71b7c7f8d7a2
  


  

1st question...hmm...wow.  Is that a patch you just wrote?!  Cool!

Hmm...since I'm running on a pre-compiled package...is this an error I 
can simply ignore?  Your patch seems to indicate you're just ignoring 
the result of the fsync call.



With dbox yes, you can just ignore the error. Maildir probably would
give user-visible errors.

  

Why the difference?  I can see a big deal for mbox - but why Maildir?

--
Daniel

Re: [Dovecot] Deliver error - fdatasync_path

[Daniel L. Miller]

Timo Sirainen wrote:

On Thu, 2009-03-05 at 14:42 -0800, Daniel L. Miller wrote:
  

With dbox yes, you can just ignore the error. Maildir probably would
give user-visible errors.

  
  

Why the difference?  I can see a big deal for mbox - but why Maildir?



It's just because of how the code was written. In maildir code if
fdatasync_path() returns -1, the code continues returning -1 which
eventually causes the entire operation to fail and an error be returned
to user.

With dbox the code just didn't bother checking if fdatasync_path()
returned -1 or 0. With mbox it never even calls that function.
  
So if dbox doesn't check - is that a "critical" function (I'm not even 
sure what it does)?  Is there any reason I shouldn't ignore the dbox 
errors and continue trying it for a new server?


--
Daniel

[Dovecot] dbox on CIFS

[Daniel L. Miller]

Ok - at this time I am using the following non-default settings:

Samba server - smb.conf
[Mailstorage]
   ea support = Yes
   use sendfile = Yes
   fake oplocks = Yes
   delete readonly = Yes

Dovecot - dovecot.conf
   mmap_disable = yes
   dotlock_use_excl = yes
   fsync_disable = yes
   lock_method = dotlock

I'm still testing - but at this time I'm seeing no errors.

Two options I can state cause severe problems are the "directio" and 
"setuids" mount options for the cifs client.

--
Daniel

[Dovecot] Upgrade from 1.0 to 1.1, plus migration from Maildir to dbox

[Daniel L. Miller]

Ok - I gave up on trying to get CIFS to work - there's obviously stuff
going on I don't understand.  So - instead of trying to get a virtual
server to work, I next tried a chroot.  That worked...but imapsync
error'ed out trying to handle one of my large mailboxes.

I was trying to setup a "new" server, migrate all my mail over, verify
everything worked - maybe even leave the virtual server operating for a
while.  But the fates are consipiring against me.  So - time to make the
leap.  I want to upgrade a live version 1.0 server to version 1.1 (from
Ubuntu repositories, using version 1.0.10 to 1.1.4).

What special steps do I need to take to have this go smoothly?  At least
I have some semi-tested configs for 1.1 that I can use.  Do I just stop
the server, upgrade, upgrade the configs, start - and pray?  Or is there
another step?

Also, my legacy installation was using maildir - I'd like to try
shifting to dbox.  Do I just need to change the prefix on mail_location
and the conversion will magically happen?
--
Daniel

[Dovecot] Convert from maildir to dbox

[Daniel L. Miller]

Ok...I made the leap.  Compiled a copy of 1.2b4 (very pleasant compile 
BTW - almost no warnings, nice code Timo!)...tweaked a config...had a 
few false starts...


1.2b4 seems to working quite nicely.  Now...for my next magical trick...

I would like to migrate from maildir to dbox.  I've tried using the 
convert plugin, but it kept failing during the conversion.  I should 
mention my paths:


home (old & new) = /var/mail/%d/%n
old mail = maildir:/var/mail/%d/%n
new mail = dbox:/var/mail/%d/%n/dbox

Note that "home" was defined as the same path as "mail" in my old config 
- I'm looking to fix that as well during the migration.


I found entries in my mail log after the convert execution similar to:
Mar 30 19:00:52 bubba dovecot: IMAP(s...@amfes.com): 
rename(/var/mail/amfes.com/sfox/.dovecot.convert.lock/cur, 
/var/mail/amfes.com/sfox/dbox/mailboxes/dovecot/convert/lock/dbox-Mails) 
failed: Not a directory
Mar 30 19:01:41 bubba dovecot: IMAP(s...@amfes.com): Mailbox conversion 
failed for mailbox dovecot.convert.lock


Now that I'm staring at that line...I'm wondering if that "home=mail" 
configuration is part of the problem.


I currently have about a dozen users on this box - but with very large 
mailboxes (I still have archived mailing lists in mine!).  If there's a 
few manual steps I need to perform to "condition" this setup prior to 
conversion please advise.  Should I manually shift the mailboxes into a 
"/var/mail/%d/%n/maildir" folder?

--
Daniel

[Dovecot] Virtual user folder configuration

[Daniel L. Miller]

Since I'm going to make a change to my folder structure, what would be 
recommended so I can avoid the need in the future?


All my mail users are virtual - they have no existence in /etc/passwd, 
nor do they have folders under /home.  At this time, all mail is being 
delivered to maildir:/var/mail/%d/%n - and the home is the same (which 
is becoming more of a problem).


As I see it, I have the choices of:
1. Place mail underneath home, as in /var/mail/%d/%n/mailbox
Probably the simplest, but adds a level to the folder hierarchy.  Does 
that matter in any way?


2. Place home underneath mail, as in /var/mail/%d/%n/home (unless THAT 
poses a problem as well, with a non-mail folder visible to the mail system)
Leaves mail alone - but means the home folder is visible from within the 
mail system.


3. Relocate home totally outside mail, as in /var/mail/home/%d/%n
Would provide isolation - but more difficulty in administration, 
particularly adding/deleting users.


Any suggestions?
--
Daniel

Re: [Dovecot] Virtual user folder configuration

[Daniel L. Miller]

Daniel L. Miller wrote:
Since I'm going to make a change to my folder structure, what would be 
recommended so I can avoid the need in the future?


All my mail users are virtual - they have no existence in /etc/passwd, 
nor do they have folders under /home.  At this time, all mail is being 
delivered to maildir:/var/mail/%d/%n - and the home is the same (which 
is becoming more of a problem).


As I see it, I have the choices of:
1. Place mail underneath home, as in /var/mail/%d/%n/mailbox
Probably the simplest, but adds a level to the folder hierarchy.  Does 
that matter in any way?


2. Place home underneath mail, as in /var/mail/%d/%n/home (unless THAT 
poses a problem as well, with a non-mail folder visible to the mail 
system)
Leaves mail alone - but means the home folder is visible from within 
the mail system.


3. Relocate home totally outside mail, as in /var/mail/home/%d/%n
Would provide isolation - but more difficulty in administration, 
particularly adding/deleting users.


Any suggestions?
Wow.  My list matches almost word for word the wiki entry on virtual 
users.  And since I see the preferred choice is option #1 - guess that's 
what I'll do unless I hear different.

--
Daniel

Re: [Dovecot] dbox - how to awaken the monster

[Daniel L. Miller]

Odhiambo Washington wrote:

I thought so. Now I have seen, read and understood (so far)
http://wiki.dovecot.org/LDA/Exim.
However I do not see any examples for configuring dovecot for dbox. Is it as
easy as s/maildir/dbox/g in the following:

mail_location =
maildir:/var/spool/virtual/%d/%n/Maildir:INDEX=/var/dovecot/%u:CONTROL=/var/dovecot/%u
  
Not quite.  And "configuration" is not "migration" - keep that in mind 
if you're trying to change an existing system.


Given the above, I'd do:
mail_location = dbox:/var/spool/virtual/%d/%n/dbox

Keep in mind dbox is 100% proprietary to Dovecot.  While indexes are 
supplemental - even optional - for the other formats, part of what makes 
dbox special is that the mails are stored in the indexes.  So your 
configuration that has the indexes split off to different folders 
probably isn't a good idea if you're going to dbox.

And suppose I wanted to redeliver mail stored in dbox, using the MTA?? For
example, I have used the following script to redeliver mail stored in
Maildir:
  
I don't think this would be an option anymore.  Maildir stores the raw 
message in a simple file - which makes such manipulation easy.  Dbox 
will store it in a proprietary format.  Unless a utility is provided 
that would accept a dbox message-id and return a raw message (haven't 
seen one advertised) you would have to use some kind of IMAP interface 
to accomplish this.  If such access is important to you - stick with 
maildir.

Ok. Sorry it looks like I have unending questions:(
  

Never apologize for seeking knowledge.
--
Daniel

Re: [Dovecot] dbox - how to awaken the monster

[Daniel L. Miller]

Odhiambo Washington wrote:

One last question: Can dbox co-exist with Maildir?
  
Depends on what you mean by "co-exist".  It is possible to define 
per-user mail storage within the userdb.  It is also possible (I 
believe) to have multiple namespaces, and the mailstorage can be 
different for each namespace (I don't understand namespaces myself yet, 
so can't help much there).

And finally, there was a mention of a script to migrate from Maildir to
dbox. Where is it?
  
I'm trying to use the convert plugin myself.  It seemed very simple to 
implement.  My difficulty is I previously had home=mail - which is a Bad 
Thing.  After I've finished splitting those out, I'll try the covert again.


My suggestion, if you want to try it, is:
1.  Setup a test mailbox, and send a couple of mails to it.
2.  Make sure no users are currently connected.
3.  Stop ALL mail services (including your MTA - otherwise delivery will 
mess things up)

4.  Backup your dovecot.conf file.
5.  Re-configure Dovecot to listen on a non-standard port, like 10143, 
to prevent other clients from re-connecting.

6.  Configure the covert plugin and your target mailstorage
7.  Have a mail client - or even telnet - connect to Dovecot on that new 
port, using the test mailbox - NOT YOUR OWN!!  I recommend telnet - less 
likely to have a problem.

8.  Examine the mailbox folder - did the conversion complete successfully?

If it worked, take a deep breath, cross your fingers...and reconfigure 
dovecot to listen on the standard ports again and allow users to connect.

--
Daniel

Re: [Dovecot] dbox benchmarks

[Daniel L. Miller]

I can see where large mailservers would benefit from significant write 
performance increases - but unless the server is being actively limited 
by the local delivery agent, what other performance benefits does this 
offer?


In particular, is there any increase in READ performance by using dbox?  
Does it reduce memory consumption at all?


--
Daniel

Re: [Dovecot] Adding Sieve Extensions

[Daniel L. Miller]

Timo Sirainen wrote:

On Fri, 2009-04-03 at 12:34 -0700, James Butler wrote:
  

How can I add an extension to Dovecot's Sieve implementation?

I would like to use 'editheader' and 'redirect'.



I'm not really sure what you mean. editheader isn't implemented,
although Konstantin is apparently trying to implement it for
dovecot-libsieve. redirect is implemented and can be used in the same
way as all other sieve implementations:

redirect "em...@example.com";
  
Will Sieve become part of the standard distribution?  Or will remain a 
patch/add-on for the foreseeable future?


--
Daniel