[Dovecot] auth service: out of memory

2012-06-28 Thread Mailing List SVR
Hi, I have some out of memory errors in my logs (file errors.txt attached) I'm using dovecot 2.0.19, I can see some memory leaks fix in hg after the 2.0.19 release but they seem related to imap-login service, I attached my config too, is something wrong there? Should I really increase the li

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Wojciech Puchar
Has anyone tried or benchmarked ZFS, perhaps ZFS+NFS as backing store for yes. long time ago. ZFS isn't useful for anything more than a toy. I/O performance is just bad.

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Wojciech Puchar
The executive summary is something like: when raid5 fails, because at that point you effectively do a raid "scrub" you tend to suddenly notice a bunch of other hidden problems which were lurking and your rebuild fails (this and no raid will protect you from every failure. You have to do backups

Re: [Dovecot] Removing specific entry in user/auth cache

2012-06-28 Thread Timo Sirainen
On 29.6.2012, at 5.18, Daniel Parthey wrote: > wouldn't it be better to use a syntax similar to other doveadm commands, > with labels for all arguments? > > doveadm auth test -u -p [] > doveadm auth cache flush -u [] > doveadm auth cache stats > > This will allow you to syntactically distinguis

Re: [Dovecot] Removing specific entry in user/auth cache

2012-06-28 Thread Daniel Parthey
Timo Sirainen wrote: > On 28.6.2012, at 9.43, Timo Sirainen wrote: > Perhaps for v2.2: > > doveadm auth test [] > doveadm auth cache flush [] > doveadm auth cache stats > > and for v2.1 a bit kludgy way: > > doveadm auth [] > doveadm auth cache flush [] > > so you couldn't test authentication

Re: [Dovecot] Removing specific entry in user/auth cache

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 9.43, Timo Sirainen wrote: > It would be possible to add a doveadm command for this.. I think the > main reason why I already didn't do it last time I was asked this was > because I wanted to use "doveadm auth cache flush" or something similar > as the command, but there already e

Re: [Dovecot] Removing specific entry in user/auth cache

2012-06-28 Thread Daniel Parthey
Angel L. Mateo wrote: > El 27/06/12 14:24, Timo Sirainen escribió: > >On 27.6.2012, at 14.10, Angel L. Mateo wrote: > >>We have dovecot configured with auth cache. > >> Is there any way to remove a specific entry (not all) from this cache? > > Nope. What do you need it for? > Because information fo

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Kelsey Cummings
On 06/28/12 05:56, Ed W wrote: So given the statistics show us that 2 disk failures are much more common than we expect, and that "silent corruption" is likely occurring within (larger) real world file stores, there really aren't many battle tested options that can protect against this - really

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Charles Marcus
On 2012-06-28 4:22 PM, Alex Crow wrote: On 28/06/12 20:28, Charles Marcus wrote: On 2012-06-28 2:04 PM, Gary Mort wrote: That's probably due to the different structures they use. sdbox can safely use either because each email message has a unique filename, and if it exists in both places it

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Ed W
On 28/06/2012 17:54, Charles Marcus wrote: On 2012-06-28 12:20 PM, Ed W wrote: Bad things are going to happen if you loose a complete chunk of your filesystem. I think the current state of the world is that you should assume that realistically you will be looking to your backups if you loose t

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Alex Crow
On 28/06/12 20:28, Charles Marcus wrote: On 2012-06-28 2:04 PM, Gary Mort wrote: That's probably due to the different structures they use. sdbox can safely use either because each email message has a unique filename, and if it exists in both places it doesn't matter. Eh?? Sdbox is like mbox

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Charles Marcus
On 2012-06-28 2:04 PM, Gary Mort wrote: That's probably due to the different structures they use. sdbox can safely use either because each email message has a unique filename, and if it exists in both places it doesn't matter. Eh?? Sdbox is like mbox - one file per mailbox/folder... it is NO

Re: [Dovecot] Setting up mixed mbox and maildir

2012-06-28 Thread Jeff Lacki
Jonathan Ryshpan wrote: > Quite right; this comes from a reading of pages in both wiki1 and wiki2. > I now surmise that this isn't a good idea since wiki1 describes v1.x > and wiki2 describes v2.x, which have different syntaxes (syntaces?). Is > all this correct? I too had a very hard time figu

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 21.04, Gary Mort wrote: > mdbox though is different, multiple messages are stored in a single file. > The index indicates in which file each message is located. When the data > is moved to alt storage, the filename can change in which case the index is > updated. > IE: > Primary/

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Gary Mort
On Thu, Jun 28, 2012 at 1:21 PM, Timo Sirainen wrote: > On 28.6.2012, at 20.14, Timo Sirainen wrote: > > >> "An upshot of the way alternate storage works is that any given storage > >> file (mailboxes//dbox-Mails/u.* (sdbox) or storage/m.* (mdbox)) > can > >> only appear *either* in the primary s

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 20.55, Gary Mort wrote: >> The indexes have to be in primary storage. >> > True, but the data they are based on I'm assuming does not include the full > email message, just a few key pieces: > uniqueid, subject, from, to, etc. > > For an always running server, the indexes are al

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Gary Mort
On Thu, Jun 28, 2012 at 1:14 PM, Timo Sirainen wrote: > On 28.6.2012, at 17.43, Gary Mort wrote: > > First I want to add AWS S3 as a storage option for alternate storage. > > > > Then instead of the above model, the new model would be that email is > > always stored in alternate storage, and may

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 20.21, Timo Sirainen wrote: > On 28.6.2012, at 20.14, Timo Sirainen wrote: > >>> "An upshot of the way alternate storage works is that any given storage >>> file (mailboxes//dbox-Mails/u.* (sdbox) or storage/m.* (mdbox)) can >>> only appear *either* in the primary storage area *o

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 20.14, Timo Sirainen wrote: >> "An upshot of the way alternate storage works is that any given storage >> file (mailboxes//dbox-Mails/u.* (sdbox) or storage/m.* (mdbox)) can >> only appear *either* in the primary storage area *or* the alternate storage >> area but not both — if th

Re: [Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 17.43, Gary Mort wrote: > http://wiki2.dovecot.org/MailboxFormat/dbox > > To make life easy, I'll stick with just single-dbox as a start, however > multi-dbox would be doable. > > With dbox, the only thing that I need to change is the alternate storage > model: > "An upshot of t

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Charles Marcus
On 2012-06-28 12:20 PM, Ed W wrote: Bad things are going to happen if you loose a complete chunk of your filesystem. I think the current state of the world is that you should assume that realistically you will be looking to your backups if you loose the wrong 2 disks in a raid1 or raid10 array.

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Ed W
On 28/06/2012 14:06, Костырев Александр Алексеевич wrote: - RAID1 pairs, plus some kind of intelligent overlay filesystem, eg md-linear+XFS / BTRFS. With the filesystem aware of the underlying arrangement it can theoretically optimise file placement and dramatically increase write speeds for smal

[Dovecot] Integrating Dovecot with Amazon Web Services

2012-06-28 Thread Gary Mort
I did some searching in the mail archives and didn't see any discussion of integration with AWS, so I wanted to through out my thoughts/plans and see if it has been done before. I am setting up my own personal website on EC2 along with an email server, and I really don't like the idea of using the

Re: [Dovecot] started with dovecot sieve

2012-06-28 Thread mailinglist
Am 2012-06-27 20:47, schrieb Daniel Parthey: Rolf wrote: LMTP would be new to me and I fear just other hard-to-understand configuration topics. LMTP (Lightweight Message Transfer Protocol) is really simple, similar to SMTP, but immediately returns a status code which tells whether the delivery

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Костырев Александр Алексеевич
>- RAID1 pairs, plus some kind of intelligent overlay filesystem, eg >md-linear+XFS / BTRFS. With the filesystem aware of the underlying >arrangement it can theoretically optimise file placement and >dramatically increase write speeds for small files in the same manner >that RAID-0 theoreticall

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Ed W
On 28/06/2012 13:46, Wojciech Puchar wrote: (unless we are talking temporary removal and re-insertion?) nope, I'm talking about complete pair's crash when two disks die. I do understand that's the possibility of such outcome (when two disks in the same pair crash) is not high, but when we have

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Wojciech Puchar
(unless we are talking temporary removal and re-insertion?) nope, I'm talking about complete pair's crash when two disks die. I do understand that's the possibility of such outcome (when two disks in the same pair crash) is not high, but when we have 12 or 24 disks in storage... then may 6-12

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Костырев Александр Алексеевич
>Note that you wouldn't get anything back from a similar fail of a RAID10 array >either I wasn't aware of it, that's interesting. >(unless we are talking temporary removal and re-insertion?) nope, I'm talking about complete pair's crash when two disks die. I do understand that's the possibility o

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Wojciech Puchar
Note that you wouldn't get anything back from a similar fail of a RAID10 array either (unless we are talking temporary removal and re-insertion?) use multiple RAID1 arrays, 2 drives each, one filesystem each.

Re: [Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Ed W
On 28/06/2012 13:01, Костырев Александр Алексеевич wrote: Hello! somewhere in maillist I've seen RAID1+md concat+XFS being promoted as mailstorage. Does anybody in here actually use this setup? I've decided to give it a try, but ended up with not being able to recover any data off survived pai

[Dovecot] RAID1+md concat+XFS as mailstorage

2012-06-28 Thread Костырев Александр Алексеевич
Hello! somewhere in maillist I've seen RAID1+md concat+XFS being promoted as mailstorage. Does anybody in here actually use this setup? I've decided to give it a try, but ended up with not being able to recover any data off survived pairs from linear array when _the_first of raid1 pairs got do

Re: [Dovecot] Mail migration to dovecot with doveadm backup

2012-06-28 Thread Reinhard Vicinus
On 28/06/12 09:03, Reinhard Vicinus wrote: and afterwards: /usr/bin/doveadm -o imapc_user=u...@example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -D -v backup -R -f -u u...@example.org imapc: dsync(u...@example.org): Error: Mailb

Re: [Dovecot] indexer-worker

2012-06-28 Thread Wojciech Puchar
29413 root 1 760 22820K 9204K kqread 1 0:17 5.86% indexer-worker It runs as root while not really doing anything, but when it starts accessing users' files it temporarily drops privileges. This is necessary if users have multiple different UIDs. to showed it with root pr

Re: [Dovecot] Default for non-present LDAP attributes?

2012-06-28 Thread Timo Sirainen
On 28.6.2012, at 12.19, Edgar Fuß wrote: >> The "mail" field defaults to mail_location setting. > Ah, yes, thanks. So simple I didn't think of it. > Will it default when the LDAP attribute is not present or will I have to > check the attribute's presence in the LDAP filter? The default settings

Re: [Dovecot] Default for non-present LDAP attributes?

2012-06-28 Thread Edgar Fuß
> The "mail" field defaults to mail_location setting. Ah, yes, thanks. So simple I didn't think of it. Will it default when the LDAP attribute is not present or will I have to check the attribute's presence in the LDAP filter?

Re: [Dovecot] Maildir Seen Flags not heeded when dovecot-shared present

2012-06-28 Thread J E Lyon
Timo & List, Just by way of a follow-up, running tests on a 1.0 installation of Dovecot confirms it. Sure enough, I was still configuring my mail stores based on my outdated understanding and hadn't fully appreciated changes to what dovecot-shared files affect in recent versions. Thanks all,

Re: [Dovecot] last hope... public namespace and directory structure

2012-06-28 Thread Daniel Fischer
Hello Timo, Thanks for your reply. I have the dovewiki a little bit misunderstod. "Public mailboxes are typically mailboxes that are visible to all users or to large user groups. They are created by defining a public namespace, under which all the shared mailboxes are" Daniel -Ursprünglic

Re: [Dovecot] Removing specific entry in user/auth cache

2012-06-28 Thread Angel L. Mateo
El 27/06/12 14:24, Timo Sirainen escribió: On 27.6.2012, at 14.10, Angel L. Mateo wrote: We have dovecot configured with auth cache. Is there any way to remove a specific entry (not all) from this cache? Nope. What do you need it for? Because information for users sometimes changes

Re: [Dovecot] Mail migration to dovecot with doveadm backup

2012-06-28 Thread Reinhard Vicinus
On 28/06/12 08:53, Timo Sirainen wrote: On Wed, 2012-06-27 at 15:10 +0200, Reinhard Vicinus wrote: Hi, if i delete the home directory and all content below an existing account u...@example.org. Then run: /usr/bin/doveadm quota recalc -u u...@example.org Are you sure quota recalc makes a diffe