Dovecot & Thunderbird on shared mbox

[Ruga]

Occasional error from TB when selecting a dovecot's shared mbox:

"Unable to open the summary file for [shared]"
"Perhaps there was an error on disk, or the full path is too long."

The server's disk is clear, and the longest full path is [a-zA-Z\/\.]{46}.

The error occurs occasionally.

Re: Dovecot & Thunderbird on shared mbox

[Ruga]

Dovecot's log (mail_debug=yes) is clear.
The error occurs systematically on a shared mbox with subfolders: selection of 
subfolders is ok, selection of the root shared folder triggers the error. the 
root shared folder does not contain any mail, which is a wanted feature.


On Fri, Aug 19, 2016 at 1:04 PM, Aki Tuomi <'aki.tu...@dovecot.fi'> wrote:
On 19.08.2016 14:00, Ruga wrote:
> Occasional error from TB when selecting a dovecot's shared mbox:
>
> "Unable to open the summary file for [shared]"
> "Perhaps there was an error on disk, or the full path is too long."
>
> The server's disk is clear, and the longest full path is [a-zA-Z/.]{46}.
>
> The error occurs occasionally.

Hi!

Is there anything in your server's logfiles?

Aki

Re: specifying elliptic curve

[Ruga]

+1

I opened a ticket (a while ago) to add manual selection of the curves.


On Mon, Aug 22, 2016 at 6:59 PM, Aki Tuomi <'aki.tu...@dovecot.fi'> wrote:
On 22.08.2016 16:21, İhsan Doğan wrote:
> Hi,
>
> I've noticed that Dovecot is using per default the elliptic curve
> sect571r1. Because not all clients might support sect571r1, I would like
> to set the elliptic curve manually. Is that possible?
>
>
>
> -Ihsan
>

Hi!

If your openssl does not support automatic curve selection (>=1.0.2), we
fall back to using what your private EC key uses, or NIST-P384 as last
resort.

Aki Tuomi
Dovecot oy

Allow selection of safe curves in dovecot

[Ruga]

http://safecurves.cr.yp.to

Re: v2.2.26.0 released

[Ruga]

Hello,

We cannot upgrade from 2.2.24, because we use libressl and the newer
dovecot versions demand openssl v1.1.

Please add the new library requirement to the INSTALL file.

All the best.









 Original Message 
Subject: v2.2.26.0 released
Local Time: 28 October 2016 6:51 PM
UTC Time: 28 October 2016 16:51
From: t...@iki.fi
To: dovecot-n...@dovecot.org, Dovecot Mailing List 

http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz
http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz.sig

v2.2.26 had a couple of nasty bugs left in it, so here's a fixup release. The 
version number is also a little bit weird, but had to be done this way 
(although 2.2.26.0.1 could have been another possibility).

- Fixed some compiling issues.
- auth: Fixed assert-crash when using NTLM or SKEY mechanisms and
multiple passdbs.
- auth: Fixed crash when exporting to auth-worker passdb extra fields
that had empty values.
- dsync: Fixed assert-crash in dsync_brain_sync_mailbox_deinit

Re: v2.2.26.0 released

[Ruga]

dovecot 2.2.26.0 uses the following functions, which are not available on 
libressl 2.4.3:

HMAC_CTX_new
HMAC_CTX_free
EVP_PKEY_get0_EC_KEY
EVP_PKEY_get0_RSA
OBJ_length
EVP_MD_CTX_new
EVP_MD_CTX_free

The result of calling a non-existent function is a runtime error,
and we do not want that on production servers.







There are additional problems. I recommend compiling with clang-llvm 3.9.0
to see them all.







 Original Message 
Subject: Re: v2.2.26.0 released
Local Time: 1 November 2016 7:30 PM
UTC Time: 1 November 2016 18:30
From: aki.tu...@dovecot.fi
To: Dovecot Mailing List , Ruga 

OpenSSL v1.0.1 is enough.

Aki

> On November 1, 2016 at 7:46 PM Ruga  wrote:
>
>
> Hello,
>
> We cannot upgrade from 2.2.24, because we use libressl and the newer
> dovecot versions demand openssl v1.1.
>
> Please add the new library requirement to the INSTALL file.
>
> All the best.
>
>
>
>
>
>
>
>
>
>  Original Message 
> Subject: v2.2.26.0 released
> Local Time: 28 October 2016 6:51 PM
> UTC Time: 28 October 2016 16:51
> From: t...@iki.fi
> To: dovecot-n...@dovecot.org, Dovecot Mailing List 
>
> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz
> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz.sig
>
> v2.2.26 had a couple of nasty bugs left in it, so here's a fixup release. The 
> version number is also a little bit weird, but had to be done this way 
> (although 2.2.26.0.1 could have been another possibility).
>
> - Fixed some compiling issues.
> - auth: Fixed assert-crash when using NTLM or SKEY mechanisms and
> multiple passdbs.
> - auth: Fixed crash when exporting to auth-worker passdb extra fields
> that had empty values.
> - dsync: Fixed assert-crash in dsync_brain_sync_mailbox_deinit

Re: v2.2.26.0 released

[Ruga]

libressl is a leaner and safer openssl

Sent from ProtonMail Mobile


On Wed, Nov 2, 2016 at 12:39 PM, Michael A. Peters <'mpet...@domblogger.net'> 
wrote:
IMHO it would be acceptable to have a LibreSSL patch that is maintained
by the people who want it.

It's free software, and that kind of is the point of Open Source.

On 11/02/2016 04:36 AM, Michael A. Peters wrote:
> They have stated they are going to remain API compatible with 1.0.1h (or
> g, forget which they forked) - their new stuff is outside of libcrypto.
>
> On 11/02/2016 04:25 AM, Aki Tuomi wrote:
>> It does work today, I am just bit worried that it will keep on breaking
>> with libressl as they evolve their API. I would personally like to avoid
>> more ifdef hell if possible...
>>
>> Aki
>>
>>
>> On 02.11.2016 13:22, Michael A. Peters wrote:
>>> Standard way to fix it (on the LibreSSL page) is to check for
>>> LIBRESSL_VERSION_NUMBER - e.g. the patch attached which I think
>>> catches them all where needed. Note the word think.
>>>
>>> It certainly appears to be working anyway with it.
>>>
>>> On 11/02/2016 04:07 AM, Aki Tuomi wrote:
>>>> After doing some testing by myself, I noticed that libressl, for some
>>>> unknown reason, defines
>>>>
>>>> #define OPENSSL_VERSION_NUMBER 0x2000L
>>>>
>>>> No idea why they decided to advertise that they are OpenSSL v2.0.0. A
>>>> local fix, if you need one, is to use
>>>>
>>>> #if OPENSSL_VERSION_NUMBER == 0x2000L
>>>> #define OPENSSL_VERSION_NUMBER 0x1000100L
>>>> #endif
>>>>
>>>> in dcrypt-openssl.c after includes.
>>>>
>>>> Aki
>>>>
>>>>
>>>> On 02.11.2016 12:39, Aki Tuomi wrote:
>>>>> Hi!
>>>>>
>>>>> Those are used if
>>>>>
>>>>> #if OPENSSL_VERSION_NUMBER >= 0x1010L
>>>>>
>>>>> So (your) libressl is providing this define. We compile our code using
>>>>> GCC and CLANG regularly, with OpenSSL v1.0.x which is the currently
>>>>> officially supported one.
>>>>>
>>>>> Aki
>>>>>
>>>>>
>>>>> On 02.11.2016 12:34, Ruga wrote:
>>>>>> dovecot 2.2.26.0 uses the following functions, which are not
>>>>>> available on libressl 2.4.3:
>>>>>>
>>>>>> HMAC_CTX_new
>>>>>> HMAC_CTX_free
>>>>>> EVP_PKEY_get0_EC_KEY
>>>>>> EVP_PKEY_get0_RSA
>>>>>> OBJ_length
>>>>>> EVP_MD_CTX_new
>>>>>> EVP_MD_CTX_free
>>>>>>
>>>>>> The result of calling a non-existent function is a runtime error,
>>>>>> and we do not want that on production servers.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> There are additional problems. I recommend compiling with clang-llvm
>>>>>> 3.9.0
>>>>>> to see them all.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  Original Message 
>>>>>> Subject: Re: v2.2.26.0 released
>>>>>> Local Time: 1 November 2016 7:30 PM
>>>>>> UTC Time: 1 November 2016 18:30
>>>>>> From: aki.tu...@dovecot.fi
>>>>>> To: Dovecot Mailing List , Ruga
>>>>>> 
>>>>>>
>>>>>> OpenSSL v1.0.1 is enough.
>>>>>>
>>>>>> Aki
>>>>>>
>>>>>>> On November 1, 2016 at 7:46 PM Ruga  wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> We cannot upgrade from 2.2.24, because we use libressl and the newer
>>>>>>> dovecot versions demand openssl v1.1.
>>>>>>>
>>>>>>> Please add the new library requirement to the INSTALL file.
>>>>>>>
>>>>>>> All the best.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  Original Message 
>>>>>>> Subject: v2.2.26.0 released
>>>>>>> Local Time: 28 October 2016 6:51 PM
>>>>>>> UTC Time: 28 October 2016 16:51
>>>>>>> From: t...@iki.fi
>>>>>>> To: dovecot-n...@dovecot.org, Dovecot Mailing List
>>>>>>> 
>>>>>>>
>>>>>>> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz
>>>>>>> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz.sig
>>>>>>>
>>>>>>> v2.2.26 had a couple of nasty bugs left in it, so here's a fixup
>>>>>>> release. The version number is also a little bit weird, but had to
>>>>>>> be done this way (although 2.2.26.0.1 could have been another
>>>>>>> possibility).
>>>>>>>
>>>>>>> - Fixed some compiling issues.
>>>>>>> - auth: Fixed assert-crash when using NTLM or SKEY mechanisms and
>>>>>>> multiple passdbs.
>>>>>>> - auth: Fixed crash when exporting to auth-worker passdb extra
>>>>>>> fields
>>>>>>> that had empty values.
>>>>>>> - dsync: Fixed assert-crash in dsync_brain_sync_mailbox_deinit
>>>

Re: v2.2.26.0 released

[Ruga]

Thank you.

Sent from ProtonMail Mobile


On Wed, Nov 2, 2016 at 12:22 PM, Michael A. Peters <'mpet...@domblogger.net'> 
wrote:
Standard way to fix it (on the LibreSSL page) is to check for
LIBRESSL_VERSION_NUMBER - e.g. the patch attached which I think catches
them all where needed. Note the word think.

It certainly appears to be working anyway with it.

On 11/02/2016 04:07 AM, Aki Tuomi wrote:
> After doing some testing by myself, I noticed that libressl, for some
> unknown reason, defines
>
> #define OPENSSL_VERSION_NUMBER 0x2000L
>
> No idea why they decided to advertise that they are OpenSSL v2.0.0. A
> local fix, if you need one, is to use
>
> #if OPENSSL_VERSION_NUMBER == 0x2000L
> #define OPENSSL_VERSION_NUMBER 0x1000100L
> #endif
>
> in dcrypt-openssl.c after includes.
>
> Aki
>
>
> On 02.11.2016 12:39, Aki Tuomi wrote:
>> Hi!
>>
>> Those are used if
>>
>> #if OPENSSL_VERSION_NUMBER >= 0x1010L
>>
>> So (your) libressl is providing this define. We compile our code using
>> GCC and CLANG regularly, with OpenSSL v1.0.x which is the currently
>> officially supported one.
>>
>> Aki
>>
>>
>> On 02.11.2016 12:34, Ruga wrote:
>>> dovecot 2.2.26.0 uses the following functions, which are not
>>> available on libressl 2.4.3:
>>>
>>> HMAC_CTX_new
>>> HMAC_CTX_free
>>> EVP_PKEY_get0_EC_KEY
>>> EVP_PKEY_get0_RSA
>>> OBJ_length
>>> EVP_MD_CTX_new
>>> EVP_MD_CTX_free
>>>
>>> The result of calling a non-existent function is a runtime error,
>>> and we do not want that on production servers.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> There are additional problems. I recommend compiling with clang-llvm
>>> 3.9.0
>>> to see them all.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  Original Message 
>>> Subject: Re: v2.2.26.0 released
>>> Local Time: 1 November 2016 7:30 PM
>>> UTC Time: 1 November 2016 18:30
>>> From: aki.tu...@dovecot.fi
>>> To: Dovecot Mailing List , Ruga
>>> 
>>>
>>> OpenSSL v1.0.1 is enough.
>>>
>>> Aki
>>>
>>>> On November 1, 2016 at 7:46 PM Ruga  wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>> We cannot upgrade from 2.2.24, because we use libressl and the newer
>>>> dovecot versions demand openssl v1.1.
>>>>
>>>> Please add the new library requirement to the INSTALL file.
>>>>
>>>> All the best.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  Original Message 
>>>> Subject: v2.2.26.0 released
>>>> Local Time: 28 October 2016 6:51 PM
>>>> UTC Time: 28 October 2016 16:51
>>>> From: t...@iki.fi
>>>> To: dovecot-n...@dovecot.org, Dovecot Mailing List
>>>> 
>>>>
>>>> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz
>>>> http://dovecot.org/releases/2.2/dovecot-2.2.26.0.tar.gz.sig
>>>>
>>>> v2.2.26 had a couple of nasty bugs left in it, so here's a fixup
>>>> release. The version number is also a little bit weird, but had to
>>>> be done this way (although 2.2.26.0.1 could have been another
>>>> possibility).
>>>>
>>>> - Fixed some compiling issues.
>>>> - auth: Fixed assert-crash when using NTLM or SKEY mechanisms and
>>>> multiple passdbs.
>>>> - auth: Fixed crash when exporting to auth-worker passdb extra fields
>>>> that had empty values.
>>>> - dsync: Fixed assert-crash in dsync_brain_sync_mailbox_deinit

Existing shared mailbox ... does not exist

[Ruga]

Hello,

Dovecot 2.2.26.0
Thunderbird 45.4.0 ESR

Scenario:
We have a shared mailbox with e-mails in its subdirectories.
There are no e-mails in the shared mailbox itself.

Example:
/Shared <--- empty
/Shared/call_for_papers <-- empty
/Shared/call_for_papers/AAMAS <-- populated
...more of the same...

Problem:
When selecting /Shared, Thunderbird chirps this message:

"The current operation on 'Shared' did not succeed. The mail server for account
@example.com responded: Mailbox doesn't exist: Shared/Shared (0.000 + 0.000 
secs)."

Re: Good email client to use with Dovecot?

[Ruga]

TB is the closest thing to a functional Outlook replacement for office 
deployment. I have seen UN staff so distressed by IBM Lotus Notes that would 
have given someone else's left arm to have TB instead. I have mutt as a 
lifeboat, but is not good enough for daily use in office.

On Mozilla not willing to spend on TB, I think it is a very good thing. I would 
rather see TB on github with a donation button than see it crippled with 
Firefox-like spyware.


On Sat, Nov 19, 2016 at 4:34 PM, Michael Felt <'mich...@felt.demon.nl'> wrote:
These discussions are at a very early stage. Finding the right solution
requires some effort. This is Mozilla focusing on a more forward looking
path, one aimed at longer term stability rather than continuing the
status quo.

Re: Good email client to use with Dovecot?

[Ruga]

Anyone using the default (TB) approach /Archive/-MM will eventually meet 
the problem of having too many folders and a slow service. The alternative, for 
the user, is to write their own filters. From the server side, it would be 
useful if dovecot would filter certain e-mails automatically. For example, it 
could move any e-mail from "dovecot@dovecot.org" into 
/Lists/dovecot@dovecot_org/.


On Tue, Nov 22, 2016 at 6:34 PM, Tanstaafl <'tansta...@libertytrek.org'> wrote:
On 11/22/2016 10:35 AM, @lbutlr  wrote:
> On Nov 22, 2016, at 7:48 AM, Tanstaafl  wrote:
>> I'm trying for the life of me to see a use case for anywhere close to
>> 1,000 folders, and am failing. That would be a major problem just from
>> the human side. How do you find anything?

> I can see it, though I think it’s excessive.
>
> List Mail
> Dovecot
> 2011-06
> 2011-07
> 2011-08



Like I said, I simply don't see it. There is simply zero reason to split
things up like this. It is trivial to limit your view to just what you
want with filters or just plain sorting (by date in this case).

Just not enough bang for the buck. Again, this is jut my opinion, if
this makes someone else feel better/more organized or whatever,
obviously they are free to have as many folders as they want.

Re: CPU for Dovecot

[Ruga]

> Xeon E5-2620v4 - 2,1GHz@8,0GT 20MB cache, 8core, HT, 85W, LGA2011


passmark: 16825
cost: 417$

> Xeon E5-2623v4 - 2,6GHz@8,0GT 10MB cache, 4core, HT, 85W, LGA2011

passmark: 8302
cost: 444$


I would get the first one.

On Fri, Nov 25, 2016 at 2:29 PM, Miloslav Hůla <'miloslav.h...@gmail.com'> 
wrote:
Hi,

we are planning to change hardware for our standalone Dovecot instance
handling ~5800 IMAP users with 1TB mailboxes on local RAID. Is there
some recommendation about CPU?

We can choose from:
- Intel Xeon E5-2620v4 - 2,1GHz@8,0GT 20MB cache, 8core, HT, 85W, LGA2011
- Intel Xeon E5-2623v4 - 2,6GHz@8,0GT 10MB cache, 4core, HT, 85W, LGA2011

The difference is about more cores vs. hi frequency.

Thank you, Miloslav

Re: JMAP support in Dovecot

[Ruga]

I am happy with IMAPS. Most of JMAP can be implemented as IMAPS incremental 
revisions.

Re: v2.2.27 released

[Ruga]

var-expand.c:686:17: warning: passing an object that undergoes default argument 
promotion to 'va_start' has undefined behavior [-Wvarargs]


va_start(args, key2);


^


var-expand.c:673:58: note: parameter of type 'char' is declared here


var_expand_table_build(char key, const char *value, char key2, ...)


^












 Original Message 
Subject: v2.2.27 released
Local Time: 3 December 2016 6:48 PM
UTC Time: 3 December 2016 17:48
From: t...@iki.fi
To: dovecot-n...@dovecot.org, Dovecot Mailing List 

https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz.sig

Note that the download URLs are now https with a certificate from Let's Encrypt.

* dovecot.list.index.log rotation sizes/times were changed so that
the .log file stays smaller and .log.2 is deleted sooner.

+ Added mail_crypt plugin that allows encryption of stored emails.
See http://wiki2.dovecot.org/Plugins/MailCrypt
+ stats: Global stats can be sent to Carbon server by setting
stats_carbon_server=ip:port
+ imap/pop3 proxy: If passdb returns proxy_not_trusted, don't send
ID/XCLIENT
+ Added generic hash modifier for %variables:
%{;rounds=,truncate=,salt=s>:field}
Hash algorithm is any of the supported ones, e.g. md5, sha1, sha256.
Also "pkcs5" is supported using SHA256. For example: %{sha256:user}
or %{md5;truncate=32:user}.
+ Added support for SHA3-256 and SHA3-512 hashes.
+ config: Support DNS wildcards in local_name, e.g.
local_name *.example.com { .. } matches anything.example.com, but
not multiple.anything.example.com.
+ config: Support multiple names in local_name, e.g.
local_name "1.example.com 2.example.com" { .. }
- Fixed crash in auth process when auth-policy was configured and
authentication was aborted/failed without a username set.
- director: If two users had different tags but the same hash,
the users may have been redirected to the wrong tag's hosts.
- Index files may have been thought incorrectly lost, causing
"Missing middle file seq=.." to be logged and index rebuild.
This happened more easily with IMAP hibernation enabled.
- Various fixes to restoring state correctly in un-hibernation.
- dovecot.index files were commonly 4 bytes per email too large. This
is because 3 bytes per email were being wasted that could have been
used for IMAP keywords.
- Various fixes to handle dovecot.list.index corruption better.
- lib-fts: Fixed assert-crash in address tokenizer with specific input.
- Fixed assert-crash in HTML to text parsing with specific input
(e.g. for FTS indexing or snippet generation)
- doveadm sync -1: Fixed handling mailbox GUID conflicts.
- sdbox, mdbox: Perform full index rebuild if corruption is detected
inside lib-index, which runs index fsck.
- quota: Don't skip quota checks when moving mails between different
quota roots.
- search: Multiple sequence sets or UID sets in search parameters
weren't handled correctly. They were incorrectly merged together.

Re: v2.2.27 released --- libressl

[Ruga]

>openssl version
Libressl 2.4.4

Patch for dovecot:

perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER < 0x1010L\s*)$/$1 || defined 
(LIBRESSL_VERSION_NUMBER)/' ./src/lib-dcrypt/dcrypt-openssl.c;
perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER < 0x1010L\s*)$/$1 || defined 
(LIBRESSL_VERSION_NUMBER)/' ./src/lib-ssl-iostream/dovecot-openssl-common.c;
perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER >= 0x1010L\s*)$/$1 && !defined 
(LIBRESSL_VERSION_NUMBER)/' ./src/lib-dcrypt/dcrypt-openssl.c;
perl -i -ple 's/^(#if OPENSSL_VERSION_NUMBER >= 0x1010L\s*)$/$1 && !defined 
(LIBRESSL_VERSION_NUMBER)/' ./src/lib-ssl-iostream/dovecot-openssl-common.c;

Both configuration and compilation are OK.

The test fails as follows:
[...]

test_load_v1_public_key .. : ok


Panic: file randgen.c: line 21 (random_fill): assertion failed: (init_refcount 
> 0)


Error: Raw backtrace: 2 libdcrypt_openssl.so 0x000103413d24 
default_fatal_finish + 36 -> 3 libdcrypt_openssl.so 0x000103413afd 
default_fatal_handler + 61 -> 4 libdcrypt_openssl.so 0x000103414069 i_panic 
+ 169 -> 5 libdcrypt_openssl.so 0x00010344110c random_fill + 220 -> 6 
libdcrypt_openssl.so 0x00010340a63d dcrypt_openssl_store_private_key + 1037 
-> 7 test-crypto 0x000103387f54 test_load_v2_key + 580 -> 8 test-crypto 
0x00010338990e test_run + 142 -> 9 test-crypto 0x000103386921 main + 81 
-> 10 libdyld.dylib 0x7fff9da95255 start + 1


/bin/sh: line 1: 56954 Abort trap: 6 ./$bin


make[2]: *** [check-test] Error 1


make[1]: *** [check-recursive] Error 1


make: *** [check-recursive] Error 1












 Original Message 
Subject: v2.2.27 released
Local Time: 3 December 2016 6:48 PM
UTC Time: 3 December 2016 17:48
From: t...@iki.fi
To: dovecot-n...@dovecot.org, Dovecot Mailing List 

https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.27.tar.gz.sig

Note that the download URLs are now https with a certificate from Let's Encrypt.

* dovecot.list.index.log rotation sizes/times were changed so that
the .log file stays smaller and .log.2 is deleted sooner.

+ Added mail_crypt plugin that allows encryption of stored emails.
See http://wiki2.dovecot.org/Plugins/MailCrypt
+ stats: Global stats can be sent to Carbon server by setting
stats_carbon_server=ip:port
+ imap/pop3 proxy: If passdb returns proxy_not_trusted, don't send
ID/XCLIENT
+ Added generic hash modifier for %variables:
%{;rounds=,truncate=,salt=s>:field}
Hash algorithm is any of the supported ones, e.g. md5, sha1, sha256.
Also "pkcs5" is supported using SHA256. For example: %{sha256:user}
or %{md5;truncate=32:user}.
+ Added support for SHA3-256 and SHA3-512 hashes.
+ config: Support DNS wildcards in local_name, e.g.
local_name *.example.com { .. } matches anything.example.com, but
not multiple.anything.example.com.
+ config: Support multiple names in local_name, e.g.
local_name "1.example.com 2.example.com" { .. }
- Fixed crash in auth process when auth-policy was configured and
authentication was aborted/failed without a username set.
- director: If two users had different tags but the same hash,
the users may have been redirected to the wrong tag's hosts.
- Index files may have been thought incorrectly lost, causing
"Missing middle file seq=.." to be logged and index rebuild.
This happened more easily with IMAP hibernation enabled.
- Various fixes to restoring state correctly in un-hibernation.
- dovecot.index files were commonly 4 bytes per email too large. This
is because 3 bytes per email were being wasted that could have been
used for IMAP keywords.
- Various fixes to handle dovecot.list.index corruption better.
- lib-fts: Fixed assert-crash in address tokenizer with specific input.
- Fixed assert-crash in HTML to text parsing with specific input
(e.g. for FTS indexing or snippet generation)
- doveadm sync -1: Fixed handling mailbox GUID conflicts.
- sdbox, mdbox: Perform full index rebuild if corruption is detected
inside lib-index, which runs index fsck.
- quota: Don't skip quota checks when moving mails between different
quota roots.
- search: Multiple sequence sets or UID sets in search parameters
weren't handled correctly. They were incorrectly merged together.

Re: v2.2.27 released --- libressl

[Ruga]

Results from the application of the following patch from Aki.

perl -i -ple 's|^(\s*#include \s*)$|$1\n\t#if 
OPENSSL_VERSION_NUMBER == 0x2000L\n\t#define OPENSSL_VERSION_NUMBER 
0x10001000L\n\t#endif|' configure.ac;

compiler-stderr.log
Description: Binary data

Re: Existing shared mailbox ... does not exist

[Ruga]

bump


On Tue, Nov 8, 2016 at 5:49 PM, Ruga <'r...@protonmail.com'> wrote:

Hello,

Dovecot 2.2.26.0
Thunderbird 45.4.0 ESR

Scenario:
We have a shared mailbox with e-mails in its subdirectories.
There are no e-mails in the shared mailbox itself.

Example:
/Shared <--- empty
/Shared/call_for_papers <-- empty
/Shared/call_for_papers/AAMAS <-- populated
...more of the same...

Problem:
When selecting /Shared, Thunderbird chirps this message:

"The current operation on 'Shared' did not succeed. The mail server for account
@example.com responded: Mailbox doesn't exist: Shared/Shared (0.000 + 0.000 
secs)."

Re: Existing shared mailbox ... does not exist

[Ruga]

# 2.2.26.0 (23d1de6): /etc/dovecot/dovecot.conf
auth_default_realm = example.com
auth_mechanisms = cram-md5
auth_realms = example.com
auth_socket_path = /var/run/dovecot/auth-userdb
auth_verbose = yes
auth_winbind_helper_path =
default_client_limit = 256
default_internal_user = _dovecot
default_login_user = _dovenull
first_valid_uid = 458
mail_access_groups = vmail
mail_cache_min_mail_count = 25
mail_gid = vmail
mail_location = maildir:/var/vmail/%d/%n
mail_plugins = " acl quota"
mail_privileged_group = vmail
mail_uid = vmail
mailbox_list_index = yes
namespace {
hidden = no
inbox = no
list = yes
location = maildir:/var/vmail/%d/shared:INDEXPVT=/var/vmail/%d/%n/shared
mailbox Junk {
auto = create
comment = Shared junk folder
special_use = \Junk
}
mailbox info {
auto = subscribe
special_use =
}
mailbox jobs {
auto = create
special_use =
}
mailbox news {
auto = create
special_use =
}
prefix = Shared/
separator = /
subscriptions = no
type = public
}
namespace inbox {
hidden = no
inbox = yes
list = yes
location =
mailbox Archive {
special_use = \Archive
}
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
comment = SPAM
special_use = \Junk
}
mailbox Notes {
special_use =
}
mailbox Sent {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
mailbox virtual/Flagged {
comment = All my flagged messages
special_use = \Flagged
}
prefix =
separator = /
subscriptions = yes
type = private
}
passdb {
args = scheme=CRAM-MD5 username_format=%n@%d /domains/%d/etc/mail/passdb
driver = passwd-file
}
plugin {
acl = vfile:/etc/dovecot/dovecot-acl.conf:cache_secs=300
acl_shared_dict = file:/var/vmail/%d/shared-mailboxes.db
quota = maildir:User quota:ns=
quota2 = maildir:Shared quota:ns=Shared/
quota_exceeded_message = Quota exceeded.
quota_rule = *:storage=0
quota_rule2 = Junk:storage=+100M
quota_rule3 = Trash:storage=+100M
}
protocols = imap
quota_full_tempfail = yes
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 300
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 0
}
inet_listener imaps {
port = 993
ssl = yes
}
service_count = 1
}
userdb {
args = uid=vmail gid=vmail home=/var/vmail/%d/%n
driver = static
override_fields = home=/var/vmail/%d/%n mail=maildir:/var/vmail/%d/%n
}
protocol imap {
mail_plugins = " acl quota imap_acl imap_quota"
}

Re: Existing shared mailbox ... does not exist

[Ruga]

From dovecot's own log, a long stream of errors:

[...] Error: 
/var/vmail/[example.com/username/shared/.foldername](http://example.com/isername/shared/.foldername)
 com/dovecot.index.pvt reset, view is now inconsistent

Re: Broken OS after upgrade

[Ruga]

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_types.h.html#tag_13_67


On Wed, Jan 11, 2017 at 6:01 PM, Odhiambo Washington <'odhia...@gmail.com'> 
wrote:
Hi,

I have broken my OS (FreeBSD) after upgrade, making me unable to compile
dovecot the usual way so I am seeking a 3rd eye.

The config.log is at : http://bit.ly/2jE5djl

Hoping someone can help me figure out what is broken.

During configure, it fails with:

checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... unknown
checking for _LARGE_FILES value needed for large files... unknown
checking for _Bool... yes
checking for uoff_t... no
checking type of off_t... unknown
configure: error: Unsupported off_t type




--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."

Re: Dovecot source code audit

[Ruga]

Congratulations.


On Fri, Jan 13, 2017 at 6:17 PM, Timo Sirainen <'t...@iki.fi'> wrote:
Mozilla sponsored source code audit for Dovecot. So thanks to them we have our 
first public code audit: 
https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#dovecot

Dates: October 2016 - January 2017

dovecot is a POP and IMAP mailserver; it is used in 68% of IMAP server 
deployments worldwide. The audit was performed by Cure53.

The team found the following problems:

• 3 Low

The Cure53 team were extremely impressed with the quality of the dovecot code. 
They wrote: "Despite much effort and thoroughly all-encompassing approach, the 
Cure53 testers only managed to assert the excellent security-standing of 
Dovecot. More specifically, only three minor security issues have been found in 
the codebase, thus translating to an exceptionally good outcome for Dovecot, 
and a true testament to the fact that keeping security promises is at the core 
of the Dovecot development and operations."

Re: Dovecot source code audit

[Ruga]

"used in 68% of IMAP server deployments worldwide"...

... this means that hackers have a new target to prove themselves, and to prove 
Cure53 is less than we think they are. We ought to brace for the storm ahead.


On Fri, Jan 13, 2017 at 6:17 PM, Timo Sirainen <'t...@iki.fi'> wrote:
Mozilla sponsored source code audit for Dovecot. So thanks to them we have our 
first public code audit: 
https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#dovecot

Dates: October 2016 - January 2017

dovecot is a POP and IMAP mailserver; it is used in 68% of IMAP server 
deployments worldwide. The audit was performed by Cure53.

The team found the following problems:

• 3 Low

The Cure53 team were extremely impressed with the quality of the dovecot code. 
They wrote: "Despite much effort and thoroughly all-encompassing approach, the 
Cure53 testers only managed to assert the excellent security-standing of 
Dovecot. More specifically, only three minor security issues have been found in 
the codebase, thus translating to an exceptionally good outcome for Dovecot, 
and a true testament to the fact that keeping security promises is at the core 
of the Dovecot development and operations."

SNI with mixed certs

[Ruga]

Dovecot SNI is failing hard today. Server with n domains, each with a startssl 
certificate of its own, all certificates expired this morning. Decision: move 
to Letsencrypt. Firsr certificate issued and installed. Other domains in the 
pipeline. Dovecot server rebooted. Expected result: one domain returning the 
new cert, and the n-1 domains returning the expiration notification. Actual 
result: the domain with LE is returning startssl expired notifications. Manual 
check of the key and pem files is OK...

Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub)

[Ruga]

https://github.com/libressl-portable/portable/issues/278

Re: Scaling to 10 Million IMAP sessions on a single server

[Ruga]

A more efficient algorithm would reduce computational complexity, and the need 
for expensive power-hungry CPUs.

Sent from ProtonMail Mobile


On Wed, Feb 22, 2017 at 5:12 AM, Christian Balzer <'ch...@gol.com'> wrote:
On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote:

> I just read this blog: 
> https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/
>  
> 
>  about scaling to 12 Million Concurrent Connections on a single server and it 
> got me thinking.
>

While that's a nice article, nothing in it was news to me or particular
complex when one does large scale stuff, like Ceph for example.

> Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions 
> on a single server?
>
I'm sure Timo's answer will (or would, if he could be bothered) be along
the lines of:
"Sure, if you give me all your gold and then some for a complete rewrite
of, well, everything".

What you're missing and what the bad idea here is that as mentioned
before scale-up only goes so far.
I was feeling that my goal of 500k users/sessions in 2-node active/active
cluster was quite ambitious and currently I'm looking at 200k sessions as
something achievable with the current Dovecot and other limitations.

But even if you were to implement something that can handle 1 million or
more sessions per server, would you want to?
As in, if that server goes down, the resulting packet, authentication
storm will be huge and most like result in a proverbial shit storm later.
Having more than 10% or so of your customers on one machine and thus
involved in an outage that you KNOW will hit you eventually strikes me as
a bad idea.

I'm not sure how the design below meshes with Timo's lofty goals and
standards when it comes to security as well.

And a push with the right people (clients) to support IMAP NOTIFY would of
course reduce the number of sessions significantly.

Finally, Dovecot in proxy mode already scales quite well.

Christian

> I think the current implementation of having a separate process manage each 
> active IMAP session (w/ the possibility of moving idling sessions to a single 
> hibernate process) will never be able to deploy a single server managing 10 
> Million IMAP sessions.
>
> But, would it be possible to implement a new IMAP server plugin that uses a 
> fixed configurable pool of "worker" processes, much like NGINX or PHP-FPM 
> does. These servers can probably scale to 10 Million TCP connections, if the 
> server is carefully tuned and has enough cores/memory to support that many 
> active sessions.
>
> I’m thinking that the new IMAP server could use some external database (e.g., 
> Redis or Memcached) to save all the sessions state and have the "worker" 
> processes poll the TCP sockets for new IMAP commands to process (fetching the 
> session state from the external database when it has a command that is 
> waiting on a response). The Dovecot IMAP proxies could even queue incoming 
> commands to proxy many incoming requests to a smaller number of backend 
> connections (like ProxySQL does for MySQL requests). That might allow each 
> Dovecot proxy to support 10 Million IMAP sessions and a single backend could 
> support multiple front end Dovecot proxies (to scale to 100 Million 
> concurrent IMAP connections using 10 proxies for 100 Million connections and 
> 1 backend server for 10 Million connections).
>
> Of course, the backend server may need to be beefy and have very fast NVMe 
> SSDs for local storage, but changing the IMAP server to manage a pool of 
> workers instead of requiring a process per active session, would allow bigger 
> scale up and could save large sites a lot of money.
>
> Is this a good idea? Or, am I missing something?
>
> Kevin


--
Christian Balzer Network/Systems Engineer
ch...@gol.com Global OnLine Japan/Rakuten Communications
http://www.gol.com/

Re: Scaling to 10 Million IMAP sessions on a single server

[Ruga]

Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box:
http://www.isode.com/whitepapers/mbox-benchmark.html

Re: Scaling to 10 Million IMAP sessions on a single server

[Ruga]

Yes, and they (isode) still use it as marketing evidence. The benchmarking tool 
project also seems out of maintenance.

Sent from ProtonMail Mobile


On Thu, Feb 23, 2017 at 8:52 AM, M. Balridge <'dove...@r.paypc.com'> wrote:
Quoting Ruga :

> Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box:
> http://www.isode.com/whitepapers/mbox-benchmark.html

Wow. That comparison is only 11.5 years old.

The "default" file system of reiserfs and gcc-3.3 were dead giveaways.

I suspect Dovecot's changed a tad since that test.

=R=

v2.2.28: patches (to use libressl 2.4.5) and test error (strftime)

[Ruga]

t_strftime and variants now .. : ok


test-time-util.c:123: Assert failed: strcmp(t_strftime(RFC2822_FMT, 
gmtime(&ts)), exp) == 0


test-time-util.c:124: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) 
== 0


t_strftime and variants fixed timestamp .. : FAILED


timings 0  : ok

configure.ac.patch
Description: Binary data


dcrypt-openssl.c.patch
Description: Binary data

Re: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime)

[Ruga]

Timo,

re: What OS is this?
OS 10.12.3 with Xcode 8.2.1 and the official clang 3.9.0

re: test-time-util.c


t_strftime and variants now .. : ok


Info: 'Thu, 08 Dec 2016 18:42:16 +0100'


test-time-util.c:124: Assert failed: strcmp(t_strftime(RFC2822_FMT, 
gmtime(&ts)), exp) == 0


Info: 'Thu, 08 Dec 2016 18:42:16 +0100'


test-time-util.c:126: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) 
== 0


t_strftime and variants fixed timestamp .. : FAILED


timings 0  : ok


re: Why do you want to disable /dev/urandom?
https://github.com/libressl-portable/portable/issues/278

re: dcrypt-openssl.c, ssl.m4 and HMAC_CTX_free(NULL)
Please discuss this with busterb at GitHub.








 Original Message 
Subject: Re: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime)
Local Time: 25 February 2017 8:54 PM
UTC Time: 25 February 2017 19:54
From: t...@iki.fi
To: Ruga 
Dovecot Mailing List 

On 25 Feb 2017, at 21.11, Ruga  wrote:
>
> t_strftime and variants now .. : ok
>
>
> test-time-util.c:123: Assert failed: strcmp(t_strftime(RFC2822_FMT, 
> gmtime(&ts)), exp) == 0
>
>
> test-time-util.c:124: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), 
> exp) == 0
>
>
> t_strftime and variants fixed timestamp .. : 
> FAILED

What OS is this? Could you try with the attached patch what it logs?



> 

Why do you want to disable /dev/urandom?

> 

Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from lib-dcrypt. 
Will be removed in v2.2.29. Attached the planned patch that should do it.

v2.2.28: var-expand.c

[Ruga]

var-expand.c:687:17: warning: passing an object that undergoes default argument 
promotion to 'va_start' has undefined behavior [-Wvarargs]


va_start(args, key2);


^


var-expand.c:674:58: note: parameter of type 'char' is declared here


var_expand_table_build(char key, const char *value, char key2, ...)


^

Re: Tip: update dovecot MD5 password from PAM

[Ruga]

It is a jolly bad idea to use the same password for both email and system 
access.

On TLS+plaintext, if your passwords are slurped by a python script, all 
accounts are compromised. Congratulations, the NSA will love you. On the other 
side of the ocean, however, there are European states where you must disclose 
the fact, or go to jail.

(I tried to protect dovecot passwords with bcrypt, but the mail clients refused 
it.)

Sent from ProtonMail Mobile

Re: Tip: update dovecot MD5 password from PAM

[Ruga]

Right. But that's what I experienced.

The next experiment is sheduled in
two weeks... I will keep notes and
logs for you.

Sent from ProtonMail Mobile

On Mon, Mar 27, 2017 at 2:21 PM, Aki Tuomi  wrote:

On 27.03.2017 01:13, Ruga wrote:
> It is a jolly bad idea to use the same password for both email and system 
> access.
>
> On TLS+plaintext, if your passwords are slurped by a python script, all 
> accounts are compromised. Congratulations, the NSA will love you. On the 
> other side of the ocean, however, there are European states where you must 
> disclose the fact, or go to jail.
>
> (I tried to protect dovecot passwords with bcrypt, but the mail clients 
> refused it.)

Uh, what? Mail clients do not see how you are storing passwords locally.

>
> Sent from ProtonMail Mobile

Aki