Some questions about code in lib-storage/index/maildir

[songliny]

Hi, Recently, I have read some code in dovecot 2.3.13. And I have 2 questions 
about some lines of code in lib-storage/index/maildir Firstly, in the function, 
maildir_scan_dir, from lib-storage/index/maildir/maildir-sync.c around line 575 
(https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f7831/src/lib-storage/index/maildir/maildir-sync.c#L575)
 I would like to know that after successfully stat the cur directory, does it 
assign the max value between st.st_mtime and start_time intensionally to 
new_check_time? Or maybe should it assign the value to the cur_check_time. if 
(stat(ctx->cur_dir, &st) == 0) { ctx->mbox->maildir_hdr.new_check_time = 
I_MAX(st.st_mtime, start_time); ctx->mbox->maildir_hdr.cur_mtime = st.st_mtime; 
ctx->mbox->maildir_hdr.cur_mtime_nsecs = ST_MTIME_NSEC(st); } Secondly, in the 
function, maildir_uidlist_sync_next_uid, from 
lib-storage/index/maildir/maildir-uidlist.c 
(https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f7831/s

rc/lib-storage/index/maildir/maildir-uidlist.c#L1760) Is it ok to remove 
MAILDIR_UIDLIST_REC_FLAG_NEW_DIR flag from the record for the record that is 
not found in ctx->files just like the behavior 
(https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f7831/src/lib-storage/index/maildir/maildir-uidlist.c#L1723)in
 the function, maildir_uidlist_sync_next_partial 
(https://github.com/dovecot/core/blob/6986a435e31dfe5bcd14985534b76b3c735f7831/src/lib-storage/index/maildir/maildir-uidlist.c#L1723),
 from the same file? I think when it gets to the function, 
maildir_uidlist_sync_next_uid, the mail file have already been moved to cur 
directory. And maybe it is ok to remove MAILDIR_UIDLIST_REC_FLAG_NEW_DIR flag 
from the record. That is, is it ok to add this line rec->flags &= 
ENUM_NEGATE(MAILDIR_UIDLIST_REC_FLAG_NEW_DIR); just before rec->flags = 
(rec->flags | flags) & ENUM_NEGATE(MAILDIR_UIDLIST_REC_FLAG_NONSYNCED); in 
maildir_uidlist_sync_next_uid? Thank you in advance to he

lp me get more understanding to these questions, Sony

doveadm sync - I/O has stalled

[songliny]

Hi All, We are using dovecot 2.3.19.1 We created a account with more than 1000 
mail folders in Maildir format to reproduce the issue. After weeks of testing, 
we have found a logic that may cause dsync to encounter the error - no activity 
for 900 seconds The function, dsync_ibc_stream_input, is the callback 
function after some data are ready for be read. This is part of what it does. 
o_stream_cork(ibc->output); ibc->ibc.io_callback(ibc->ibc.io_context); 
o_stream_uncork(ibc->output); Normally, 
ibc->ibc.io_callback(ibc->ibc.io_context) reads some data and then processes 
it. But when dsync connects over tcps, it uses function implementations in 
lib-ssl-iostream to send and receive data. And this simplified call stack would 
result in some data are read when calling o_stream_uncork o_stream_uncork => 
o_stream_flush => o_stream_ssl_flush_buffer => openssl_iostream_bio_sync => 
openssl_iostream_bio_input If some data arrive after 
ibc->ibc.io_callback(ibc->ibc.io_context) and before o_stre

am_uncork, o_stream_uncork would read the data and then return. After 
o_stream_uncork returns, dsync then waits for new data to be read or written. 
But because the data had been read in o_stream_uncork, and there may be no new 
data to be read, dsync may then wait until timeout is met. It may happen, but 
it is hard to reproduce. If you also encounter this situation, you may try to 
use dsync over tcp connection. It does not read data when calling 
o_stream_uncork. As a result, it may not stuck. Back to the error itself, Maybe 
openssl-stream should not read data when doing uncork(flush)? Song-Lin Yang