Question having similiar replication issues
pls read everything below and advise the folder counts on the
non-replicated users?
i find the total number of folders / account seems to be a factor and
NOT the size of the mail box
ie i have customers with 40G of emails no problem over 40 or so folders
and it works ok
300+ folders seems to be the issue
i have been going through the replication code
no errors being logged
i am assuming that the replication --> dhclient --> other server is
timing out or not reading the folder lists correctly (ie dies after X
folders read)
thus i am going through the code patching for log entries etc to find
the issues.
see
[13:33:57] mail18.scom.ca [root:0] /usr/local/var/lib/dovecot
# ll
total 86
drwxr-xr-x 2 root wheel uarch4B Apr 24 11:11 .
drwxr-xr-x 4 root wheel uarch4B Mar 8 2021 ..
-rw-r--r-- 1 root wheel uarch 73B Apr 24 11:11 instances
-rw-r--r-- 1 root wheel uarch 160K Apr 24 13:33 replicator.db
[13:33:58] mail18.scom.ca [root:0] /usr/local/var/lib/dovecot
#
replicator.db seems to get updated ok but never processed properly.
# sync.users
n...@elirpa.com high 00:09:41 463:47:01 -
y
ke...@elirpa.com high 00:09:23 463:45:43 -
y
p...@scom.ca high 00:09:41 463:46:51 -
y
e...@scom.cahigh 00:09:43 463:47:01 -
y
ed.ha...@dssmgmt.com high 00:09:42 463:46:58 -
y
p...@paulkudla.nethigh 00:09:44 463:47:03 580:35:07
y
so
two things :
first to get the production stuff to work i had to write a script that
whould find the bad sync's and the force a dsync between the servers
i run this every five minutes or each server.
in crontab
*/10****root/usr/bin/nohup
/programs/common/sync.recover > /dev/null
python script to sort things out
# cat /programs/common/sync.recover
#!/usr/local/bin/python3
#Force sync between servers that are reporting bad?
import os,sys,django,socket
from optparse import OptionParser
from lib import *
#Sample Re-Index MB
#doveadm -D force-resync -u p...@scom.ca -f INBOX*
USAGE_TEXT = '''\
usage: %%prog %s[options]
'''
parser = OptionParser(usage=USAGE_TEXT % '', version='0.4')
parser.add_option("-m", "--send_to", dest="send_to", help="Send Email To")
parser.add_option("-e", "--email", dest="email_box", help="Box to Index")
parser.add_option("-d", "--detail",action='store_true',
dest="detail",default =False, help="Detailed report")
parser.add_option("-i", "--index",action='store_true',
dest="index",default =False, help="Index")
options, args = parser.parse_args()
print (options.email_box)
print (options.send_to)
print (options.detail)
#sys.exit()
print ('Getting Current User Sync Status')
command = commands("/usr/local/bin/doveadm replicator status '*'")
#print command
sync_user_status = command.output.split('\n')
#print sync_user_status
synced = []
for n in range(1,len(sync_user_status)) :
user = sync_user_status[n]
print ('Processing User : %s' %user.split(' ')[0])
if user.split(' ')[0] != options.email_box :
if options.email_box != None :
continue
if options.index == True :
command = '/usr/local/bin/doveadm -D force-resync -u %s
-f INBOX*' %user.split(' ')[0]
command = commands(command)
command = command.output
#print user
for nn in range (len(user)-1,0,-1) :
#print nn
#print user[nn]
if user[nn] == '-' :
#print 'skipping ... %s' %user.split(' ')[0]
break
if user[nn] == 'y': #Found a Bad Mailbox
print ('syncing ... %s' %user.split(' ')[0])
if options.detail == True :
command = '/usr/local/bin/doveadm -D
sync -u %s -d -N -l 30 -U' %user.split(' ')[0]
print (command)
command = commands(command)
command = command.output.split('\n')
print (command)
print ('Processed Mailbox for ... %s'
%user.split(' ')[0] )
synced.append('Processed Mailbox for
... %s' %user.split(' ')[0])
for nnn in range(len(command)):
synced.append(command[nnn] + '\n')
break
if options.detail == False :
#command = '/usr/local/bin/doveadm -D
sync -u %s -d -N -l 30 -U' %user.split(' ')[0]
#print (command)
#command = os.system(command)