Re: Ongoing inaccessible mailbox woes

[Patrick Goetz via Info-cyrus]

Hi Bron -

  [root@www cyrus]# pwd
  /etc/cyrus
  [root@www cyrus]# grep improved_mboxlist_sort imapd.conf
  improved_mboxlist_sort: yes

But in any case, it is seeing the mailbox.  The "Archives Staff" mailbox 
shows up with messages and one sub-mailbox -- the other sub-folders are 
all missing.


Question:  when I'm running reconstruct, what does it even mean to get 
the message "Mailbox already exists"?  Of course the mailbox already 
exists; else there would be no point in running reconstruct!


Next, when I look in

  /var/imap/user/d/djones/djones.sub

the folders I'm not seeing and which give the error messages are missing 
from this list -- could this be the problem?  How can I force cyrus to 
reconstruct this list, or is this one of the things that reconstruct does?



On 12/26/2015 01:14 AM, Bron Gondwana via Info-cyrus wrote:

It smells like maybe you have improved_mboxlist_sort set to the wrong value 
(off),
and hence it's unable to see the mailbox at all!  This is really bogus of the 
way that
mailboxes are stored, and I have master plans of fixing it better... *sigh*

Bron.

On Sat, Dec 26, 2015, at 02:47, Patrick Goetz via Info-cyrus wrote:

When I upgraded from 2.4.17 to 2.5.3, I had one user who experienced
inaccessible mail folders post upgrade.  Running reconstruct:

 # systemctl stop cyrus-master
 # su - cyrus
 # /usr/lib/cyrus/bin/reconstruct -r -f user/djones
 # -d
 # systemctl start cyrus-master

temporarily resolved the problem, but then it came back, so I thought
this might be a 2.5.3 bug.  I just upgraded to 2.5.7, and the problem
persists.  Furthermore, running reconstruct no longer brings the mail
messages back and I get the following error messages during the
reconstruction on precisely the folders which are not accessible:


user/djones/World Mission Conference
user/djones/World Mission Conference/2008
user/djones/World Mission Conference/2011
createmailbox user.djones.Archives Staff.Keely Davis: Mailbox already exists
createmailbox user.djones.Archives Staff.Lauren Kelsey: Mailbox already
exists
createmailbox user.djones.Archives Staff.Stacey Williams: Mailbox
already exists
createmailbox user.djones.Archives Staff.Drew Smith: Mailbox already exists
createmailbox user.djones.Archives Staff.Whitney: Mailbox already exists
createmailbox user.djones.Archives Staff.Delia Alexander: Mailbox
already exists
createmailbox user.djones.Archives Staff.Whitney Howell: Mailbox already
exists
createmailbox user.djones.Archives Staff.Former Staff: Mailbox already
exists
createmailbox user.djones.Archives Staff.Julia Miller: Mailbox already
exists
createmailbox user.djones.Archives Staff.Weekly Reports: Mailbox already
exists
createmailbox user.djones.Archives Staff.Eleanor Meuller: Mailbox
already exists
createmailbox user.djones.Archives Staff.Corrine Smith.Inquiries:
Mailbox already exists
createmailbox user.djones.Archives Staff.Corrine Smith.Destruction:
Mailbox already exists
createmailbox user.djones.Archives Staff.Corrine Smith.SharePoint:
Mailbox already exists


The folders and files in the filesystem look completely ordinary and
have the correct permissions.  However, under user/djones, this user has
the following mail folders:

drwx--  18 cyrus mail   98304 Dec  7 11:11 Archives Staff
drwx--   5 cyrus mail4096 Oct  3 05:21 Archives Staff Retreat


Now I'm starting to wonder if there isn't some kind of folder name
length bug at work here (maybe just in reconstruct?) so that the mail
folders

Archives Staff
Archives Staff Retreat

are being seen by reconstruct as the same folder?

I'm kind of grasping at straws here, unfortunately.  Last time this
happened, Bron suggested that I needed to set up logging with debugging
turned on.  If this is the only way to get to the bottom of this, I
guess I'll have to do that.  (Since the mail server is a systemd system,
traditional syslog logging is not turned on by defaul and will require
some effort to enable.)



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus





Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Archiving old email?

[Patrick Goetz via Info-cyrus]

I have some users whose mail folders are approaching 2GB in size. 
Unfortunately they can't delete any of this mail, but that doesn't mean 
that it needs to be lugged around, indexed, and copied to local mail 
caches as part of a live mail system.  I find that the probability of 
needing to look at mail more than 5 years old is roughly 0%/year.


I was going to ask if anyone had any ideas about archiving old mail, but 
then noticed that cyrus 3.0 is going to support some kind of archiving 
system:


  Archive support has arrived! Requires addition of an archive 
partition. (See archive_* options in imapd.conf)


Can anyone elaborate on what this actually means?



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Sieve vacation script?

[Patrick Goetz via Info-cyrus]

I've been struggling with figuring out how to set up a sieve vacation 
reply script.  The documentation on this is pretty sparse, and the stuff 
I do find doesn't work.  For example,


[cyrus@www ~]$ sieveshell --user=resea...@episcopalarchives.org 
--authname=cyrus localhost

connecting to localhost
unable to connect to server at /usr/bin/sieveshell line 191.
[cyrus@www ~]$ sieveshell --user=resea...@episcopalarchives.org 
--authname=cyrus episcopalarchives.org

connecting to episcopalarchives.org
unable to connect to server at /usr/bin/sieveshell line 191.

I'm also not sure what the point is of sieveshell is if I'm running it 
on the mail server and not remotely; similarly for timsieved.


First question:  is there step by step explanation for manually creating 
and invoking a cyrus sieve script for a particular user?


Second question:  Ancient unix/linux users will recall a simple vacation 
program that any user could run to throw up or take down an automated 
vacation reply.  The age of sieve has made this process considerably 
more complicated, but I would like to write (or obtain) a script that 
performs a similar function for cyrus sieve; i.e. the user runs vacation 
and it automatically sets up an automated vacation reply message. 
Anyone have something like this already?


I've been beating my head against google for many hours at this point 
looking for information on how this all works, particularly with cyrus 
and haven't found anything.




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Sieve vacation script?

[Sebastian Hagedorn via Info-cyrus]

Hi,


I've been struggling with figuring out how to set up a sieve vacation
reply script.  The documentation on this is pretty sparse, and the stuff
I do find doesn't work.  For example,

[cyrus@www ~]$ sieveshell --user=resea...@episcopalarchives.org
--authname=cyrus localhost
connecting to localhost
unable to connect to server at /usr/bin/sieveshell line 191.
[cyrus@www ~]$ sieveshell --user=resea...@episcopalarchives.org
--authname=cyrus episcopalarchives.org
connecting to episcopalarchives.org
unable to connect to server at /usr/bin/sieveshell line 191.


first of all you have to add lines to /etc/cyrus.conf to make the server 
listen to the right ports:


At least in our /etc/services file, "sieve" is still port 2000 even though 
that's been superseded by RFC 5804. We listen on both ports:


 sieve cmd="timsieved" listen="cyrus.rrz.uni-koeln.de:sieve" 
prefork=0
 managesieve   cmd="timsieved" listen="cyrus.rrz.uni-koeln.de:4190" 
prefork=0



I'm also not sure what the point is of sieveshell is if I'm running it on
the mail server and not remotely; similarly for timsieved.


You can run sieveshell on any computer you like. Just make sure the port 
mentioned above are reachable.



First question:  is there step by step explanation for manually creating
and invoking a cyrus sieve script for a particular user?


Not really. I prefer to use a GUI. We offer Smartsieve to our clients:



There's also an Add-On for Thunderbird:


--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
Universität zu Köln / Cologne University - Tel. +49-221-470-89578

pgpcMwNkJNLSS.pgp
Description: PGP signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Ongoing inaccessible mailbox woes

[Bron Gondwana via Info-cyrus]

On Sun, Dec 27, 2015, at 01:14, Patrick Goetz via Info-cyrus wrote:
> Hi Bron -
> 
>[root@www cyrus]# pwd
>/etc/cyrus
>[root@www cyrus]# grep improved_mboxlist_sort imapd.conf
>improved_mboxlist_sort: yes
> 
> But in any case, it is seeing the mailbox.  The "Archives Staff" mailbox 
> shows up with messages and one sub-mailbox -- the other sub-folders are 
> all missing.
> 
> Question:  when I'm running reconstruct, what does it even mean to get 
> the message "Mailbox already exists"?  Of course the mailbox already 
> exists; else there would be no point in running reconstruct!
> 
> Next, when I look in
> 
>/var/imap/user/d/djones/djones.sub
> 
> the folders I'm not seeing and which give the error messages are missing 
> from this list -- could this be the problem?  How can I force cyrus to 
> reconstruct this list, or is this one of the things that reconstruct does?

The subs file isn't significant at all.

create_namecheck returns MAILBOX_EXISTS, as does a rename to the same
name or on the same partition.  It's obviously create_namecheck that will be
triggering this.

I'd be very interested in seeing the exact files that are causing this.  Is it 
possible
for me to get a login to the system where this is running?  Happy to sign any
non-disclosure you need.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Archiving old email?

[Bron Gondwana via Info-cyrus]

On Sun, Dec 27, 2015, at 04:22, Patrick Goetz via Info-cyrus wrote:
> I have some users whose mail folders are approaching 2GB in size. 
> Unfortunately they can't delete any of this mail, but that doesn't mean 
> that it needs to be lugged around, indexed, and copied to local mail 
> caches as part of a live mail system.  I find that the probability of 
> needing to look at mail more than 5 years old is roughly 0%/year.
> 
> I was going to ask if anyone had any ideas about archiving old mail, but 
> then noticed that cyrus 3.0 is going to support some kind of archiving 
> system:
> 
>Archive support has arrived! Requires addition of an archive 
> partition. (See archive_* options in imapd.conf)
> 
> Can anyone elaborate on what this actually means?

Each mailbox can have its storage split between two drives - one that's
high speed with recent mail, and slower speed drives for bulk storage.

At FastMail, we store the current week's email on SSDs in RAID1, and the
remainder (almost all of it!) on big SATA drives in RAID6.

There are potential plans to switch to using an object store rather than
the RAID6, if we can make it stable enough - we can reduce the number
of copies required without reducing the safety/redundancy - which is
always nice for managing costs.

Bron.

-- 
  Bron Gondwana
  br...@fastmail.fm

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus