I found that expecially the function "_index_thread_ref" is quite different (arguments and implementation) in 2.4.12, while it's very similar in 2.5.8 : is it safe to upgrade binaries from 2.4.12 to 2.5.8 on a running system, or do I need any kind of conversion? Also, the diff around "index_msgdata_free" where "if (!md) continue;" is added, is quite different from 2.5.8 too, but it's probably not necessary, as far as I can see...can you check this? Gabriele ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu Data: 19 luglio 2016 15.02.36 CEST Oggetto: Re: thread=refs I don't think much has changed in the threading code for a while. I would expect that the patch would apply pretty cleanly to 2.4.x. On 07/19/2016 02:58 AM, Gabriele Bulfon wrote: Wow! This is really interesting! What minimum version of cyrus sources can I use for these changes? At the moment I'm running servers with 2.4.12, on our illumos based distro XStreamOS. ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu Data: 14 luglio 2016 17.14.18 CEST Oggetto: Re: thread=refs I went ahead and committed an implementation of THREAD-REFS: https://github.com/cyrusimap/cyrus-imapd/commit/16747e608f32f9dd9348988d3f83cb8f1b037ff6 Per the draft, grouping by subject is skipped and threads (toplevel messages) are sorted by INTERNALDATE, while the messages within the threads are still sorted by SENTDATE. I confirmed that THREAD=REFERENCES is still correct, but I have nothing to compare THREAD=REFS results to. The current threading in Thunderbird is close, but it might be using INTERNALDATE throughout. On 07/12/2016 04:44 PM, Ken Murchison via Info-cyrus wrote: Gabriele, The attached patch is what I was thinking in terms of implementation. It skips the grouping by subject for REFS, but I didn't do the REFS-specific date handling. Contrary to what the THREAD=REFS draft says, I'm not sure if the special date handling should be done in step 4 or 6. I would have to did deeper into the code to see where is belongs. On 07/08/2016 11:36 AM, Gabriele Bulfon via Info-cyrus wrote: Mainly web clients, installed clients usually implements threading internally to overcome problems with the original references algorithm that is often confusing. The problem with references is that it includes subject grouping, that is an old netscape model of the 90s: today, we just need references within the headers ids, or we may take a wrong message in the thread just because it has a similar subject (for example automatic mails with same subjects would be treated as a thread, which is wrong). Now, we're staring to implement threading view on our web collaboration software, running on cyrus. So we are investigating how RoundCube is doing threading on a dovecot installation, and we found it to be the same as the client algrithm of Thunderbird, which is fine. Looking at the protocol, it uses REFS first, probably because it has no subject grouping by definition, and it should have a better date sorting. Should, because I found that Dovecot does not sort it reversed... Maybe I will ask Dovecot guys why they choose to keep sort same as references: I suspect that claim to support refs, but actually the do the same references functions, but never do subject grouping. ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu Data: 8 luglio 2016 17.17.32 CEST Oggetto: Re: thread=refs On 07/08/2016 11:08 AM, Gabriele Bulfon wrote: Ok, sure, but still two issues remain other than the draft: - we need to get rid of subject grouping in REFS, it only brings disorder, merging stuff that is not related I believe that the parameterization of the core functions should be able to handle this. - I would try to guess why dovecot does not change sorting in REFS, keeping it same as REFERENCES I would contact that Dovecot authors and find out which version of the THREAD=REFS draft they based their work on. BTW, which clients use THREAD=REFS? ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu Data: 8 luglio 2016 16.39.17 CEST Oggetto: Re: thread=refs Is there an actual RFC? All I find is draft-ietf-morg-inthread-01. Looking at that draft, the only difference between REFS ad REFERENCES is this: THREAD=REFS sorts threads by the most recent INTERNALDATE in each thread, replacing THREAD=REFERENCES step (4). This means that when a new message arrives, its thread becomes the latest thread. (Note that while threads are sorted by arrival date, messages within a thread are sorted by sent date, just as for THREAD=REFERENCES.) This being the case, I don't think we need two copies of the threading functions. I'd modify the exiting functions to take an additional parameter to specify whether we're doing REFS or REFERENCES and then have 2 wrapper functions which call the main function with the parameter set appropriately for the given algorithm. On 07/08/2016 10:03 AM, Gabriele Bulfon wrote: Ok, it works :) but checking against dovecot implementation, it looks like they have refs order same as references, but without subject grouping. AFAIK the RFC on refs says ordering of dates within the group should be reversed. Am I wrong? ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Gabriele Bulfon via Info-cyrus A: Ken Murchison info-cyrus@lists.andrew.cmu.edu Data: 8 luglio 2016 15.22.56 CEST Oggetto: Re: thread=refs Testing ;) and checking against a dovecot machine with refs and same messages. Will let you know ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu Data: 8 luglio 2016 15.02.38 CEST Oggetto: Re: thread=refs On 07/07/2016 02:03 PM, Gabriele Bulfon via Info-cyrus wrote: I can finally get back to this after so many months! I checked the sources, and I actually see it doesn't look very hard. Looks like: - renaming all functions like "index_thread_ref" into "index_thread_references" - duplicate them as "index_thread_refs" - let "references" alg call the "references" funcs - add support for "refs" in thread_algs and let them call the "refs" funcs Makes sense. then: - completely remove the call to "ref_group_subjects", we don't want it at all in refs - change the sortcrit to use the SORT_REVERSE modifier As long as you mean making these changes for just the "refs" variant and not both. what do you think? may be fine? ---------------------------------------------------------------------------------------- Sonicle S.r.l. : http://www.sonicle.com Music: http://www.gabrielebulfon.com Quantum Mechanics : http://www.cdbaby.com/cd/gabrielebulfon Da: Ken Murchison A: info-cyrus@lists.andrew.cmu.edu Data: 5 ottobre 2015 14.04.02 CEST Oggetto: Re: thread=refs As far as I can tell, the last specification for thread=refs was here: https://tools.ietf.org/html/draft-ietf-morg-inthread-01 To implement this you want to look at index.c in the Cyrus source and add another entry to the thread_algs[] array. I'm guessing that you can reuse a lot of the existing index_thread_ref() code (which is probably needs to be renamed to index_thread_references()). On 10/05/2015 06:07 AM, Gabriele Bulfon wrote: Great, Ken. Can you give me some advice / pointer to the sources I should look at? Gabriele Da: Ken Murchison A: info-cyrus@lists.andrew.cmu.edu Data: 2 ottobre 2015 19.08.04 CEST Oggetto: Re: thread=refs On 10/02/2015 10:53 AM, Gabriele Bulfon wrote: Nice, it's not a big deal for us to upgrade to new versions, surely easier than porting to Dovecot! ;) So, maybe we can help with the implementation. In my mind, it's almost about changing the "thread=reference" and let it omit the subject matching, change sorting and...maybe just this? How much hard do you think it is? That sounds about right from what I remember of THREAD=REFERENCES (which I co-authored and implemented) and THREAD=REFS (which I think was last documented in 2010). Da: Bron Gondwana A: info-cyrus@lists.andrew.cmu.edu Data: 2 ottobre 2015 12.59.08 CEST Oggetto: Re: thread=refs No, there isn't. The conversations work in 3.0 beta contains a lot of what would be required to efficiently implement THREAD=REFS, but nobody has done the work to implement it. It certainly will never be backported to the 2.4 series, which is only getting security updates and fixes for major bugs now. Regards, Bron. On Fri, Oct 2, 2015, at 18:40, Gabriele Bulfon wrote: Hi, we have systems running cyrus 2.4.12, where thread algorithms are only references and orderedsubject. Is there support for the thread=refs algorithm? Thanks Gabriele ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Bron Gondwana br...@fastmail.fm ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University ----Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University -- Kenneth MurchisonPrincipal Systems Software EngineerCarnegie Mellon University
---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
