Package: release.debian.org Severity: normal Tags: buster User: release.debian....@packages.debian.org Usertags: pu
Hey, Akonadi in Buster has some bad issues, that make it nearly impossible for some users to use. I have tested the patches locally and on another friends laptop, who was affected by the unusable Akonadi. As Buster and sid are not that far away, the same patchset is tested by all sid users. I backported an important fix about the mutiple merge candidates issue from Akonadi 19.08 to 18.08. It does not fix the actual issue from happening, but it allows Akonadi to resolve the issue without user interaction (#939012). Than Akonadi components tend to crash, when you shutdown or logout. This is handled by (#939013). Sometime AKonadi gets into a dedlock and not even restarting your computer helps to solve the deadlock (#935981). On top of those patches I fixed the symbolfiles for the stable version. To make sure, that with further updates to Buster, we don't break the ABI. I know, that the patchset is not small. As KDEPIM is not usable for some users without these patches, the severity is grave for them. Other users have the luck of not hitting the bug. Unfortunately it is unclear, what triggers the behavior in first place, so there are no workaround to solve these issues otherwise. -- System Information: Debian Release: bullseye/sid APT prefers unstable-debug APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.2.0-2-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=en_US (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled
diff -Nru akonadi-18.08.3/debian/changelog akonadi-18.08.3/debian/changelog --- akonadi-18.08.3/debian/changelog 2019-04-29 16:24:10.000000000 +0200 +++ akonadi-18.08.3/debian/changelog 2019-08-30 22:11:22.000000000 +0200 @@ -1,3 +1,34 @@ +akonadi (4:18.08.3-7+deb10u1) buster; urgency=medium + + * Rebuild for buster. + + -- Sandro Knauß <he...@debian.org> Fri, 30 Aug 2019 22:11:22 +0200 + +akonadi (4:18.08.3-7) unstable; urgency=medium + + * Team upload. + + [ Sandro Knauß ] + * Add patch to fix: Akonadi components crash on logout. (Closes: #939013) + * Add patch to fix: Automatic recovery from Multiple Merge Candidates + error (Closes: #939012) + * Add patch with files, that are needed for other patches. + * Update symbols from buildds for 4:18.08.3 + + -- Sandro Knauß <he...@debian.org> Fri, 30 Aug 2019 12:59:47 +0200 + +akonadi (4:18.08.3-6) unstable; urgency=medium + + * Team upload. + + [ Sandro Knauß ] + * Fix "Akonadi don't anwser any requests and ends in deadlock" (Closes: #935981) + by adding upstream patches. + - Akonadi-fix-dangling-transaction-after-itemsync-fail.patch + - ItemSync-skip-handling-remote-items-if-local-changes.patch + + -- Sandro Knauß <he...@debian.org> Wed, 28 Aug 2019 19:31:31 +0200 + akonadi (4:18.08.3-5) unstable; urgency=medium * Team upload. diff -Nru akonadi-18.08.3/debian/libkf5akonadiagentbase5.symbols akonadi-18.08.3/debian/libkf5akonadiagentbase5.symbols --- akonadi-18.08.3/debian/libkf5akonadiagentbase5.symbols 2019-02-08 23:19:51.000000000 +0100 +++ akonadi-18.08.3/debian/libkf5akonadiagentbase5.symbols 2019-08-30 12:58:59.000000000 +0200 @@ -1,4 +1,4 @@ -# SymbolsHelper-Confirmed: 4:18.07.90 amd64 +# SymbolsHelper-Confirmed: 4:18.08.3 alpha amd64 arm64 armel armhf hppa hurd-i386 i386 m68k mips64el mipsel ppc64 ppc64el riscv64 s390x x32 libKF5AkonadiAgentBase.so.5 libkf5akonadiagentbase5 #MINVER# * Build-Depends-Package: libkf5akonadi-dev _ZN7Akonadi12ResourceBase10cancelTaskERK7QString@Base 4:15.07.90 @@ -197,7 +197,6 @@ _ZNK7Akonadi9AgentBase8isOnlineEv@Base 4:15.07.90 _ZNK7Akonadi9AgentBase8progressEv@Base 4:15.07.90 _ZNK7Akonadi9AgentBase9agentNameEv@Base 4:15.07.90 - (optional=templinst)_ZSt4swapIN7Akonadi10CollectionEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS6_ESt18is_move_assignableIS6_EEE5valueEvE4typeERS6_SG_@Base 4:18.07.90 _ZTI12QDBusContext@Base 4:15.07.90 _ZTIN7Akonadi12ResourceBaseE@Base 4:15.07.90 _ZTIN7Akonadi16PreprocessorBaseE@Base 4:15.07.90 diff -Nru akonadi-18.08.3/debian/libkf5akonadicore5abi2.symbols akonadi-18.08.3/debian/libkf5akonadicore5abi2.symbols --- akonadi-18.08.3/debian/libkf5akonadicore5abi2.symbols 2019-02-13 19:42:05.000000000 +0100 +++ akonadi-18.08.3/debian/libkf5akonadicore5abi2.symbols 2019-08-30 12:58:21.000000000 +0200 @@ -1,4 +1,4 @@ -# SymbolsHelper-Confirmed: 4:18.08.3 alpha amd64 arm64 armel armhf hppa hurd-i386 i386 mips mips64el mipsel powerpc ppc64 ppc64el s390x +# SymbolsHelper-Confirmed: 4:18.08.3 alpha amd64 arm64 armel armhf hppa hurd-i386 i386 m68k mips mips64el mipsel powerpc ppc64 ppc64el riscv64 s390x x32 libKF5AkonadiCore.so.5abi2 libkf5akonadicore5abi2 #MINVER# * Build-Depends-Package: libkf5akonadi-dev ABI_5_2@ABI_5_2 4:18.07.90 @@ -27,7 +27,8 @@ _ZN7Akonadi10Collection7fromUrlERK4QUrl@ABI_5_2 4:18.07.90 _ZN7Akonadi10Collection7setNameERK7QString@ABI_5_2 4:18.07.90 _ZN7Akonadi10Collection8mimeTypeEv@ABI_5_2 4:18.07.90 - (optional=templinst|arch=hurd-i386 i386 m68k)_ZN7Akonadi10Collection9attributeINS_25PersistentSearchAttributeEEEPT_NS0_12CreateOptionE@ABI_5_2 4:18.07.90 + (optional=templinst|arch=!mips !powerpc)_ZN7Akonadi10Collection9attributeINS_22EntityDisplayAttributeEEEPT_NS0_12CreateOptionE@ABI_5_2 4:18.08.3 + (optional=templinst|arch=alpha amd64 arm64 armel armhf hppa hurd-i386 i386 m68k mips64el mipsel ppc64 ppc64el riscv64 s390x x32)_ZN7Akonadi10Collection9attributeINS_25PersistentSearchAttributeEEEPT_NS0_12CreateOptionE@ABI_5_2 4:18.07.90 _ZN7Akonadi10Collection9setRightsE6QFlagsINS0_5RightEE@ABI_5_2 4:18.07.90 _ZN7Akonadi10CollectionC1ERKS0_@ABI_5_2 4:18.07.90 _ZN7Akonadi10CollectionC1Ev@ABI_5_2 4:18.07.90 @@ -1597,7 +1598,7 @@ _ZN7Akonadi9UnlinkJobD2Ev@ABI_5_2 4:18.07.90 _ZN9QHashData9hasShrunkEv@ABI_5_2 4:18.07.90 (optional=templinst)_ZNK12KConfigGroup9readEntryIxEE5QListIT_EPKcRKS3_@ABI_5_2 4:18.07.90 - (optional=templinst|arch=alpha hppa mips64el ppc64 ppc64el s390x)_ZNK12KConfigGroup9readEntryIxEET_PKcRKS1_@ABI_5_2 4:18.07.90 + (optional=templinst|arch=alpha hppa mips64el ppc64 ppc64el riscv64 s390x)_ZNK12KConfigGroup9readEntryIxEET_PKcRKS1_@ABI_5_2 4:18.07.90 _ZNK7Akonadi10Collection10attributesEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi10Collection10referencedEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi10Collection10shouldListENS0_11ListPurposeE@ABI_5_2 4:18.07.90 @@ -1620,7 +1621,9 @@ _ZNK7Akonadi10Collection8remoteIdEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi10Collection8resourceEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi10Collection9attributeERK10QByteArray@ABI_5_2 4:18.07.90 + (optional=templinst|arch=!mips !powerpc)_ZNK7Akonadi10Collection9attributeINS_22EntityDeletedAttributeEEEPT_v@ABI_5_2 4:18.08.3 (optional=templinst)_ZNK7Akonadi10Collection9attributeINS_22EntityDisplayAttributeEEEPT_v@ABI_5_2 4:18.07.90 + (optional=templinst|arch=!mips !powerpc)_ZNK7Akonadi10Collection9attributeINS_24CollectionQuotaAttributeEEEPT_v@ABI_5_2 4:18.08.3 _ZNK7Akonadi10Collection9isVirtualEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi10CollectioneqERKS0_@ABI_5_2 4:18.07.90 _ZNK7Akonadi10CollectionltERKS0_@ABI_5_2 4:18.07.90 @@ -2055,7 +2058,7 @@ (optional=templinst)_ZNK7Akonadi4Item11payloadImplINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENSt9enable_ifIXntsrNS_8Internal12PayloadTraitIT_EE13isPolymorphicESB_E4typeEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi4Item11payloadPathEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi4Item12hasAttributeERK10QByteArray@ABI_5_2 4:18.07.90 - (optional=templinst|arch=armel armhf mips mipsel powerpc)_ZNK7Akonadi4Item12hasAttributeINS_22EntityDisplayAttributeEEEbv@ABI_5_2 4:18.07.90 + (optional=templinst|arch=mips powerpc)_ZNK7Akonadi4Item12hasAttributeINS_22EntityDisplayAttributeEEEbv@ABI_5_2 4:18.07.90 _ZNK7Akonadi4Item13payloadBaseV2Eii@ABI_5_2 4:18.07.90 _ZNK7Akonadi4Item14remoteRevisionEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi4Item16ensureMetaTypeIdEi@ABI_5_2 4:18.07.90 @@ -2153,8 +2156,8 @@ _ZNK7Akonadi9ItemModel8rowCountERK11QModelIndex@ABI_5_2 4:18.07.90 _ZNK7Akonadi9ItemModel9mimeTypesEv@ABI_5_2 4:18.07.90 _ZNK7Akonadi9UnlinkJob10metaObjectEv@ABI_5_2 4:18.07.90 - (optional=templinst|arch=armel)_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE10_M_releaseEv@ABI_5_2 4:18.07.90 - (optional=templinst|arch=!armel)_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE10_M_releaseEv@ABI_5_2 4:18.07.90 + (optional=templinst|arch=armel riscv64)_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE10_M_releaseEv@ABI_5_2 4:18.07.90 + (optional=templinst|arch=!armel !riscv64)_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE10_M_releaseEv@ABI_5_2 4:18.07.90 (optional=templinst)_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE12_M_constructIPcEEvT_S7_St20forward_iterator_tag@ABI_5_2 4:18.07.90 _ZTIN5boost16exception_detail10clone_baseE@ABI_5_2 4:18.07.90 _ZTIN5boost9exceptionE@ABI_5_2 4:18.07.90 @@ -2259,11 +2262,11 @@ _ZTIN7Akonadi9ExceptionE@ABI_5_2 4:18.07.90 _ZTIN7Akonadi9ItemModelE@ABI_5_2 4:18.07.90 _ZTIN7Akonadi9UnlinkJobE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTIN9__gnu_cxx7__mutexE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTISt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.07.90 - (arch=!armel)_ZTISt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTISt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.07.90 - (arch=!armel)_ZTISt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 + (arch=armel riscv64)_ZTIN9__gnu_cxx7__mutexE@ABI_5_2 4:18.08.3 + (arch=armel riscv64)_ZTISt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.08.3 + (arch=!armel !riscv64)_ZTISt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 + (arch=armel riscv64)_ZTISt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.08.3 + (arch=!armel !riscv64)_ZTISt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 _ZTSN5boost16exception_detail10clone_baseE@ABI_5_2 4:18.07.90 _ZTSN5boost9exceptionE@ABI_5_2 4:18.07.90 _ZTSN7Akonadi10ConnectionE@ABI_5_2 4:18.07.90 @@ -2367,11 +2370,11 @@ _ZTSN7Akonadi9ExceptionE@ABI_5_2 4:18.07.90 _ZTSN7Akonadi9ItemModelE@ABI_5_2 4:18.07.90 _ZTSN7Akonadi9UnlinkJobE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTSN9__gnu_cxx7__mutexE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTSSt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.07.90 - (arch=!armel)_ZTSSt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 - (arch=armel)_ZTSSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.07.90 - (arch=!armel)_ZTSSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 + (arch=armel riscv64)_ZTSN9__gnu_cxx7__mutexE@ABI_5_2 4:18.08.3 + (arch=armel riscv64)_ZTSSt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.08.3 + (arch=!armel !riscv64)_ZTSSt11_Mutex_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 + (arch=armel riscv64)_ZTSSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE1EE@ABI_5_2 4:18.08.3 + (arch=!armel !riscv64)_ZTSSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE@ABI_5_2 4:18.07.90 _ZTVN5boost16exception_detail10clone_baseE@ABI_5_2 4:18.07.90 _ZTVN5boost9exceptionE@ABI_5_2 4:18.07.90 _ZTVN7Akonadi10ConnectionE@ABI_5_2 4:18.07.90 diff -Nru akonadi-18.08.3/debian/libkf5akonadiwidgets5abi1.symbols akonadi-18.08.3/debian/libkf5akonadiwidgets5abi1.symbols --- akonadi-18.08.3/debian/libkf5akonadiwidgets5abi1.symbols 2019-02-13 19:42:06.000000000 +0100 +++ akonadi-18.08.3/debian/libkf5akonadiwidgets5abi1.symbols 2019-08-30 12:58:21.000000000 +0200 @@ -1,4 +1,4 @@ -# SymbolsHelper-Confirmed: 4:18.08.3 alpha amd64 arm64 armel armhf hppa hurd-i386 i386 mips mips64el mipsel powerpc ppc64 ppc64el s390x +# SymbolsHelper-Confirmed: 4:18.08.3 alpha amd64 arm64 armel armhf hppa hurd-i386 i386 m68k mips mips64el mipsel powerpc ppc64 ppc64el riscv64 s390x x32 libKF5AkonadiWidgets.so.5abi1 libkf5akonadiwidgets5abi1 #MINVER# * Build-Depends-Package: libkf5akonadi-dev ABI_5_1@ABI_5_1 4:17.12.1 @@ -403,7 +403,8 @@ _ZN7Akonadi9TagWidgetD1Ev@ABI_5_1 4:17.12.1 _ZN7Akonadi9TagWidgetD2Ev@ABI_5_1 4:17.12.1 (optional=templinst)_ZNK12KConfigGroup9readEntryI5QSizeEET_PKcRKS2_@ABI_5_1 4:17.12.1 - (optional=templinst|arch=!amd64 !arm64)_ZNK12KConfigGroup9readEntryIbEET_PKcRKS1_@ABI_5_1 4:18.08.3 + (optional=templinst|arch=!amd64 !arm64 !x32)_ZNK12KConfigGroup9readEntryIbEET_PKcRKS1_@ABI_5_1 4:18.08.3 + (optional=templinst|arch=!mips !powerpc)_ZNK7Akonadi10Collection9attributeINS_22EntityDisplayAttributeEEEPT_v@ABI_5_1 4:18.08.3 _ZNK7Akonadi10ControlGui10metaObjectEv@ABI_5_1 4:17.12.1 _ZNK7Akonadi13TagEditWidget10metaObjectEv@ABI_5_1 4:17.12.1 _ZNK7Akonadi13TagEditWidget9selectionEv@ABI_5_1 4:17.12.1 diff -Nru akonadi-18.08.3/debian/patches/Akonadi-fix-dangling-transaction-after-itemsync-fail.patch akonadi-18.08.3/debian/patches/Akonadi-fix-dangling-transaction-after-itemsync-fail.patch --- akonadi-18.08.3/debian/patches/Akonadi-fix-dangling-transaction-after-itemsync-fail.patch 1970-01-01 01:00:00.000000000 +0100 +++ akonadi-18.08.3/debian/patches/Akonadi-fix-dangling-transaction-after-itemsync-fail.patch 2019-08-28 18:39:24.000000000 +0200 @@ -0,0 +1,70 @@ +From ca67354dcc5b4640f26de0b3e46c79cf1e50bc32 Mon Sep 17 00:00:00 2001 +From: David Faure <fa...@kde.org> +Date: Sun, 3 Mar 2019 11:10:36 +0100 +Subject: [PATCH 1/2] Akonadi: fix dangling transaction after itemsync failure + +Summary: +TransactionSequence was emitting result() twice when rolling back. + +* How did this happen? +The TransactionRollbackJob is (automatically) added as a subjob of the +TransactionSequence, so when it finishes, slotResult is called (like for +all subjobs), as well as rollbackResult(). +Since the latter emits result() already [mostly for symmetry with +commitResult()], we don't need to do that in slotResult (which doesn't do +it for the case of committing, either). + +* Why is it a problem to emit result() twice? +Well, first, it's against the law in KJob world. In practice, +ItemSyncPrivate::slotTransactionResult was called twice (for the same +TransactionSequence job) which made it decrement mTransactionJobs one +time too many. +As a result, checkDone() finished too early and didn't go into the +"commit transaction" branch for other transactions. +Leaving a transaction "open" is a good recipe for database deadlocks further +down the line. + +* Why did the TransactionSequence roll back in the first place? +In my case because of the infamous and not-yet fixed "Multiple merge +candidates" problem, but it seems that it can also happen when having +items without a part, according to Volker's investigations. +All of these issues still need to be fixed, but at least akonadi seems +to be still usable after they happen. + +CCBUG: 399167 + +Test Plan: Ctrl+L in kmail, with a folder having multiple items for the same RID + +Reviewers: dvratil, vkrause + +Reviewed By: dvratil + +Subscribers: kfunk, kde-pim + +Tags: #kde_pim + +Differential Revision: https://phabricator.kde.org/D19487 + +(cherry picked from commit f1281cf18f40fd69acd61c31b48f5ce43e138eea) +(cherry picked from commit 8ff596c4fe15199b66262c624d8b7c8d8ec7368f) +(cherry picked from commit 15c91a0ac93051465b37807efceb6e9fd36cb73b) +--- + src/core/jobs/transactionsequence.cpp | 2 -- + 1 file changed, 2 deletions(-) + +diff --git a/src/core/jobs/transactionsequence.cpp b/src/core/jobs/transactionsequence.cpp +index a7b50f075..89b956d5f 100644 +--- a/src/core/jobs/transactionsequence.cpp ++++ b/src/core/jobs/transactionsequence.cpp +@@ -133,8 +133,6 @@ void TransactionSequence::slotResult(KJob *job) + d->mState = TransactionSequencePrivate::Committing; + TransactionCommitJob *job = new TransactionCommitJob(this); + connect(job, &TransactionCommitJob::result, [d](KJob *job) { d->commitResult(job);}); +- } else if (d->mState == TransactionSequencePrivate::RollingBack) { +- emitResult(); + } + } + } else { +-- +2.23.0 + diff -Nru akonadi-18.08.3/debian/patches/Automatic-recovery-from-Multiple-Merge-Candidates-er.patch akonadi-18.08.3/debian/patches/Automatic-recovery-from-Multiple-Merge-Candidates-er.patch --- akonadi-18.08.3/debian/patches/Automatic-recovery-from-Multiple-Merge-Candidates-er.patch 1970-01-01 01:00:00.000000000 +0100 +++ akonadi-18.08.3/debian/patches/Automatic-recovery-from-Multiple-Merge-Candidates-er.patch 2019-08-30 12:20:10.000000000 +0200 @@ -0,0 +1,285 @@ +From 8332cf8a5aa39df6fb665cdbff1a48286d5698f5 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Daniel=20Vr=C3=A1til?= <dvra...@kde.org> +Date: Fri, 31 May 2019 13:20:27 +0200 +Subject: [PATCH 1/2] Automatic recovery from Multiple Merge Candidates error + +Summary: +Introduce a recovery codepath when Multiple Merge Candidates error +occurs during Item merging. Since clients generally do not use +merging, this really only happens during ItemSync. In such case we +quitely delete all the conflicting items from the database and reschedule +the collection sync. The next sync should then succeed and bring the +collection into a consistent state. + +Note that this does not fix the Multiple Merge Candidates bug - it can +still happen (and we still don't know how), but Akonadi should now be +able to recover from it automatically without user intervention, thus +making this issue less of a PITA. + +CCBUG: 338658 + +Test Plan: Successfuly auto-recovered a broken collection on my setup. + +Reviewers: #kde_pim, dfaure + +Reviewed By: dfaure + +Subscribers: vkrause, dfaure, ngraham, asturmlechner, kde-pim + +Tags: #kde_pim + +Differential Revision: https://phabricator.kde.org/D21455 + +(cherry picked from commit 8f230d7d7f8a4e2b97273585374a68902b5ef6cf) +--- + autotests/server/fakedatastore.cpp | 6 +-- + autotests/server/fakedatastore.h | 2 +- + src/server/handler/akappend.cpp | 72 +++++++++++++++++++++++++++--- + src/server/handler/akappend.h | 4 ++ + src/server/storage/datastore.cpp | 34 +++++++------- + src/server/storage/datastore.h | 4 +- + 6 files changed, 96 insertions(+), 26 deletions(-) + +diff --git a/autotests/server/fakedatastore.cpp b/autotests/server/fakedatastore.cpp +index d384e423c..7f6101b82 100644 +--- a/autotests/server/fakedatastore.cpp ++++ b/autotests/server/fakedatastore.cpp +@@ -259,11 +259,11 @@ bool FakeDataStore::appendPimItem(QVector<Part> &parts, + remote_id, remoteRevision, gid, pimItem); + } + +-bool FakeDataStore::cleanupPimItems(const PimItem::List &items) ++bool FakeDataStore::cleanupPimItems(const PimItem::List &items, bool silent) + { + mChanges.insert(QStringLiteral("cleanupPimItems"), +- QVariantList() << QVariant::fromValue(items)); +- return DataStore::cleanupPimItems(items); ++ QVariantList() << QVariant::fromValue(items) << silent); ++ return DataStore::cleanupPimItems(items, silent); + } + + bool FakeDataStore::unhidePimItem(PimItem &pimItem) +diff --git a/autotests/server/fakedatastore.h b/autotests/server/fakedatastore.h +index 01f7714b4..be1b5efcf 100644 +--- a/autotests/server/fakedatastore.h ++++ b/autotests/server/fakedatastore.h +@@ -104,7 +104,7 @@ public: + const QString &gid, + PimItem &pimItem) override; + +- virtual bool cleanupPimItems(const PimItem::List &items) override; ++ virtual bool cleanupPimItems(const PimItem::List &items, bool silent = false) override; + + virtual bool unhidePimItem(PimItem &pimItem) override; + virtual bool unhideAllPimItems() override; +diff --git a/src/server/handler/akappend.cpp b/src/server/handler/akappend.cpp +index 7c3afef3d..4d47c8171 100644 +--- a/src/server/handler/akappend.cpp ++++ b/src/server/handler/akappend.cpp +@@ -30,8 +30,12 @@ + #include "storage/partstreamer.h" + #include "storage/parthelper.h" + #include "storage/selectquerybuilder.h" ++#include "storage/itemretrievalmanager.h" + #include <private/externalpartstorage_p.h> + ++#include "shared/akranges.h" ++#include "shared/akscopeguard.h" ++ + #include <numeric> //std::accumulate + + using namespace Akonadi; +@@ -354,6 +358,61 @@ bool AkAppend::notify(const PimItem &item, const Collection &collection, + return true; + } + ++void AkAppend::recoverFromMultipleMergeCandidates(const PimItem::List &items, const Collection &collection) ++{ ++ // HACK HACK HACK: When this happens within ItemSync, we are running inside a client-side ++ // transaction, so just calling commit here won't have any effect, since this handler will ++ // ultimately fail and the client will rollback the transaction. To circumvent this, we ++ // will forcibly commit the transaction, do our changes here within a new transaction and ++ // then we open a new transaction so that the client won't notice. ++ ++ DataStore *db = DataStore::self(); ++ int transactionDepth = 0; ++ while (db->inTransaction()) { ++ ++transactionDepth; ++ db->commitTransaction(); ++ } ++ const AkScopeGuard restoreTransaction([&]() { ++ for (int i = 0; i < transactionDepth; ++i) { ++ db->beginTransaction(QStringLiteral("RestoredTransactionAfterMMCRecovery")); ++ } ++ }); ++ ++ Transaction transaction(db, QStringLiteral("MMC Recovery Transaction")); ++ ++ // If any of the conflicting items is dirty or does not have a remote ID, we don't want to remove ++ // them as it would cause data loss. There's a chance next changeReplay will fix this, so ++ // next time the ItemSync hits this multiple merge candidates, all changes will be committed ++ // and this check will succeed ++ if (items | any([](const auto &item) { return item.dirty() || item.remoteId().isEmpty(); })) { ++ qCWarning(AKONADISERVER_LOG) << "Automatic multiple merge candidates recovery failed: at least one of the candidates has uncommitted changes!"; ++ return; ++ } ++ ++ // This cannot happen with ItemSync, but in theory could happen during individual GID merge. ++ if (items | any([collection](const auto &item) { return item.collectionId() != collection.id(); })) { ++ qCWarning(AKONADISERVER_LOG) << "Automatic multiple merge candidates recovery failed: all candidates do not belong to the same collection."; ++ return; ++ } ++ ++ db->cleanupPimItems(items, DataStore::Silent); ++ if (!transaction.commit()) { ++ qCWarning(AKONADISERVER_LOG) << "Automatic multiple merge candidates recovery failed: failed to commit database transaction."; ++ return; ++ } ++ ++ ++ // Schedule a new sync of the collection, one that will succeed ++ const auto resource = collection.resource().name(); ++ QMetaObject::invokeMethod(ItemRetrievalManager::instance(), "triggerCollectionSync", ++ Qt::QueuedConnection, ++ Q_ARG(QString, resource), Q_ARG(qint64, collection.id())); ++ ++ qCInfo(AKONADISERVER_LOG) << "Automatic multiple merge candidates recovery successful: conflicting items" << (items | transform([](const auto &i) { return i.id(); }) | toQVector) ++ << "in collection" << collection.name() << "(ID:" << collection.id() << ") were removed and a new sync was scheduled in the resource" ++ << resource; ++} ++ + bool AkAppend::parseStream() + { + const auto &cmd = Protocol::cmdCast<Protocol::CreateItemCommand>(m_command); +@@ -372,7 +431,6 @@ bool AkAppend::parseStream() + return false; + } + +- + if (cmd.mergeModes() == Protocol::CreateItemCommand::None) { + if (!insertItem(cmd, item, parentCol)) { + return false; +@@ -436,16 +494,20 @@ bool AkAppend::parseStream() + } + storageTrx.commit(); + } else { +- qCDebug(AKONADISERVER_LOG) << "Multiple merge candidates:"; ++ qCWarning(AKONADISERVER_LOG) << "Multiple merge candidates, will attempt to recover:"; + for (const PimItem &item : result) { + qCDebug(AKONADISERVER_LOG) << "\tID:" << item.id() << ", RID:" << item.remoteId() + << ", GID:" << item.gid() + << ", Collection:" << item.collection().name() << "(" << item.collectionId() << ")" + << ", Resource:" << item.collection().resource().name() << "(" << item.collection().resourceId() << ")"; + } +- // Nor GID or RID are guaranteed to be unique, so make sure we don't merge +- // something we don't want +- return failureResponse(QStringLiteral("Multiple merge candidates, aborting")); ++ ++ transaction.commit(); // commit the current transaction, before we attempt MMC recovery ++ recoverFromMultipleMergeCandidates(result, parentCol); ++ ++ // Even if the recovery was successful, indicate error to force the client to abort the ++ // sync, since we've interfered with the overall state. ++ return failureResponse(QStringLiteral("Multiple merge candidates in collection '%1', aborting").arg(item.collection().name())); + } + } + +diff --git a/src/server/handler/akappend.h b/src/server/handler/akappend.h +index 98c9bc206..99de77907 100644 +--- a/src/server/handler/akappend.h ++++ b/src/server/handler/akappend.h +@@ -27,6 +27,8 @@ namespace Akonadi + namespace Server + { + ++class Transaction; ++ + /** + @ingroup akonadi_server_handler + +@@ -61,6 +63,8 @@ private: + bool notify(const PimItem &item, bool seen, const Collection &collection); + bool notify(const PimItem &item, const Collection &collection, + const QSet<QByteArray> &changedParts); ++ ++ void recoverFromMultipleMergeCandidates(const PimItem::List &items, const Collection &collection); + }; + + } // namespace Server +diff --git a/src/server/storage/datastore.cpp b/src/server/storage/datastore.cpp +index f846af5b1..ff1330ee5 100644 +--- a/src/server/storage/datastore.cpp ++++ b/src/server/storage/datastore.cpp +@@ -1158,29 +1158,31 @@ bool DataStore::unhideAllPimItems() + return false; + } + +-bool DataStore::cleanupPimItems(const PimItem::List &items) ++bool DataStore::cleanupPimItems(const PimItem::List &items, bool silent) + { + // generate relation removed notifications +- for (const PimItem &item : items) { +- SelectQueryBuilder<Relation> relationQuery; +- relationQuery.addValueCondition(Relation::leftIdFullColumnName(), Query::Equals, item.id()); +- relationQuery.addValueCondition(Relation::rightIdFullColumnName(), Query::Equals, item.id()); +- relationQuery.setSubQueryMode(Query::Or); ++ if (!silent) { ++ for (const PimItem &item : items) { ++ SelectQueryBuilder<Relation> relationQuery; ++ relationQuery.addValueCondition(Relation::leftIdFullColumnName(), Query::Equals, item.id()); ++ relationQuery.addValueCondition(Relation::rightIdFullColumnName(), Query::Equals, item.id()); ++ relationQuery.setSubQueryMode(Query::Or); + +- if (!relationQuery.exec()) { +- throw HandlerException("Failed to obtain relations"); +- } +- const Relation::List relations = relationQuery.result(); +- for (const Relation &relation : relations) { +- notificationCollector()->relationRemoved(relation); ++ if (!relationQuery.exec()) { ++ throw HandlerException("Failed to obtain relations"); ++ } ++ const Relation::List relations = relationQuery.result(); ++ for (const Relation &relation : relations) { ++ notificationCollector()->relationRemoved(relation); ++ } + } +- } + +- // generate the notification before actually removing the data +- notificationCollector()->itemsRemoved(items); ++ // generate the notification before actually removing the data ++ notificationCollector()->itemsRemoved(items); ++ } + + // FIXME: Create a single query to do this +- Q_FOREACH (const PimItem &item, items) { ++ for (const auto &item : items) { + if (!item.clearFlags()) { + return false; + } +diff --git a/src/server/storage/datastore.h b/src/server/storage/datastore.h +index 2f6fb36df..1eb934fd7 100644 +--- a/src/server/storage/datastore.h ++++ b/src/server/storage/datastore.h +@@ -103,6 +103,8 @@ class DataStore : public QObject + { + Q_OBJECT + public: ++ const constexpr static bool Silent = true; ++ + /** + Closes the database connection and destroys the DataStore object. + */ +@@ -199,7 +201,7 @@ public: + /** + * Removes the pim item and all referenced data ( e.g. flags ) + */ +- virtual bool cleanupPimItems(const PimItem::List &items); ++ virtual bool cleanupPimItems(const PimItem::List &items, bool silent = false); + + /** + * Unhides the specified PimItem. Emits the itemAdded() notification as +-- +2.23.0 + diff -Nru akonadi-18.08.3/debian/patches/Backport-missing-files-from-master.patch akonadi-18.08.3/debian/patches/Backport-missing-files-from-master.patch --- akonadi-18.08.3/debian/patches/Backport-missing-files-from-master.patch 1970-01-01 01:00:00.000000000 +0100 +++ akonadi-18.08.3/debian/patches/Backport-missing-files-from-master.patch 2019-08-30 12:26:27.000000000 +0200 @@ -0,0 +1,894 @@ +From 80f8e85cee83afd5af5b9694aef0b7fbaffb68a2 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Sandro=20Knau=C3=9F?= <skna...@kde.org> +Date: Fri, 30 Aug 2019 10:13:59 +0200 +Subject: [PATCH] Backport missing files from master. + +src/shared/akhelpers.h d6f65c5733f35de3061e9c73c54eb3bf53ffe04b +src/shared/akranges.h 354139b155cc9295c97fcd7887738fedce1b6837 +src/shared/akscopeguard.h 179bd0e766b314d976951e00a9fc4d4d6a8938a2 +src/shared/aktraits.h f85c06a7b3253882d8a1fba626fde07291df6892 +--- + src/shared/akhelpers.h | 119 +++++++++ + src/shared/akranges.h | 511 ++++++++++++++++++++++++++++++++++++++ + src/shared/akscopeguard.h | 52 ++++ + src/shared/aktraits.h | 165 ++++++++++++ + 4 files changed, 847 insertions(+) + create mode 100644 src/shared/akhelpers.h + create mode 100644 src/shared/akranges.h + create mode 100644 src/shared/akscopeguard.h + create mode 100644 src/shared/aktraits.h + +diff --git a/src/shared/akhelpers.h b/src/shared/akhelpers.h +new file mode 100644 +index 000000000..e1ab2ba29 +--- /dev/null ++++ b/src/shared/akhelpers.h +@@ -0,0 +1,119 @@ ++/* ++ Copyright (C) 2018 - 2019 Daniel Vrátil <dvra...@kde.org> ++ ++ This library is free software; you can redistribute it and/or modify it ++ under the terms of the GNU Library General Public License as published by ++ the Free Software Foundation; either version 2 of the License, or (at your ++ option) any later version. ++ ++ This library is distributed in the hope that it will be useful, but WITHOUT ++ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or ++ FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public ++ License for more details. ++ ++ You should have received a copy of the GNU Library General Public License ++ along with this library; see the file COPYING.LIB. If not, write to the ++ Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ++ 02110-1301, USA. ++*/ ++ ++#ifndef AKONADI_AKHELPERS_H_ ++#define AKONADI_AKHELPERS_H_ ++ ++#include <type_traits> ++#include <utility> ++ ++namespace Akonadi ++{ ++namespace detail ++{ ++ ++// C++14-compatible implementation of std::invoke(), based on implementation ++// from https://en.cppreference.com/w/cpp/utility/functional/invoke ++ ++template<typename T> ++struct is_reference_wrapper : std::false_type {}; ++template<typename T> ++struct is_reference_wrapper<std::reference_wrapper<T>> : std::true_type {}; ++template<typename T> ++constexpr bool is_reference_wrapper_v = is_reference_wrapper<T>::value; ++ ++template<typename T> ++constexpr bool is_member_function_pointer_v = std::is_member_function_pointer<std::remove_cv_t<std::remove_reference_t<T>>>::value; ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ... args) -> ++ std::enable_if_t<is_member_function_pointer_v<decltype(fun)> ++ && std::is_base_of<T, std::decay_t<T1>>::value, ++ std::result_of_t<decltype(fun)(T, Args ...)>> ++{ ++ return (std::forward<T1>(t1).*fun)(std::forward<Args>(args) ...); ++} ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ... args) -> ++ std::enable_if_t<is_member_function_pointer_v<decltype(fun)> ++ && is_reference_wrapper_v<std::decay_t<T1>>, ++ std::result_of_t<decltype(fun)(T, Args ...)>> ++{ ++ return (t1.get().*fun)(std::forward<Args>(args) ...); ++} ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ... args) -> ++ std::enable_if_t<is_member_function_pointer_v<decltype(fun)> ++ && !std::is_base_of<T, std::decay_t<T1>>::value ++ && !is_reference_wrapper_v<std::decay_t<T1>>, ++ std::result_of_t<decltype(fun)(T1, Args ...)>> ++{ ++ return ((*std::forward<T1>(t1)).*fun)(std::forward<Args>(args) ...); ++} ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ...) -> ++ std::enable_if_t<!is_member_function_pointer_v<decltype(fun)> ++ && std::is_base_of<T1, std::decay_t<T1>>::value, ++ std::result_of_t<decltype(fun)(T1, Args ...)>> ++{ ++ return std::forward<T1>(t1).*fun; ++} ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ...) -> ++ std::enable_if_t<!is_member_function_pointer_v<decltype(fun)> ++ && is_reference_wrapper_v<std::decay_t<T1>>, ++ std::result_of_t<decltype(fun)(T1, Args ...)>> ++{ ++ return t1.get().*fun; ++} ++ ++template<typename T, typename Type, typename T1, typename ... Args> ++auto invoke(Type T::* fun, T1 &&t1, Args && ...) -> ++ std::enable_if_t<!is_member_function_pointer_v<decltype(fun)> ++ && !std::is_base_of<T1, std::decay_t<T1>>::value ++ && !is_reference_wrapper_v<std::decay_t<T1>>, ++ std::result_of_t<decltype(fun)(T1, Args ...)>> ++{ ++ return (*std::forward<T1>(t1)).*fun; ++} ++ ++template<typename Fun, typename ... Args> ++auto invoke(Fun &&fun, Args && ... args) ++{ ++ return std::forward<Fun>(fun)(std::forward<Args>(args) ...); ++} ++ ++} // namespace detail ++ ++template<typename Fun, typename ... Args> ++auto invoke(Fun &&fun, Args && ... args) ++{ ++ return detail::invoke(std::forward<Fun>(fun), std::forward<Args>(args) ...); ++} ++ ++static const auto IsNull = [](auto ptr) { return !(bool)ptr; }; ++static const auto IsNotNull = [](auto ptr) { return (bool)ptr; }; ++ ++} // namespace Akonadi ++ ++#endif +diff --git a/src/shared/akranges.h b/src/shared/akranges.h +new file mode 100644 +index 000000000..6ee22f092 +--- /dev/null ++++ b/src/shared/akranges.h +@@ -0,0 +1,511 @@ ++/* ++ Copyright (C) 2018 - 2019 Daniel Vrátil <dvra...@kde.org> ++ ++ This library is free software; you can redistribute it and/or modify it ++ under the terms of the GNU Library General Public License as published by ++ the Free Software Foundation; either version 2 of the License, or (at your ++ option) any later version. ++ ++ This library is distributed in the hope that it will be useful, but WITHOUT ++ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or ++ FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public ++ License for more details. ++ ++ You should have received a copy of the GNU Library General Public License ++ along with this library; see the file COPYING.LIB. If not, write to the ++ Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ++ 02110-1301, USA. ++*/ ++ ++#ifndef AKONADI_AKRANGES_H ++#define AKONADI_AKRANGES_H ++ ++#include "aktraits.h" ++#include "akhelpers.h" ++ ++#include <QVector> ++#include <QSet> ++#include <QMap> ++ ++#include <algorithm> ++#include <functional> ++#include <utility> ++#include <type_traits> ++#include <iterator> ++ ++namespace Akonadi { ++ ++namespace detail { ++ ++template<template<typename> class Cont> ++struct To_ ++{ ++ template<typename T> using Container = Cont<T>; ++}; ++ ++template<template<typename, typename> class Cont> ++struct ToAssoc_ ++{ ++ template<typename Key, typename Value> using Container = Cont<Key, Value>; ++}; ++ ++struct Values_ {}; ++struct Keys_ {}; ++ ++template<typename RangeLike, typename OutContainer, ++ AK_REQUIRES(AkTraits::isAppendable<OutContainer>)> ++OutContainer copyContainer(const RangeLike &range) ++{ ++ OutContainer rv; ++ rv.reserve(range.size()); ++ for (auto &&v : range) { ++ rv.push_back(std::move(v)); ++ } ++ return rv; ++} ++ ++template<typename RangeLike, typename OutContainer, ++ AK_REQUIRES(AkTraits::isInsertable<OutContainer>)> ++OutContainer copyContainer(const RangeLike &range) ++{ ++ OutContainer rv; ++ rv.reserve(range.size()); ++ for (const auto &v : range) { ++ rv.insert(v); ++ } ++ return rv; ++} ++ ++template<typename RangeList, typename OutContainer> ++OutContainer copyAssocContainer(const RangeList &range) ++{ ++ OutContainer rv; ++ for (const auto &v : range) { ++ rv.insert(v.first, v.second); ++ } ++ return rv; ++} ++ ++template<typename Iterator> ++struct IteratorTrait { ++ using iterator_category = typename Iterator::iterator_category; ++ using value_type = typename Iterator::value_type; ++ using difference_type = typename Iterator::difference_type; ++ using pointer = typename Iterator::pointer; ++ using reference = typename Iterator::reference; ++}; ++ ++// Without QT_STRICT_ITERATORS QVector and QList iterators do not satisfy STL ++// iterator concepts since they are nothing more but typedefs to T* - for those ++// we need to provide custom traits. ++template<typename Iterator> ++struct IteratorTrait<Iterator*> { ++ // QTypedArrayData::iterator::iterator_category ++ using iterator_category = std::random_access_iterator_tag; ++ using value_type = Iterator; ++ using difference_type = int; ++ using pointer = Iterator*; ++ using reference = Iterator&; ++}; ++ ++template<typename Iterator> ++struct IteratorTrait<const Iterator *> { ++ using iterator_category = std::random_access_iterator_tag; ++ using value_type = Iterator; ++ using difference_type = int; ++ using pointer = const Iterator *; ++ using reference = const Iterator &; ++}; ++ ++template<typename IterImpl, typename RangeLike, typename Iterator = typename RangeLike::const_iterator> ++struct IteratorBase ++{ ++public: ++ using iterator_category = typename IteratorTrait<Iterator>::iterator_category; ++ using value_type = typename IteratorTrait<Iterator>::value_type; ++ using difference_type = typename IteratorTrait<Iterator>::difference_type; ++ using pointer = typename IteratorTrait<Iterator>::pointer; ++ using reference = typename IteratorTrait<Iterator>::reference; ++ ++ IteratorBase(const IteratorBase<IterImpl, RangeLike> &other) ++ : mIter(other.mIter), mRange(other.mRange) ++ {} ++ ++ IterImpl &operator++() ++ { ++ ++static_cast<IterImpl*>(this)->mIter; ++ return *static_cast<IterImpl*>(this); ++ } ++ ++ IterImpl operator++(int) ++ { ++ auto ret = *static_cast<IterImpl*>(this); ++ ++static_cast<IterImpl*>(this)->mIter; ++ return ret; ++ } ++ ++ bool operator==(const IterImpl &other) const ++ { ++ return mIter == other.mIter; ++ } ++ ++ bool operator!=(const IterImpl &other) const ++ { ++ return !(*static_cast<const IterImpl*>(this) == other); ++ } ++ ++ bool operator<(const IterImpl &other) const ++ { ++ return mIter < other.mIter; ++ } ++ ++ auto operator-(const IterImpl &other) const ++ { ++ return mIter - other.mIter; ++ } ++ ++ auto operator*() const ++ { ++ return *mIter; ++ } ++ ++protected: ++ IteratorBase(const Iterator &iter, const RangeLike &range) ++ : mIter(iter), mRange(range) ++ {} ++ IteratorBase(const Iterator &iter, RangeLike &&range) ++ : mIter(iter), mRange(std::move(range)) ++ {} ++ ++ Iterator mIter; ++ RangeLike mRange; ++}; ++ ++template<typename RangeLike, typename TransformFn, typename Iterator = typename RangeLike::const_iterator> ++struct TransformIterator : public IteratorBase<TransformIterator<RangeLike, TransformFn>, RangeLike> ++{ ++private: ++ template<typename ... T> ++ struct ResultOf; ++ ++ template<typename R, typename ... Args> ++ struct ResultOf<R(Args ...)> { ++ using type = R; ++ }; ++ ++ template<typename ... Ts> ++ using FuncHelper = decltype(Akonadi::invoke(std::declval<Ts>() ...))(Ts ...); ++ using IteratorValueType = typename ResultOf<FuncHelper<TransformFn, typename IteratorTrait<Iterator>::value_type>>::type; ++public: ++ using value_type = IteratorValueType; ++ using pointer = IteratorValueType *; // FIXME: preserve const-ness ++ using reference = const IteratorValueType &; // FIXME: preserve const-ness ++ ++ TransformIterator(const Iterator &iter, const TransformFn &fn, const RangeLike &range) ++ : IteratorBase<TransformIterator<RangeLike, TransformFn>, RangeLike>(iter, range) ++ , mFn(fn) ++ { ++ } ++ ++ auto operator*() const ++ { ++ return Akonadi::invoke(mFn, *this->mIter); ++ } ++ ++private: ++ TransformFn mFn; ++}; ++ ++template<typename RangeLike, typename Predicate, typename Iterator = typename RangeLike::const_iterator> ++class FilterIterator : public IteratorBase<FilterIterator<RangeLike, Predicate>, RangeLike> ++{ ++public: ++ FilterIterator(const Iterator &iter, const Iterator &end, const Predicate &predicate, const RangeLike &range) ++ : IteratorBase<FilterIterator<RangeLike, Predicate>, RangeLike>(iter, range) ++ , mPredicate(predicate), mEnd(end) ++ { ++ while (this->mIter != mEnd && !Akonadi::invoke(mPredicate, *this->mIter)) { ++ ++this->mIter; ++ } ++ } ++ ++ auto &operator++() ++ { ++ if (this->mIter != mEnd) { ++ do { ++ ++this->mIter; ++ } while (this->mIter != mEnd && !Akonadi::invoke(mPredicate, *this->mIter)); ++ } ++ return *this; ++ } ++ ++ auto operator++(int) ++ { ++ auto it = *this; ++ ++(*this); ++ return it; ++ } ++ ++private: ++ Predicate mPredicate; ++ Iterator mEnd; ++}; ++ ++ ++template<typename Container, int Pos, typename Iterator = typename Container::const_key_value_iterator> ++class AssociativeContainerIterator ++ : public IteratorBase<AssociativeContainerIterator<Container, Pos>, Container, Iterator> ++{ ++public: ++ using value_type = std::remove_const_t<std::remove_reference_t< ++ typename std::tuple_element<Pos, typename Iterator::value_type>::type>>; ++ using pointer = std::add_pointer_t<value_type>; ++ using reference = std::add_lvalue_reference_t<value_type>; ++ ++ AssociativeContainerIterator(const Iterator &iter, const Container &container) ++ : IteratorBase<AssociativeContainerIterator<Container, Pos>, Container, Iterator>(iter, container) ++ {} ++ ++ auto operator*() const ++ { ++ return std::get<Pos>(*this->mIter); ++ } ++}; ++ ++template<typename Container> ++using AssociativeContainerKeyIterator = AssociativeContainerIterator<Container, 0>; ++template<typename Container> ++using AssociativeContainerValueIterator = AssociativeContainerIterator<Container, 1>; ++ ++ ++template<typename Iterator> ++struct Range ++{ ++public: ++ using iterator = Iterator; ++ using const_iterator = Iterator; ++ using value_type = typename detail::IteratorTrait<Iterator>::value_type; ++ ++ Range(Iterator &&begin, Iterator &&end) ++ : mBegin(std::move(begin)) ++ , mEnd(std::move(end)) ++ {} ++ ++ Iterator begin() const ++ { ++ return mBegin; ++ } ++ ++ Iterator cbegin() const ++ { ++ return mBegin; ++ } ++ ++ Iterator end() const ++ { ++ return mEnd; ++ } ++ ++ Iterator cend() const ++ { ++ return mEnd; ++ } ++ ++ auto size() const ++ { ++ return std::distance(mBegin, mEnd); ++ } ++ ++private: ++ Iterator mBegin; ++ Iterator mEnd; ++}; ++ ++template<typename T> ++using IsRange = typename std::is_same<T, Range<typename T::iterator>>; ++ ++template<typename TransformFn> ++struct Transform_ ++{ ++ TransformFn mFn; ++}; ++ ++template<typename PredicateFn> ++struct Filter_ ++{ ++ PredicateFn mFn; ++}; ++ ++template<typename EachFun> ++struct ForEach_ ++{ ++ EachFun mFn; ++}; ++ ++template<typename Predicate> ++struct All_ ++{ ++ Predicate mFn; ++}; ++ ++template<typename Predicate> ++struct Any_ ++{ ++ Predicate mFn; ++}; ++ ++} // namespace detail ++} // namespace Akonadi ++ ++// Generic operator| for To_<> convertor ++template<typename RangeLike, template<typename> class OutContainer, typename T = typename RangeLike::value_type> ++auto operator|(const RangeLike &range, const Akonadi::detail::To_<OutContainer> &) -> OutContainer<T> ++{ ++ using namespace Akonadi::detail; ++ return copyContainer<RangeLike, OutContainer<T>>(range); ++} ++ ++// Specialization for case when InContainer and OutContainer are identical ++// Create a copy, but for Qt container this is very cheap due to implicit sharing. ++template<template<typename> class InContainer, typename T> ++auto operator|(const InContainer<T> &in, const Akonadi::detail::To_<InContainer> &) -> InContainer<T> ++{ ++ return in; ++} ++ ++// Generic operator| for ToAssoc_<> convertor ++template<typename RangeLike, template<typename, typename> class OutContainer, ++ typename T = typename RangeLike::value_type> ++auto operator|(const RangeLike &range, const Akonadi::detail::ToAssoc_<OutContainer> &) -> ++ OutContainer<typename T::first_type, typename T::second_type> ++{ ++ using namespace Akonadi::detail; ++ return copyAssocContainer<RangeLike, OutContainer<typename T::first_type, typename T::second_type>>(range); ++} ++ ++// Generic operator| for transform() ++template<typename RangeLike, typename TransformFn> ++auto operator|(const RangeLike &range, const Akonadi::detail::Transform_<TransformFn> &t) ++{ ++ using namespace Akonadi::detail; ++ using OutIt = TransformIterator<RangeLike, TransformFn>; ++ return Range<OutIt>(OutIt(std::cbegin(range), t.mFn, range), OutIt(std::cend(range), t.mFn, range)); ++} ++ ++// Generic operator| for filter() ++template<typename RangeLike, typename PredicateFn> ++auto operator|(const RangeLike &range, const Akonadi::detail::Filter_<PredicateFn> &p) ++{ ++ using namespace Akonadi::detail; ++ using OutIt = FilterIterator<RangeLike, PredicateFn>; ++ return Range<OutIt>(OutIt(std::cbegin(range), std::cend(range), p.mFn, range), ++ OutIt(std::cend(range), std::cend(range), p.mFn, range)); ++} ++ ++// Generic operator| fo foreach() ++template<typename RangeLike, typename EachFun> ++auto operator|(const RangeLike&range, Akonadi::detail::ForEach_<EachFun> fun) ++{ ++ std::for_each(std::cbegin(range), std::cend(range), ++ [&fun](const auto &val) { ++ Akonadi::invoke(fun.mFn, val); ++ }); ++ return range; ++} ++ ++// Generic operator| for all ++template<typename RangeLike, typename PredicateFn> ++auto operator|(const RangeLike &range, Akonadi::detail::All_<PredicateFn> fun) ++{ ++ return std::all_of(std::cbegin(range), std::cend(range), fun.mFn); ++} ++ ++// Generic operator| for any ++template<typename RangeLike, typename PredicateFn> ++auto operator|(const RangeLike &range, Akonadi::detail::Any_<PredicateFn> fun) ++{ ++ return std::any_of(std::cbegin(range), std::cend(range), fun.mFn); ++} ++ ++// Generic operator| for keys ++template<typename Container> ++auto operator|(const Container &in, Akonadi::detail::Keys_) ++{ ++ using namespace Akonadi::detail; ++ using OutIt = AssociativeContainerKeyIterator<Container>; ++ return Range<OutIt>(OutIt(in.constKeyValueBegin(), in), OutIt(in.constKeyValueEnd(), in)); ++} ++ ++ ++// Generic operator| for values ++template<typename Container> ++auto operator|(const Container &in, Akonadi::detail::Values_) ++{ ++ using namespace Akonadi::detail; ++ using OutIt = AssociativeContainerValueIterator<Container>; ++ return Range<OutIt>(OutIt(in.constKeyValueBegin(), in), OutIt(in.constKeyValueEnd(), in)); ++} ++ ++ ++namespace Akonadi { ++ ++/// Non-lazily convert given range or container to QVector ++static constexpr auto toQVector = detail::To_<QVector>{}; ++/// Non-lazily convert given range or container to QSet ++static constexpr auto toQSet = detail::To_<QSet>{}; ++/// Non-lazily convert given range or container to QList ++static constexpr auto toQList = detail::To_<QList>{}; ++/// Non-lazily convert given range or container of pairs to QMap ++static constexpr auto toQMap = detail::ToAssoc_<QMap>{}; ++/// Lazily extract values from an associative container ++static constexpr auto values = detail::Values_{}; ++/// Lazily extract keys from an associative container ++static constexpr auto keys = detail::Keys_{}; ++ ++/// Lazily transform each element of a range or container using given transformation ++template<typename TransformFn> ++detail::Transform_<TransformFn> transform(TransformFn &&fn) ++{ ++ return detail::Transform_<TransformFn>{std::forward<TransformFn>(fn)}; ++} ++ ++/// Lazily filters a range or container by applying given predicate on each element ++template<typename PredicateFn> ++detail::Filter_<PredicateFn> filter(PredicateFn &&fn) ++{ ++ return detail::Filter_<PredicateFn>{std::forward<PredicateFn>(fn)}; ++} ++ ++/// Non-lazily call EachFun for each element of the container or range ++template<typename EachFun> ++detail::ForEach_<EachFun> forEach(EachFun &&fn) ++{ ++ return detail::ForEach_<EachFun>{std::forward<EachFun>(fn)}; ++} ++ ++/// Create a range, a view on a container from the given pair fo iterators ++template<typename Iterator1, typename Iterator2, ++ typename It = std::remove_reference_t<Iterator1> ++ > ++detail::Range<It> range(Iterator1 begin, Iterator2 end) ++{ ++ return detail::Range<It>(std::move(begin), std::move(end)); ++} ++ ++/// Non-lazily check that all elements in the range satisfy given predicate ++template<typename Predicate> ++detail::All_<Predicate> all(Predicate &&fn) ++{ ++ return detail::All_<Predicate>{std::forward<Predicate>(fn)}; ++} ++ ++/// Non-lazily check that at least one element in range satisfies the given predicate ++template<typename Predicate> ++detail::Any_<Predicate> any(Predicate &&fn) ++{ ++ return detail::Any_<Predicate>{std::forward<Predicate>(fn)}; ++} ++ ++} // namespace Akonadi ++ ++#endif +diff --git a/src/shared/akscopeguard.h b/src/shared/akscopeguard.h +new file mode 100644 +index 000000000..e371383ee +--- /dev/null ++++ b/src/shared/akscopeguard.h +@@ -0,0 +1,52 @@ ++/* ++ Copyright (C) 2019 Daniel Vrátil <dvra...@kde.org> ++ ++ This library is free software; you can redistribute it and/or modify it ++ under the terms of the GNU Library General Public License as published by ++ the Free Software Foundation; either version 2 of the License, or (at your ++ option) any later version. ++ ++ This library is distributed in the hope that it will be useful, but WITHOUT ++ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or ++ FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public ++ License for more details. ++ ++ You should have received a copy of the GNU Library General Public License ++ along with this library; see the file COPYING.LIB. If not, write to the ++ Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ++ 02110-1301, USA. ++*/ ++ ++#ifndef AKONADI_AKSCOPEGUARD_H_ ++#define AKONADI_AKSCOPEGUARD_H_ ++ ++#include <functional> ++#include <type_traits> ++ ++namespace Akonadi { ++ ++class AkScopeGuard ++{ ++public: ++ template<typename U> ++ AkScopeGuard(U &&fun) ++ : mFun(std::move(fun)) ++ {} ++ ++ AkScopeGuard(const AkScopeGuard &) = delete; ++ AkScopeGuard(AkScopeGuard &&) = default; ++ AkScopeGuard &operator=(const AkScopeGuard &) = delete; ++ AkScopeGuard &operator=(AkScopeGuard &&) = delete; ++ ++ ~AkScopeGuard() ++ { ++ mFun(); ++ } ++ ++private: ++ std::function<void()> mFun; ++}; ++ ++} // namespace Akonadi ++ ++#endif +diff --git a/src/shared/aktraits.h b/src/shared/aktraits.h +new file mode 100644 +index 000000000..67cd60c40 +--- /dev/null ++++ b/src/shared/aktraits.h +@@ -0,0 +1,165 @@ ++/* ++ Copyright (C) 2019 Daniel Vrátil <dvra...@kde.org> ++ ++ This library is free software; you can redistribute it and/or modify it ++ under the terms of the GNU Library General Public License as published by ++ the Free Software Foundation; either version 2 of the License, or (at your ++ option) any later version. ++ ++ This library is distributed in the hope that it will be useful, but WITHOUT ++ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or ++ FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public ++ License for more details. ++ ++ You should have received a copy of the GNU Library General Public License ++ along with this library; see the file COPYING.LIB. If not, write to the ++ Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA ++ 02110-1301, USA. ++*/ ++ ++#ifndef AKONADI_AKTRAITS_H_ ++#define AKONADI_AKTRAITS_H_ ++ ++#include <type_traits> ++#include <utility> ++ ++namespace Akonadi { ++namespace AkTraits { ++ ++namespace detail { ++ ++ /// Helpers from C++17 ++ template<typename ...> ++ using void_t = void; ++ ++ template<typename ...> ++ struct conjunction : std::true_type {}; ++ template<typename T> ++ struct conjunction<T> : T {}; ++ template<typename T, typename ... Ts> ++ struct conjunction<T, Ts...> : std::conditional_t<bool(T::value), conjunction<Ts...>, T> {}; ++ ++ #define DECLARE_HAS_MEBER_TYPE(type_name) \ ++ template<typename T, typename U = void_t<>> \ ++ struct hasMember_##type_name { \ ++ static constexpr bool value = false; \ ++ }; \ ++ \ ++ template<typename T> \ ++ struct hasMember_##type_name<T, void_t<typename T:: type_name>> : std::true_type {}; ++ ++ DECLARE_HAS_MEBER_TYPE(value_type) ++ ++ /// TODO: Use Boost TTI instead? ++ #define DECLARE_HAS_METHOD_GENERIC_IMPL(name, fun, sign) \ ++ template<typename T, typename F = sign> \ ++ struct hasMethod_##name { \ ++ public: \ ++ template<typename UType, UType> \ ++ struct helperClass; \ ++ \ ++ using True = char; \ ++ using False = struct { char dummy_[2]; }; \ ++ \ ++ template<typename U> \ ++ static True helper(helperClass<F, &U::fun>*);\ ++ template<typename> \ ++ static False helper(...); \ ++ public: \ ++ static constexpr bool value = sizeof(helper<T>(nullptr)) == sizeof(True); \ ++ }; ++ ++ #define DECLARE_HAS_METHOD_GENERIC_CONST(fun, R, ...) \ ++ DECLARE_HAS_METHOD_GENERIC_IMPL(fun##_const, fun, R(T::*)(__VA_ARGS__) const) ++ ++ #define DECLARE_HAS_METHOD_GENERIC(fun, R, ...) \ ++ DECLARE_HAS_METHOD_GENERIC_IMPL(fun, fun, R(T::*)(__VA_ARGS__)) ++ ++ DECLARE_HAS_METHOD_GENERIC_CONST(size, int, void) ++ DECLARE_HAS_METHOD_GENERIC(push_back, void, const typename T::value_type &) ++ DECLARE_HAS_METHOD_GENERIC(insert, typename T::iterator, const typename T::value_type &) ++ DECLARE_HAS_METHOD_GENERIC(reserve, void, int) ++ ++ #define DECLARE_HAS_FUNCTION(name, fun) \ ++ template<typename T> \ ++ struct has_##name { \ ++ template<typename U> \ ++ struct helperClass; \ ++ \ ++ using True = char; \ ++ using False = struct { char dummy_[2]; }; \ ++ \ ++ template<typename U> \ ++ static True helper(helperClass<decltype(fun(std::declval<T>()))>*); \ ++ template<typename> \ ++ static False helper(...); \ ++ public: \ ++ static constexpr bool value = sizeof(helper<T>(nullptr)) == sizeof(True); \ ++ }; ++ ++ // For some obscure reason QVector::begin() actually has a default ++ // argument, but QList::begin() does not, thus a regular hasMethod_* check ++ // won't cut it here. Instead we check whether the container object can be ++ // used with std::begin() and std::end() helpers. ++ // Check for constness can be performed by passing "const T" to the type. ++ DECLARE_HAS_FUNCTION(begin, std::begin) ++ DECLARE_HAS_FUNCTION(end, std::end) ++ ++ /// This is a very incomplete set of Container named requirement, but I'm ++ /// too lazy to implement all of them, but this should be good enough to match ++ /// regular Qt containers and /not/ match arbitrary non-container types ++ template<typename T> ++ struct isContainer : conjunction< ++ std::is_constructible<T>, ++ hasMember_value_type<T>, ++ has_begin<T>, ++ has_begin<const T>, ++ has_end<T>, ++ has_end<const T>, ++ hasMethod_size_const<T> ++ > {}; ++ ++ /// Matches anything that is a container and has push_back() method. ++ template<typename T> ++ struct isAppendable : conjunction< ++ isContainer<T>, ++ hasMethod_push_back<T> ++ > {}; ++ ++ /// Matches anything that is a container and has insert() method. ++ template<typename T> ++ struct isInsertable : conjunction< ++ isContainer<T>, ++ hasMethod_insert<T> ++ > {}; ++ ++ /// Matches anything that is a container and has reserve() method. ++ template<typename T> ++ struct isReservable : conjunction< ++ isContainer<T>, ++ hasMethod_reserve<T> ++ > {}; ++} ++ ++template<typename T> ++constexpr bool isAppendable = detail::isAppendable<T>::value; ++ ++template<typename T> ++constexpr bool isInsertable = detail::isInsertable<T>::value; ++ ++template<typename T> ++constexpr bool isReservable = detail::isReservable<T>::value; ++ ++} ++} ++ ++#define AK_PP_CAT_(X, Y) X ## Y ++#define AK_PP_CAT(X, Y) AK_PP_CAT_(X, Y) ++ ++#define AK_REQUIRES(...) \ ++ bool AK_PP_CAT(_ak_requires_, __LINE__) = false, \ ++ std::enable_if_t< \ ++ AK_PP_CAT(_ak_requires_, __LINE__) || (__VA_ARGS__) \ ++ >* = nullptr ++ ++#endif +-- +2.23.0 + diff -Nru akonadi-18.08.3/debian/patches/Fix-unhandled-exception-from-DataStream-operator.patch akonadi-18.08.3/debian/patches/Fix-unhandled-exception-from-DataStream-operator.patch --- akonadi-18.08.3/debian/patches/Fix-unhandled-exception-from-DataStream-operator.patch 1970-01-01 01:00:00.000000000 +0100 +++ akonadi-18.08.3/debian/patches/Fix-unhandled-exception-from-DataStream-operator.patch 2019-08-29 20:58:54.000000000 +0200 @@ -0,0 +1,39 @@ +From 40a43c8860ebfad49b06b8db75ac4f7431d19e49 Mon Sep 17 00:00:00 2001 +From: Filipe Azevedo <pas...@gmail.com> +Date: Fri, 22 Mar 2019 22:11:10 +0100 +Subject: [PATCH 2/2] Fix unhandled exception from DataStream::operator<< + +Summary: Depends on D19982 + +Reviewers: dvratil + +Reviewed By: dvratil + +Subscribers: cfeck, kde-pim + +Tags: #kde_pim + +Differential Revision: https://phabricator.kde.org/D19983 + +(cherry picked from commit 06f1a32b5a5aa3f5a5dcf842e8de1358480fede5) +--- + src/core/connection.cpp | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/core/connection.cpp b/src/core/connection.cpp +index 7eeed3f83..df51240df 100644 +--- a/src/core/connection.cpp ++++ b/src/core/connection.cpp +@@ -334,8 +334,8 @@ void Connection::doSendCommand(qint64 tag, const Protocol::CommandPtr &cmd) + + if (mSocket && mSocket->isOpen()) { + Protocol::DataStream stream(mSocket.data()); +- stream << tag; + try { ++ stream << tag; + Protocol::serialize(mSocket.data(), cmd); + } catch (const Akonadi::ProtocolException &e) { + qCWarning(AKONADICORE_LOG) << "Protocol Exception:" << QString::fromUtf8(e.what()); +-- +2.23.0 + diff -Nru akonadi-18.08.3/debian/patches/ItemSync-skip-handling-remote-items-if-local-changes.patch akonadi-18.08.3/debian/patches/ItemSync-skip-handling-remote-items-if-local-changes.patch --- akonadi-18.08.3/debian/patches/ItemSync-skip-handling-remote-items-if-local-changes.patch 1970-01-01 01:00:00.000000000 +0100 +++ akonadi-18.08.3/debian/patches/ItemSync-skip-handling-remote-items-if-local-changes.patch 2019-08-28 18:40:42.000000000 +0200 @@ -0,0 +1,47 @@ +From 46d5d3fe22f80b0452c0dffe02a48ca30a840697 Mon Sep 17 00:00:00 2001 +From: David Faure <fa...@kde.org> +Date: Sun, 7 Apr 2019 14:07:00 +0200 +Subject: [PATCH 2/2] ItemSync: skip handling remote items if local changes + failed + +Summary: +The infamous "Multiple merge candidates" error would still leave ItemSync +in a forever-stuck state when mRemoteItemsQueue was not empty. + +Reproduced by adding a unittest that calls setFullSyncItems (with +a duplicate item), while the existing unittest for a duplicate item +was callling setIncrementalSyncItems(). + +Test Plan: Unittest passes. + +Reviewers: dvratil, vkrause + +Reviewed By: dvratil + +Subscribers: asn, kde-pim + +Tags: #kde_pim + +Differential Revision: https://phabricator.kde.org/D20243 + +(cherry picked from commit 257598195d09b1022ebf08cd58245cd3493f5a2e) +(cherry picked from commit 4fb179ce46ad3e7da8b25c921b0c8a68f9dfb4ee) +--- + src/core/itemsync.cpp | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/src/core/itemsync.cpp b/src/core/itemsync.cpp +index d01db1a80..538f87574 100644 +--- a/src/core/itemsync.cpp ++++ b/src/core/itemsync.cpp +@@ -442,6 +442,7 @@ void ItemSyncPrivate::slotLocalChangeDone(KJob *job) + { + if (job->error()) { + qCWarning(AKONADICORE_LOG) << "Creating/updating items from the akonadi database failed:" << job->errorString(); ++ mRemoteItemQueue.clear(); // don't try to process any more items after a rollback + } + mPendingJobs--; + mProgress++; +-- +2.23.0 + diff -Nru akonadi-18.08.3/debian/patches/series akonadi-18.08.3/debian/patches/series --- akonadi-18.08.3/debian/patches/series 2019-02-08 23:19:51.000000000 +0100 +++ akonadi-18.08.3/debian/patches/series 2019-08-30 12:26:27.000000000 +0200 @@ -2,3 +2,8 @@ disable_secure_file_priv_check.diff enable_debianabimanager.diff Call-QSqlQuery-finish-on-all-SELECT-queries-when-don.patch +Akonadi-fix-dangling-transaction-after-itemsync-fail.patch +ItemSync-skip-handling-remote-items-if-local-changes.patch +Automatic-recovery-from-Multiple-Merge-Candidates-er.patch +Fix-unhandled-exception-from-DataStream-operator.patch +Backport-missing-files-from-master.patch
