On 03/02/16 13:33, Ximin Luo wrote: >>> >>> As part of the fix to (k) earlier, I pushed another patch to a side branch >>> of calendarserver: [1]. You haven't applied it to debian/sid yet, but I >>> think it may be necessary. To test, you should try to add/remove events >>> with non-ascii unicode characters in them such as "ßßß". If it fails, then >>> try the patch and if it works please add it to git as well. >>> >>> X >>> >>> [1] >>> https://anonscm.debian.org/cgit/calendarserver/calendarserver.git/diff/debian/patches/unicode-fixes.patch?h=debian/_wip_sid&id=4476a73ff4df39baa297606e00e66241f371178c >>> >> I will see how I could test unicode characters. Also, I do not know the >> impact of the change. Should I get it reviewed by upstream? >> > > For me, I could reproduce the bug by adding an event in a client (e.g. > icedove+iceowl) called "ßßß" or some other thing, then the event wouldn't > actually be created, and I could see some decode errors in the caldavd error > logs. > > The change should have minimal impact - it merely allows that function to > accept more values for self.scheduleTag than it was doing previously (raising > exception when it was of type 'unicode' containing non-ascii chars). But yes > it would be good to ask upstream to review it. >
My mistake - the "adding-and-removing events test" that I just described, triggered the bug that I described in (k) above and was fixed by my patch to pg8000. OTOH the patch I am talking about here ([1] from above), fixes a bug that appeared in my log which was *also* a unicode error. I *guess* (but I'm not sure) that it happens the local client syncs with the server, and *old events* (that the client and server both already have [note]) *already contains* unicode characters. The server trieds to generate an etag for it, and 7.0 fails for this. The exception looks like this: 2016-01-23 22:49:26+0100 [-] [caldav-0] [-] [twext.enterprise.jobs.jobitem#error] JobItem: 759, WorkItem: 113481 failed: [Failure instance: Traceback: <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xe2 in position 335: ordinal not in range(128) 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1274:unwindGenerator 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1128:_inlineCallbacks 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:4336:upgradeData 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1274:unwindGenerator 2016-01-23 22:49:26+0100 [-] [caldav-0] --- <exception caught here> --- 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1128:_inlineCallbacks 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:4032:updateDatabase 2016-01-23 22:49:26+0100 [-] [caldav-0] /usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:3906:_generateEtag 2016-01-23 22:49:26+0100 [-] [caldav-0] ] You *might* need to patch /usr/lib/python2.7/dist-packages/twext/enterprise/jobs/jobitem.py to say log.error([etc], exc_info=sys.exc_info()) to get this stack trace to show, I can't remember exactly what I did. [note] because of (k) you will either need to create such an event in server <7.0 then upgrade to 7.0, or else install our newer patched version of pg8000. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE git://github.com/infinity0/pubkeys.git
