On Wednesday 03 February 2016 11:41 PM, Ximin Luo wrote:
On 03/02/16 13:33, Ximin Luo wrote:
As part of the fix to (k) earlier, I pushed another patch to a side branch of
calendarserver: [1]. You haven't applied it to debian/sid yet, but I think it may be
necessary. To test, you should try to add/remove events with non-ascii unicode characters
in them such as "ßßß". If it fails, then try the patch and if it works please
add it to git as well.
X
[1]
https://anonscm.debian.org/cgit/calendarserver/calendarserver.git/diff/debian/patches/unicode-fixes.patch?h=debian/_wip_sid&id=4476a73ff4df39baa297606e00e66241f371178c
I will see how I could test unicode characters. Also, I do not know the impact
of the change. Should I get it reviewed by upstream?
For me, I could reproduce the bug by adding an event in a client (e.g. icedove+iceowl)
called "ßßß" or some other thing, then the event wouldn't actually be created,
and I could see some decode errors in the caldavd error logs.
The change should have minimal impact - it merely allows that function to
accept more values for self.scheduleTag than it was doing previously (raising
exception when it was of type 'unicode' containing non-ascii chars). But yes it
would be good to ask upstream to review it.
My mistake - the "adding-and-removing events test" that I just described,
triggered the bug that I described in (k) above and was fixed by my patch to pg8000.
OTOH the patch I am talking about here ([1] from above), fixes a bug that
appeared in my log which was *also* a unicode error. I *guess* (but I'm not
sure) that it happens the local client syncs with the server, and *old events*
(that the client and server both already have [note]) *already contains*
unicode characters. The server trieds to generate an etag for it, and 7.0 fails
for this. The exception looks like this:
2016-01-23 22:49:26+0100 [-] [caldav-0] [-] [twext.enterprise.jobs.jobitem#error]
JobItem: 759, WorkItem: 113481 failed: [Failure instance: Traceback: <type
'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xe2 in position
335: ordinal not in range(128)
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1274:unwindGenerator
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1128:_inlineCallbacks
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:4336:upgradeData
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1274:unwindGenerator
2016-01-23 22:49:26+0100 [-] [caldav-0] --- <exception caught here> ---
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:1128:_inlineCallbacks
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:4032:updateDatabase
2016-01-23 22:49:26+0100 [-] [caldav-0]
/usr/lib/python2.7/dist-packages/txdav/caldav/datastore/sql.py:3906:_generateEtag
2016-01-23 22:49:26+0100 [-] [caldav-0] ]
You *might* need to patch
/usr/lib/python2.7/dist-packages/twext/enterprise/jobs/jobitem.py to say
log.error([etc], exc_info=sys.exc_info()) to get this stack trace to show, I
can't remember exactly what I did.
[note] because of (k) you will either need to create such an event in server
<7.0 then upgrade to 7.0, or else install our newer patched version of pg8000.
X
I have been unable to reproduce the unicode bug related to logging. I
created a event with the non-ascii characters you gave in both a fresh
installation and older installation and in both cases it worked perfectly.
The pg8000 fix seems necessary though and that patch has already been
included.
Now the only pending action is to upload pg8000, post which
calendarserver could finally be uploaded. Request you to do at the earliest.
Thanks,
Rahul.
