One more check... is the encoding of the database UTF-8? you can check with psql -l
2015-04-27 19:01 GMT+02:00 <bodobam...@gmail.com>: > > which postgresql Version? > > PSQL 9.3.6 @ Ubuntu 14.04.2 LTS > > In the meantime, I played around with pgadmin a little bit: > > 1) Using pgadmin I can create records containing 'ἀγοραζε' (the sample > above) or German Umlaute ä, ö, ü, so it's definetly no database problem. > > 2) Going to Appadmin > https://testmachine/testapp/appadmin/select/db?query=db.test.id%3E0 displays > the records without Problem > > 3) As soon as I edit and save, I'll get a > <type 'exceptions.UnicodeDecodeError'> 'ascii' codec can't decode byte > 0xe1 in position 114: ordinal not in range(128) > > > Traceback (most recent call last): > File "/home/www-data/web2py/gluon/restricted.py", line 227, in restricted > exec ccode in environment > File > "/home/www-data/web2py/applications/testapp/controllers/appadmin.py", line > 704, in <module> > File "/home/www-data/web2py/gluon/globals.py", line 393, in <lambda> > self._caller = lambda f: f() > File > "/home/www-data/web2py/applications/testapp/controllers/appadmin.py", line > 343, in update > if form.accepts(request.vars, session): > File "/home/www-data/web2py/gluon/sqlhtml.py", line 1677, in accepts > if pk: > File "/home/www-data/web2py/gluon/packages/dal/pydal/objects.py", line > 2117, in update > uploadfolder = pjoin( > File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", > line 988, in update > raise e > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 114: > ordinal not in range(128) > > > > > > Am Sonntag, 26. April 2015 15:27:06 UTC+2 schrieb Massimo Di Pierro: > >> which postgresql version? I am failing to reproduce this. >> >> On Saturday, 25 April 2015 10:28:07 UTC-5, bodob...@gmail.com wrote: >>> >>> Hey guys, >>> >>> 2.10.3 doesn't seem ready for prime time :-( In fact this bug makes >>> 2.10.3 unusable for postgres users with need for non ASCII-characters.. >>> >>> For reference let's link to the corresponding issue here too: >>> https://github.com/web2py/web2py/issues/910 >>> >>> Is there any solution yet? >>> >>> Bodo >>> >>> Am Samstag, 11. April 2015 16:38:13 UTC+2 schrieb Ian W. Scott: >>> >>>> Thanks Richard. My production server is working fine with 2.9.11. I'll >>>> open an issue on the web2py github repo, since it sounds like that's the >>>> better forum for this. >>>> >>>> Ian >>>> >>>> >>>> On Friday, April 10, 2015 at 3:40:16 PM UTC-4, Richard wrote: >>>>> >>>>> Ok, ok, I guess there is an issue with web2py or pyDAL... Can you test >>>>> this with an earlier version of web2py? If it works consider open an issue >>>>> in web2py on github with condition to reproduce the error and if it coming >>>>> form pyDAL the issue will be migrate to pyDAL repo once the origin of the >>>>> issue is found. >>>>> >>>>> Richard >>>>> >>>>> On Fri, Apr 10, 2015 at 3:24 PM, Ian W. Scott <scot...@gmail.com> >>>>> wrote: >>>>> >>>>>> Also, as I pointed out in my question I can insert the same string >>>>>> (with the same encoding) directly into the postgre database using the raw >>>>>> SQL code that is being sent by pydal to the postgre adapter. I inserted a >>>>>> print statement into adapters/base.py to give me the exact SQL command, >>>>>> copied that SQL command (data and all), then ran it in pgAdmin with no >>>>>> problem. So it can't be a problem with the encoding of the string being >>>>>> received by the pydal. >>>>>> >>>>>> Ian >>>>>> >>>>>> On Friday, April 10, 2015 at 2:05:25 PM UTC-4, Richard wrote: >>>>>>> >>>>>>> Can you please show the code that lead to this error? >>>>>>> >>>>>>> If you hard code this you can't use a regular string (str()) which >>>>>>> is ascii characters only you have to pass by unicode something like >>>>>>> this : >>>>>>> >>>>>>> string = u'ἀγοραζε'.decode(utf-8) >>>>>>> >>>>>>> http://stackoverflow.com/questions/6289474/working- >>>>>>> with-utf-8-encoding-in-python-source >>>>>>> >>>>>>> If the greek character come form user input web2py deal with this to >>>>>>> my knowledge... So I am pretty sure you hard code greek character in >>>>>>> your >>>>>>> code... So you need to read about encoding in python... >>>>>>> >>>>>>> Richard >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Apr 10, 2015 at 1:29 PM, Ian W. Scott <scot...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi folks, >>>>>>>> >>>>>>>> I'm running into an error with 2.10.3 when I try to write unicode >>>>>>>> text (outside the Latin ranges) to a regular string field in a postgre >>>>>>>> database. Since my app is a greek language-learning application this >>>>>>>> error >>>>>>>> breaks things completely for me. >>>>>>>> >>>>>>>> When I try, for example, to insert the string 'ἀγοραζε' I get the >>>>>>>> following error: >>>>>>>> >>>>>>>> Traceback (most recent call last): >>>>>>>> >>>>>>>> File "applications/paideia/modules/paideia.py", line 168, in >>>>>>>> start >>>>>>>> pre_bug_step_id=pre_bug_step_id) >>>>>>>> File "applications/paideia/modules/paideia.py", line 384, in >>>>>>>> reply >>>>>>>> response_string) >>>>>>>> File "applications/paideia/modules/paideia.py", line 655, in >>>>>>>> _record_step >>>>>>>> log_record_id = db.attempt_log.insert(**log_args) >>>>>>>> File "/home/ian/web/web2py/gluon/packages/dal/pydal/objects.py", >>>>>>>> line 745, in insert >>>>>>>> ret = self._db._adapter.insert(self, self._listify(fields)) >>>>>>>> File "/home/ian/web/web2py/gluon/packages/dal/pydal/adapters/ >>>>>>>> base.py", line 729, in insert >>>>>>>> raise e >>>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in >>>>>>>> position 105: ordinal not in range(128) >>>>>>>> >>>>>>>> >>>>>>>> The SQL being sent to the postgre adapter in this case was this: >>>>>>>> >>>>>>>> INSERT INTO attempt_log(name,user_response,step,score,modified_on, >>>>>>>> dt_attempted,in_path,uuid) VALUES (19,'ἀγοραζε',773,1.0,'2015-04-10 >>>>>>>> 12:33:25','2015-04-10 16:33:26',657,'5918a65a-6d6f- >>>>>>>> 42c6-9af2-f5d0e313701e') RETURNING id; >>>>>>>> >>>>>>>> When I execute that SQL command directly on the postgre database it >>>>>>>> doesn't raise any errors and the new row is created fine. >>>>>>>> >>>>>>>> I've created a simple app that duplicates the error and attached >>>>>>>> the packed app here. Since the error doesn't happen with sqlite, I've >>>>>>>> set >>>>>>>> up the app db for a postgre database with the following connection >>>>>>>> details: >>>>>>>> >>>>>>>> username = 'postgres' >>>>>>>> postgre host = 'localhost' >>>>>>>> postgre db_name = 'unicode_bug_test' >>>>>>>> >>>>>>>> I've left the db password field empty in the db.py file for obvious >>>>>>>> reasons (i.e., my local password wouldn't do anyone any good anyway). >>>>>>>> >>>>>>>> If you try to create a new record in the 'lemmas' table using >>>>>>>> appadmin with non-Latin text in the 'lemma' field (something like >>>>>>>> 'βλα') >>>>>>>> you should get the same error. >>>>>>>> >>>>>>>> Ian >>>>>>>> >>>>>>>> -- >>>>>>>> Resources: >>>>>>>> - http://web2py.com >>>>>>>> - http://web2py.com/book (Documentation) >>>>>>>> - http://github.com/web2py/web2py (Source code) >>>>>>>> - https://code.google.com/p/web2py/issues/list (Report Issues) >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "web2py-users" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to web2py+un...@googlegroups.com. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> Resources: >>>>>> - http://web2py.com >>>>>> - http://web2py.com/book (Documentation) >>>>>> - http://github.com/web2py/web2py (Source code) >>>>>> - https://code.google.com/p/web2py/issues/list (Report Issues) >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "web2py-users" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to web2py+un...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- > Resources: > - http://web2py.com > - http://web2py.com/book (Documentation) > - http://github.com/web2py/web2py (Source code) > - https://code.google.com/p/web2py/issues/list (Report Issues) > --- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to web2py+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
