Yeah , the problem is you cannot store unicode .. Its not web2py fault tho . thats how Python works.
You have to "Encode" before storing. So here is how it works: When you read data from a source , decode. When you store data to source , encode After retrieving data from a source (File,Database) and you want to mainipulate that data , decode it. When you've done something with that data Encode it. So when you read data from text file , Decode that text file . --- Do what you want with that file. and then Encode (ever field's values) before inserting into database. Use appropriate encoding , and if you can;t get encoding use UTF-8. On Fri, Mar 9, 2012 at 1:15 PM, weheh <richard_gor...@verizon.net> wrote: > > Did you ever get your unicode issue resolved last year? I've been > having trouble with unicode and was wondering if you had a general > method for dealing with it? I'm uploading text into a db by 2 methods: > first method is reading a text field, 2nd method is uploading a text > file. I need a reliable way of making sure all text is in same utf8 > format. Do you know how to do this? > > > On Jun 1 2011, 5:01 pm, Phyo Arkar <phyo.arkarl...@gmail.com> wrote: > > So the value has to be Endcoded FromUnicodeinto UTF8 before > > inserting? i am sure keys are str. > > > > We cannot storeUnicodevalues ? > > > > On 5/31/11, Massimo Di Pierro <massimo.dipie...@gmail.com> wrote: > > > > > > > > > > > > > > > > > I am not sure there is a problem here. I cannot help without seeing an > > > example of the data you insert. > > > > > If you do: > > > > > db.table.insert(**a) > > > > > a must be a dictionary with str key and str (ut8 encoded) values. You > > > cannot haveunicodefield names. > > > > > Massimo > > > > > On May 31, 2:40 am, Phyo Arkar <phyo.arkarl...@gmail.com> wrote: > > >> I have the exactly same problem here. > > > > >> Can anyone look into it? thanks alot. > > > > >> Stef have you found a solution? > > > > >> On Fri, Nov 5, 2010 at 5:47 PM, Stef Mientki <stef.mien...@gmail.com> > > >> wrote: > > > > >> > I'm trying to insert a record with a filed value > > >> > u"ëLocatie" > > > > >> > and I get an error in sql.Table._insert at the last line: > > >> > return 'INSERT INTO %s(%s) VALUES (%s);' % (sql_t, sql_f, > sql_v) > > > > >> > here the traceback > > >> > File "D:\Data_Python_25\support\Web2Py_DAL_support.py", line 250, > in > > >> > DAL_Table > > >> > Description = Value[3][:-1]) > > >> > File "P:\Web2PY\web2py_src\web2py\gluon\sql.py", line 2035, in > insert > > >> > query = self._insert(**fields) > > >> > File "P:\Web2PY\web2py_src\web2py\gluon\sql.py", line 2028, in > _insert > > >> > return 'INSERT INTO %s(%s) VALUES (%s);' % (sql_t, sql_f, sql_v) > > >> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in > position 35: > > >> > ordinal not in range(128) > > > > >> > I don't understand the problem and know what I'm doing wrong > > > > >> > Wwith print statements I tried to narrow the problem, and in : > > >> > def sql_represent(obj, fieldtype, dbname, db_codec='UTF-8'): > > > > >> > there's is this part > > >> > if isinstance(obj,unicode): > > >> > print '????', type(obj),obj > > >> > if len(obj)>0 : print ord(obj[0]) > > >> > obj = obj.encode(db_codec) > > >> > print '????', type(obj) > > >> > if len(obj)>0 : print ord(obj[0]) > > > > >> > which seems to convert theunicodeto a string with byte values larger > > >> > than 128. > > > > >> > Am I doing something wrong, or is this a bug ? > > >> > And of course far more interesting, how do I solve this problem ? > > > > >> > thanks, > > >> > Stef Mientki >
