I think this is. Can you please open an issue about it? On Tuesday, 27 March 2018 13:32:15 UTC-5, alexandre.he...@gmail.com wrote: > > Hello All, > > I'm an absolute newbie with web2py so I'm not sure if the issue lies in > web2py/3.6 or my own mistakes... My apologies in advance. > > I encounter this error in a db.my_table.update_or_insert(..) > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 96328: > invalid start byte' > > > What I'm trying to do: > I have a text file that is the representation of a datamodel in a specific > COTS format. > I want to parse it, list the tables to be created and create them actually. > I know that the file countains non-utf-8 characters, that's why I open it > in binary: > > dpm_file = gzip.open(filepath,'rb') > > ... parse it using binary regex: (I imported the 'regex' module) > regexp = regex.compile(b':columns > (\(((?>[^()]+)|(?1))*\))',regex.DOTALL) > header_classes = regexp.findall(one_format_text)[0] > > ... and insert the results in db blob fields: (the error occurs here) > db.dpm_header.update_or_insert(db.dpm_header.dpm == my_dpm_id, dpm = > my_dpm_id, header_classes = header_classes, header_objects = > header_objects) > > I defined blob fields because I thought that it was the way to write > directly in binary in database. > I really wanted to use binary preciselly to avoid to manage the > encoding/decoding issues. > > In addition, this project is an attempt to transport in web2py an older > project that was working perfectly in pure python 3.6.3 (unix and windows) > (except that instead of creating tables in a DB, I created text files in a > filer...). > Is it a issue on web2py/python 3.6? Or my misunderstanding on database > management... > > Thank you for your help! > > Here is the trace > > Traceback (most recent call last): > File "D:\web\zz\web2py\gluon\restricted.py", line 219, in restricted > exec(ccode, environment) > File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" > <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, > line 175, in <module> > File "D:\web\zz\web2py\gluon\globals.py", line 419, in <lambda> > self._caller = lambda f: f() > File "D:\web\zz\web2py\gluon\tools.py", line 3982, in f > return action(*a, **b) > File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" > <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, > line 131, in analyze > header_classes = header_classes) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 811, in > update_or_insert > record = self(_key) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 565, in > __call__ > orderby_on_limitby=False).first() > File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 2250, in > select > return adapter.select(self.query, fields, attributes) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\sqlite.py", line > 82, in select > return super(SQLite, self).select(query, fields, attributes) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 762, in select > return self._select_aux(sql, fields, attributes, colnames) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 741, in _select_aux > return processor(rows, fields, colnames, cacheable=cacheable) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 305, in parse > for row in rows > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 302, in <listcomp> > self._parse( > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 231, in _parse > value = self.parse_value(value, fit, ft, blob_decode) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line > 198, in parse_value > return self.parser.parse(value, field_itype, field_type) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line > 101, in parse > return self.registered[field_itype](value, field_type) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line > 76, in __call__ > return self.call(value, field_type) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line > 73, in _call > return self.f(self.parser, value) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\base.py", line 37, > in _blob > return to_native(b64decode(to_bytes(value))) > File "D:\web\zz\web2py\gluon\packages\dal\pydal\_compat.py", line 82, in > to_native > return obj.decode(charset, errors) > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 96328: > invalid start byte > >
-- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
