[issue16783] sqlite3 accepts strings it cannot return
New submission from William D. Colburn: This seems to be just like issue6010, which is closed as "not as bug" which is simple irresponsible on the part of Python. The problem is that I can store data into sqlite3 which cannot be retrieved. The data I encountered was a string with a \xa0 in it that was read from urllib2. I deposited that data into an sqlite3 database with no trouble at all. Later, however, sqlite3 would throw an exception when I tried to retrieve the data. Martin v. Löwis says this is user error because sqlite3 assumes UTF-8. It makes no sense that storeable data cannot be retrieved and that the failure of sqlite3 is a user-error. If sqlite3 assumes UTF-8, then the error should have been caught when I did the store because I was not storing UTF-8 data. Accepting bad input and blaming the user later down the line is bad bug handling, especially because the two events might be separated by a tremendous gap of time and code which makes debugging quite hard. I was only able to find the errant character by popping the streamed webpage into emacs as both python and firefox showed me a normal looking string with nothing wrong with it. -- components: Unicode messages: 178189 nosy: William.D..Colburn, ezio.melotti priority: normal severity: normal status: open title: sqlite3 accepts strings it cannot return type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue16783> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16783] sqlite3 accepts strings it cannot (by default) return
William D. Colburn added the comment: I'm ranting against an improper bug closure and bad behavior from the python sqlite3. Treating invalid data as sometimes valid and sometime as invalid is a problem. Problems with a Python module are not anuser-error, they are an implementation error. Users should never have to work around bugs in Python. I'm saddened by your poor response to this issue. On Fri, Dec 28, 2012 at 6:04 PM, Terry J. Reedy wrote: > > Terry J. Reedy added the comment: > > Opening a duplicate issue to rant against the developers is not responsible > behavior. Since you do not seem to understand Martin's 2.x solution, ask for > help on python-list or elsewhere (and read below). The proper fix for > multiple Unicode and text coding problems was and is to use Unicode for text, > as we did and do in 3.x. > > Note that while we link to sqlite3 with a Python interface, and choose that > as the database to link to in the stdlib, we do not control sqlite3 itself. > As documented and as Martin wrote, sqlite *assumes*, by default, that > byte-encoded text handed to it is error-free utf-8 encoded. However, docs and > Martin both say that you can override that assumption by replacing its > text_factory. Sqlite should not reject *any* bytes because anything *could* > be just what the use intended. > > The problem of multiple byte encodings for text and of encoding info getting > separated from encoded bytes is a general one. We constantly get questions on > python-list like "how do I determine the real encoding of a web page if the > encoding information is missing or wrong". We are doing our part to solve it > by using unicode for text and pushing utf-8 as the one, true encoding that > everyone should use whenever possible. > > If you need more explanation, try python-list, as I said before. > > -- > nosy: +terry.reedy > resolution: -> duplicate > status: open -> closed > title: sqlite3 accepts strings it cannot return -> sqlite3 accepts strings it > cannot (by default) return > > ___ > Python tracker > <http://bugs.python.org/issue16783> > ___ -- ___ Python tracker <http://bugs.python.org/issue16783> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16783] sqlite3 accepts strings it cannot (by default) return
William D. Colburn added the comment: What it should do is be consistent (predictable) in it's handling of input and output. If it accepts unicode and outputs unicode, then it should accept unicode and output unicode, not accept garbage and then barf. Valid data should be consistantly valid, and invalid data should be consistently invalid. It is bad behavior for a system to allow invalid data in that can't be recovered or handled. On Sun, Dec 30, 2012 at 9:32 AM, Ezio Melotti wrote: > > Ezio Melotti added the comment: > >> Treating invalid data as sometimes valid and sometime as invalid is a >> problem. > > What is valid is defined by your application. AFAIU sqlite3 defaults to > utf-8, but it's able to work with latin1 data as well. The fact that you are > mixing utf-8 and latin1 is an error in your application, and while it might > be nice if sqlite3 warned you about it, it's not necessarily its > responsibility. Even thought it's a really bad idea, you might want to store > data with different encodings and still being able to retrieve them using the > right text_factory. > > However while trying to reproduce the issue I noticed a possible > inconsistency and reported it on #6010. > > -- > stage: -> committed/rejected > superseder: -> unable to retrieve latin-1 encoded data from sqlite3 > > ___ > Python tracker > <http://bugs.python.org/issue16783> > ___ -- ___ Python tracker <http://bugs.python.org/issue16783> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25463] 2.7.10 glibc double free detected
New submission from William D Colburn: Linking cython against the static libpython in a python install compiled with shared libraries causes a glibc error that a double free or corruption was found while importing site. I don't think it is cython's fault. Happens on RHEL6.6. Linux anotheruvula 2.6.32-504.30.3.el6.x86_64 #1 SMP Thu Jul 9 15:20:47 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) I am attaching a shell script which demonstrates the problem. -- components: Interpreter Core messages: 253354 nosy: wcolburnnrao priority: normal severity: normal status: open title: 2.7.10 glibc double free detected type: crash versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue25463> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25463] 2.7.10 glibc double free detected
William D Colburn added the comment:
I don't see the shell script attached anywhere.
I'll just paste it here!
-cut here-
PYDEST=`pwd`
wget https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tgz
wget http://cython.org/release/Cython-0.23.4.tar.gz
rm -rf Python-2.7.10 Cython-0.23.4 ${PYDEST}/pybug
tar xvf Python-2.7.10.tgz
cd Python-2.7.10
LD_RUN_PATH=$PYDEST/pybug/lib:$LD_LIBRARY_PATH
export LD_RUN_PATH
./configure --prefix=$PYDEST/pybug --enable-shared
make -j 8
make install prefix=$PYDEST/pybug
cd ..
tar xvf Cython-0.23.4.tar.gz
cd Cython-0.23.4
$PYDEST/pybug/bin/python2.7 setup.py install
cd ..
cat > crash.py <
<http://bugs.python.org/issue25463>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25463] 2.7.10 glibc double free detected
William D Colburn added the comment: I tried the test with Python 2.7.11, and it still crashes. -- Added file: http://bugs.python.org/file41454/debug.sh ___ Python tracker <http://bugs.python.org/issue25463> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25463] 2.7.10 glibc double free detected
William D Colburn added the comment:
wcolburn@anotheruvula$ strace doublefree 2>&1
| grep site.py
open("/home/anotheruvula/python/debug/pybug/lib/python2.7/site.py", O_RDONLY) =
3
open("/home/anotheruvula/python/debug/pybug/lib/python2.7/site.pyc", O_RDONLY)
= 4
*** glibc detected *** doublefree: double free or corruption (out):
0x7fa1eb491030 ***
=== Backtrace: =
It's pulling in it's own site.py, not the system site.py when I do it.
--
___
Python tracker
<http://bugs.python.org/issue25463>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
