[issue11456] Documentation csv RFC4180
New submission from Tony Wallace : Change to documentation preamble for csv module: From: There is no “CSV standard”, so the format is operationally defined by the many applications which read and write it. The lack of a standard means that subtle differences often exist in the data produced and consumed by different applications. To: CSV has been used for many years prior to attempts to standardise it in RFC4180. This has resulted in subtle differences often exist in the data produced and consumed by different applications. -- assignee: docs@python components: Documentation messages: 130469 nosy: docs@python, tonywallace priority: normal severity: normal status: open title: Documentation csv RFC4180 type: feature request versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue11456> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
New submission from Tony Wallace <[EMAIL PROTECTED]>: [EMAIL PROTECTED] Python-2.5.1]$ ./configure --prefix=/home/tony/root/usr/local/python-2.5.2 --enable-shared --enable-static [EMAIL PROTECTED] bin]$ file python python: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped [EMAIL PROTECTED] bin]$ uname -a Linux gossamer.ambric.local 2.4.21-40.ELsmp #1 SMP Wed Mar 15 13:46:01 EST 2006 x86_64 x86_64 x86_64 GNU/Linux [EMAIL PROTECTED] bin]$ cat /etc/redhat-release CentOS release 3.6 (Final) -- components: Demos and Tools messages: 68188 nosy: hushp1pt severity: normal status: open title: memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit type: resource usage versions: Python 2.5 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: > how do you know Here is the story, sorry I skipped it before- I was at work then. I was doing the basic build-from-source on RHEL (Centos) Linux, because I don't have root and I need to install it in $HOME/something. I don't try to change anything except the install location. See the ./configure command given before. I tried 2.5.2 because it was the latest + greatest. When I ran "make test", it ran OK until it got to "test_list". Then it got stuck until I killed it. Its very repeatable. In another window, "top" shows the python process's memory growing and growing, until it owns basically all available memory (8GB) after less than a minute or so. Then I tried python 2.5.1, built exactly the same way, and make test works OK- test_list completes in seconds. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: > are you using gcc 4.3 No, I don't think so. [EMAIL PROTECTED] tony]$ gcc --version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-53) Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. And make is definitely using gcc, not something else. > prompt 2.5.1 Good eye. I grabbed the file-python info from python 2.5.1 after I installed it (because 2.5.1 passed make test, I am planning to use that). Its true, I should have got that from the python 2.5.2, which I never did install- but unfortunately, it was deleted too soon. If it is important I can rebuild 2.5.2 again. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: > are you willing Yes, so long as I don't need root, I can follow instructions OK. By the way, the same thing (memory leak 2.5.2) occurred on Centos 4.6, a different Linux box. Lets proceed on that Centos 4.6 box. Here are the particulars: Python-2.5.2]$ file python python: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped [EMAIL PROTECTED] Python-2.5.2]$ gcc --version gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-9) Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [EMAIL PROTECTED] Python-2.5.2]$ cat /etc/[EMAIL PROTECTED] CentOS release 4.6 (Final) [EMAIL PROTECTED] Python-2.5.2]$ uname -a Linux hathi.ambric.local 2.6.9-67.0.15.ELsmp #1 SMP Thu May 8 10:50:20 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux (after make test) test_largefile test_linuxaudiodev test_linuxaudiodev skipped -- Use of the `audio' resource not enabled test_list (stalls here so I dumped it) make: *** [test] Quit (another window, just before I dumped it) top - 01:36:31 up 4 days, 15:07, 5 users, load average: 2.70, 0.84, 0.32 Tasks: 87 total, 1 running, 86 sleeping, 0 stopped, 0 zombie Cpu(s): 3.8% us, 9.7% sy, 0.0% ni, 0.0% id, 86.0% wa, 0.0% hi, 0.5% si Mem: 15639112k total, 15610836k used,28276k free, 280k buffers Swap: 24579440k total, 1533172k used, 23046268k free,57676k cached PID USER PR NI %CPUTIME+ %MEM VIRT RES SHR S COMMAND 17190 tony 24 0 17 1:05.49 90.9 32.2g 13g 7200 D python 69 root 16 06 0:12.97 0.0 000 D kswapd0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: make test not only fails "test_list", it also fails "test_tuple" and "test_userlist". In all cases, the behavior looks the same -- memory expands to > 90% and you kill it. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: in test_list.py, the following shows where it hit the memory leak: [EMAIL PROTECTED] Python-2.5.2]$ LD_LIBRARY_PATH=/home/tony/src/Python-2.5.2/Lib/:$LD_LIBRARY_PATH ./python -v Lib/test/test_list.py # installing zipimport hook import zipimport # builtin <<...>> Python 2.5.2 (r252:60911, Jun 14 2008, 01:31:25) [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. # /home/tony/src/Python-2.5.2/Lib/unittest.pyc matches /home/tony/src/Python-2.5.2/Lib/unittest.py import unittest # precompiled from /home/tony/src/Python-2.5.2/Lib/unittest.pyc dlopen("/home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/time.so", 2); import time # dynamically loaded from /home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/time.so # /home/tony/src/Python-2.5.2/Lib/traceback.pyc matches /home/tony/src/Python-2.5.2/Lib/traceback.py import traceback # precompiled from /home/tony/src/Python-2.5.2/Lib/traceback.pyc import test # directory /home/tony/src/Python-2.5.2/Lib/test # /home/tony/src/Python-2.5.2/Lib/test/__init__.pyc matches /home/tony/src/Python-2.5.2/Lib/test/__init__.py import test # precompiled from /home/tony/src/Python-2.5.2/Lib/test/__init__.pyc # /home/tony/src/Python-2.5.2/Lib/test/test_support.pyc matches /home/tony/src/Python-2.5.2/Lib/test/test_support.py import test.test_support # precompiled from /home/tony/src/Python-2.5.2/Lib/test/test_support.pyc # /home/tony/src/Python-2.5.2/Lib/test/list_tests.pyc has bad mtime import test.list_tests # from /home/tony/src/Python-2.5.2/Lib/test/list_tests.py # wrote /home/tony/src/Python-2.5.2/Lib/test/list_tests.pyc # /home/tony/src/Python-2.5.2/Lib/test/seq_tests.pyc matches /home/tony/src/Python-2.5.2/Lib/test/seq_tests.py import test.seq_tests # precompiled from /home/tony/src/Python-2.5.2/Lib/test/seq_tests.pyc dlopen("/home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/itertools.so", 2); import itertools # dynamically loaded from /home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/itertools.so test_addmul (__main__.ListTest) ... ok test_append (__main__.ListTest) ... ok Terminated === in test_tuple.py, the following shows where it hit the memory leak: [EMAIL PROTECTED] Python-2.5.2]$ LD_LIBRARY_PATH=/home/tony/src/Python-2.5.2/Lib/:$LD_LIBRARY_PATH ./python -v Lib/test/test_tuple.py # installing zipimport hook import zipimport # builtin <<...>> Python 2.5.2 (r252:60911, Jun 14 2008, 01:31:25) [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. # /home/tony/src/Python-2.5.2/Lib/unittest.pyc matches /home/tony/src/Python-2.5.2/Lib/unittest.py import unittest # precompiled from /home/tony/src/Python-2.5.2/Lib/unittest.pyc dlopen("/home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/time.so", 2); import time # dynamically loaded from /home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/time.so # /home/tony/src/Python-2.5.2/Lib/traceback.pyc matches /home/tony/src/Python-2.5.2/Lib/traceback.py import traceback # precompiled from /home/tony/src/Python-2.5.2/Lib/traceback.pyc import test # directory /home/tony/src/Python-2.5.2/Lib/test # /home/tony/src/Python-2.5.2/Lib/test/__init__.pyc matches /home/tony/src/Python-2.5.2/Lib/test/__init__.py import test # precompiled from /home/tony/src/Python-2.5.2/Lib/test/__init__.pyc # /home/tony/src/Python-2.5.2/Lib/test/test_support.pyc matches /home/tony/src/Python-2.5.2/Lib/test/test_support.py import test.test_support # precompiled from /home/tony/src/Python-2.5.2/Lib/test/test_support.pyc # /home/tony/src/Python-2.5.2/Lib/test/seq_tests.pyc matches /home/tony/src/Python-2.5.2/Lib/test/seq_tests.py import test.seq_tests # precompiled from /home/tony/src/Python-2.5.2/Lib/test/seq_tests.pyc dlopen("/home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/itertools.so", 2); import itertools # dynamically loaded from /home/tony/src/Python-2.5.2/build/lib.linux-x86_64-2.5/itertools.so test_addmul (__main__.TupleTest) ... ok Terminated === in test_userlist.py, the following shows where it hit the memory leak: [EMAIL PROTECTED] Python-2.5.2]$ LD_LIBRARY_PATH=/home/tony/src/Python-2.5.2/Lib/:$LD_LIBRARY_PATH ./python -v Lib/test/test_userlist.py # installing zipimport hook import zipimport # builtin <<...>> Python 2.5.2 (r252:60911, Jun 14 2008, 01:31:25) [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. # /home/tony/src/Python-2.5.2/Lib/UserList.pyc matches /home/tony/src/Python-2.5.2/Lib/UserList.py import UserList # precompiled from /home/tony/src/Python-2.5.2/Lib/UserList.pyc # /home/tony/src/Python-2.5.2/Lib/unittest.pyc matches /home/tony
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: tried again with /configure --prefix=/home/tony/root/usr/local/python-2.5.2 --with-tcl --disable-shared No change But I noticed this when it recompiled. Maybe it is related. gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Objects/obmalloc.o Objects/obmalloc.c Objects/obmalloc.c: In function `new_arena': Objects/obmalloc.c:529: warning: comparison is always false due to limited range of data type (code fragment) /* Double the number of arena objects on each allocation. * Note that it's possible for `numarenas` to overflow. */ numarenas = maxarenas ? maxarenas << 1 : INITIAL_ARENA_OBJECTS; if (numarenas <= maxarenas) return NULL;/* overflow */ if (numarenas > PY_SIZE_MAX / sizeof(*arenas)) /* line 529 here */ return NULL;/* overflow */ nbytes = numarenas * sizeof(*arenas); arenaobj = (struct arena_object *)realloc(arenas, nbytes); if (arenaobj == NULL) return NULL; arenas = arenaobj; ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linux 64bit
Tony Wallace <[EMAIL PROTECTED]> added the comment: > Objects/obmalloc.c:529: warning: > comparison is always false due to limited range of data type This compile complaint was definitely introduced in 2.5.2 by source changes from 2.5.1. So, there's a minor problem that could be fixed, anyway. However, replacing the 2.5.2 version of obmalloc.c with the 2.5.1 version and rebuilding (with incremental make this time) did NOT help the memory leak in test_list - I still get it. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3107] test_list uses unreasonable amounts of memory on 64-bit Linux
Tony Wallace <[EMAIL PROTECTED]> added the comment: It worked- I took a patch of r65334, as svn diff -c 65334 "http://svn.python.org/projects/python/branches/release25-maint"; and applied that patch ONLY to a clean release 2.5.2 source, ignoring the patch failure in Misc/NEWS. Built it all over again the same as before (that is, CentOS release 4.4 (Final) Linux manfred 2.6.9-42.0.8.ELsmp #1 SMP Tue Jan 30 12:18:01 EST 2007 x86_64 x86_64 x86_64 GNU/Linux ./configure --prefix=/home/tools/sqa/amd64_rhel4/Python-2.5.2 --enable-shared --build=x86_64-redhat-linux --enable-static ) Now, make test runs all the way through with no difficulty. Thanks, everyone. I consider this closed. Maintainer, please dispose this bug as you think appropriate. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3107> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com