[issue28663] Higher virtual memory usage on recent Linux versions

[ProgVal]

New submission from ProgVal:

Hi,

I use `resource.setrlimit(resource.RLIMIT_DATA, ...)` to limit the memory usage 
of a child process.
I noticed that on recent Linux versions, my processes are much more likely to 
raise MemoryError or behave inconsistently (which I believe is caused by a 
MemoryError being silently ignored somewhere).

With Linux 4.7 under Python 3.4 (tested on a Debian 8.0 chroot) and 3.5 (tested 
on Debian Testing and Unstable), the attached script crashes on the `assert 
False`, which it should not encounter since all execution paths go through a 
`q.put`.

With Linux 3.16 under Python 3.4 (tested on a Debian 8.0 virtual machine), the 
attached script terminates without an error.
Even when restricting the memory to 1MB instead of 10MB, it still works.


It may look like I just have to increase the limit to a larger value, eg. 20MB. 
However, when there is more imported modules in the parent process, the 
required value gets incredibly high (eg. 1GB).

--
components: Interpreter Core
files: rlimit_difference_linux_versions.py
messages: 280566
nosy: Valentin.Lorentz
priority: normal
severity: normal
status: open
title: Higher virtual memory usage on recent Linux versions
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file45438/rlimit_difference_linux_versions.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27838] test_os.test_chown() failure on koobs-freebsd-current

[Kubilay Kocak]

Kubilay Kocak added the comment:

It appears something has changed in the past few weeks (with no changes made to 
the buildbot worker).

Now only 3.5 branch (both debug and non-debug) builders are failing, except 
there are now many failing tests, many different errors, and test clean is also 
failing.

Among others errors:

FileExistsError: [Errno 17] File exists: '@test_42517_tmp/TEST1/SUB1/SUB11'
PermissionError: [Errno 1] Operation not permitted: '@test_42517_tmp'
IsADirectoryError: [Errno 21] Is a directory: '@test_42517_tmp'
PermissionError: [Errno 13] Permission denied: 'SUB21'

Attached is the full log.

Looking into some of the test_x_tmp directories referenced, I see a SUB21 
directory created with no permissions

[root@CURRENT-amd64:/usr/home/buildbot/python/3.5.koobs-freebsd-current/build/build/test_python_85793/@test_85793_tmp/TEST1/SUB2.new]
 ls -la
total 5
drwx--  3 buildbot  buildbot5 Nov 11 16:44 .
drwx--  3 buildbot  buildbot4 Nov 11 16:44 ..
lrwx--  1 buildbot  buildbot   11 Nov 11 16:44 broken_link2 -> tmp3/broken
lrwx--  1 buildbot  buildbot  103 Nov 11 16:44 link -> 
/usr/home/buildbot/python/3.5.koobs-freebsd-current/build/build/test_python_85793/@test_85793_tmp/TEST2
d-  2 buildbot  buildbot3 Nov 11 16:44 SUB21

This is observed across all test_python_ directories:

[root@CURRENT-amd64:/usr/home/buildbot/python/3.5.koobs-freebsd-current/build/build]
 find . -perm 000
./test_python_85793/@test_85793_tmp/TEST1/SUB2.new/SUB21
./test_python_96334/@test_96334_tmp/TEST1/SUB2.new/SUB21
./test_python_53788/@test_53788_tmp/TEST1/SUB2.new/SUB21
./test_python_56622/@test_56622_tmp/TEST1/SUB2.new/SUB21
./test_python_38482/@test_38482_tmp/TEST1/SUB2.new/SUB21
./test_python_58380/@test_58380_tmp/TEST1/SUB2.new/SUB21
./test_python_42517/@test_42517_tmp/TEST1/SUB2.new/SUB21

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27838] test_os.test_chown() failure on koobs-freebsd-current

[Kubilay Kocak]

Changes by Kubilay Kocak :


Added file: 
http://bugs.python.org/file45439/koobs-freebsd-current-python-3.5-debug-build-773.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

Final result on speed-python:

haypo@speed-python$ python3 -m perf compare_to 
json_8nov/2016-11-10_15-39-default-8ebaa546a033.json 
2016-11-11_02-13-default-59b91b4e9506.json -G

Slower (12):
- scimark_sparse_mat_mult: 8.71 ms +- 0.19 ms -> 9.28 ms +- 0.12 ms: 1.07x 
slower
- nbody: 244 ms +- 2 ms -> 252 ms +- 4 ms: 1.03x slower
- json_loads: 71.4 us +- 0.8 us -> 72.9 us +- 1.4 us: 1.02x slower
- fannkuch: 1.07 sec +- 0.01 sec -> 1.09 sec +- 0.01 sec: 1.01x slower
- scimark_lu: 502 ms +- 19 ms -> 509 ms +- 12 ms: 1.01x slower
- chaos: 302 ms +- 3 ms -> 305 ms +- 3 ms: 1.01x slower
- xml_etree_iterparse: 224 ms +- 3 ms -> 226 ms +- 6 ms: 1.01x slower
- regex_dna: 299 ms +- 1 ms -> 300 ms +- 1 ms: 1.00x slower
- pickle_list: 9.21 us +- 0.33 us -> 9.24 us +- 0.56 us: 1.00x slower
- crypto_pyaes: 245 ms +- 1 ms -> 246 ms +- 2 ms: 1.00x slower
- meteor_contest: 219 ms +- 1 ms -> 219 ms +- 1 ms: 1.00x slower
- unpack_sequence: 128 ns +- 2 ns -> 128 ns +- 0 ns: 1.00x slower

Faster (39):
- logging_silent: 997 ns +- 40 ns -> 803 ns +- 13 ns: 1.24x faster
- regex_effbot: 6.16 ms +- 0.24 ms -> 5.17 ms +- 0.27 ms: 1.19x faster
- mako: 45.9 ms +- 0.7 ms -> 42.9 ms +- 0.6 ms: 1.07x faster
- xml_etree_process: 253 ms +- 4 ms -> 237 ms +- 4 ms: 1.07x faster
- call_simple: 13.9 ms +- 0.3 ms -> 13.1 ms +- 0.4 ms: 1.06x faster
- spectral_norm: 274 ms +- 2 ms -> 260 ms +- 2 ms: 1.05x faster
- xml_etree_generate: 300 ms +- 4 ms -> 285 ms +- 5 ms: 1.05x faster
- call_method_slots: 17.1 ms +- 0.2 ms -> 16.2 ms +- 0.3 ms: 1.05x faster
- telco: 21.8 ms +- 0.5 ms -> 20.7 ms +- 0.3 ms: 1.05x faster
- call_method: 17.3 ms +- 0.3 ms -> 16.5 ms +- 0.2 ms: 1.05x faster
- pickle_pure_python: 1.42 ms +- 0.02 ms -> 1.36 ms +- 0.03 ms: 1.04x faster
- pathlib: 51.9 ms +- 0.8 ms -> 50.6 ms +- 0.4 ms: 1.03x faster
- xml_etree_parse: 295 ms +- 8 ms -> 287 ms +- 7 ms: 1.03x faster
- chameleon: 31.0 ms +- 0.3 ms -> 30.2 ms +- 0.2 ms: 1.03x faster
- deltablue: 19.3 ms +- 0.3 ms -> 18.8 ms +- 0.2 ms: 1.02x faster
- django_template: 484 ms +- 4 ms -> 472 ms +- 5 ms: 1.02x faster
- call_method_unknown: 18.7 ms +- 0.2 ms -> 18.3 ms +- 0.2 ms: 1.02x faster
- html5lib: 261 ms +- 6 ms -> 256 ms +- 6 ms: 1.02x faster
- unpickle_pure_python: 973 us +- 12 us -> 954 us +- 15 us: 1.02x faster
- regex_v8: 47.6 ms +- 0.8 ms -> 46.7 ms +- 0.4 ms: 1.02x faster
- richards: 202 ms +- 4 ms -> 198 ms +- 5 ms: 1.02x faster
- logging_simple: 37.8 us +- 0.6 us -> 37.1 us +- 0.4 us: 1.02x faster
- sympy_integrate: 50.8 ms +- 0.9 ms -> 49.9 ms +- 1.4 ms: 1.02x faster
- dulwich_log: 189 ms +- 2 ms -> 186 ms +- 1 ms: 1.02x faster
- sqlalchemy_declarative: 343 ms +- 3 ms -> 339 ms +- 3 ms: 1.01x faster
- hexiom: 25.0 ms +- 0.1 ms -> 24.7 ms +- 0.1 ms: 1.01x faster
- logging_format: 44.6 us +- 0.6 us -> 44.1 us +- 0.6 us: 1.01x faster
- 2to3: 787 ms +- 4 ms -> 777 ms +- 4 ms: 1.01x faster
- tornado_http: 440 ms +- 4 ms -> 435 ms +- 4 ms: 1.01x faster
- json_dumps: 30.7 ms +- 0.4 ms -> 30.5 ms +- 0.3 ms: 1.01x faster
- go: 637 ms +- 10 ms -> 632 ms +- 8 ms: 1.01x faster
- regex_compile: 397 ms +- 2 ms -> 394 ms +- 3 ms: 1.01x faster
- nqueens: 266 ms +- 2 ms -> 264 ms +- 2 ms: 1.01x faster
- python_startup: 16.8 ms +- 0.0 ms -> 16.7 ms +- 0.0 ms: 1.01x faster
- python_startup_no_site: 9.91 ms +- 0.01 ms -> 9.86 ms +- 0.01 ms: 1.01x faster
- scimark_sor: 513 ms +- 13 ms -> 510 ms +- 8 ms: 1.01x faster
- raytrace: 1.41 sec +- 0.02 sec -> 1.40 sec +- 0.02 sec: 1.00x faster
- genshi_text: 95.2 ms +- 1.1 ms -> 94.7 ms +- 0.8 ms: 1.00x faster
- sympy_str: 529 ms +- 5 ms -> 528 ms +- 4 ms: 1.00x faster

Benchmark hidden because not significant (13): float, genshi_xml, pickle, 
pickle_dict, pidigits, scimark_fft, scimark_monte_carlo, sqlalchemy_imperative, 
sqlite_synth, sympy_expand, sympy_sum, unpickle, unpickle_list

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28664] test_bz2 fails with BrokenPipeError when bunzip2 is missing

[Xavier de Gaye]

New submission from Xavier de Gaye:

bunzip2 is missing on Android and the following tests of test_bz2 fail 
randomly: test_implicit_binary_modes, test_binary_modes and testAppend, with 
the following backtrace:
ERROR: testAppend (test.test_bz2.BZ2FileTest)
--
Traceback (most recent call last):
File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_bz2.py", line 
302, in testAppend
self.assertEqual(self.decompress(f.read()), self.TEXT * 2)
File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_bz2.py", line 88, 
in decompress
pop.stdin.close()
BrokenPipeError: [Errno 32] Broken pipe

Patch attached.

--
components: Tests
files: decompress.patch
keywords: patch
messages: 280569
nosy: nadeem.vawda, xdegaye
priority: normal
severity: normal
stage: patch review
status: open
title: test_bz2 fails with BrokenPipeError when bunzip2 is missing
type: behavior
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45440/decompress.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27838] test_os.test_chown() failure on koobs-freebsd-{current, 9}

[Kubilay Kocak]

Kubilay Kocak added the comment:

I also note *one* failure on koobs-freebsd-9 on 3.x and 3.6 branches, identical 
errors:

Nov 09 14:53 c27269c0d619... failure AMD64 FreeBSD 9.x 3.x #5304 Failed test
Nov 09 14:53 b671ac7ae620... failure AMD64 FreeBSD 9.x 3.6 #282  Failed test

==
ERROR: test_chown (test.test_os.ChownFileTests)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.6.koobs-freebsd9/build/Lib/test/test_os.py", line 
1200, in test_chown
os.chown(support.TESTFN, uid, gid_1)
PermissionError: [Errno 1] Operation not permitted: '@test_83654_tmp'

--


==
ERROR: test_chown (test.test_os.ChownFileTests)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/test/test_os.py", line 
1200, in test_chown
os.chown(support.TESTFN, uid, gid_1)
PermissionError: [Errno 1] Operation not permitted: '@test_93884_tmp'

--

I cannot explain what would cause persistent failure on one host and 
intermittent failure on another, except perhaps different host resources 
(cpu/mem) creating favourable conditions on one, but not the other.

--
title: test_os.test_chown() failure on koobs-freebsd-current -> 
test_os.test_chown() failure on koobs-freebsd-{current,9}

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23839] Clear caches after every test

[Roundup Robot]

Roundup Robot added the comment:

New changeset bc81f2137706 by Serhiy Storchaka in branch '2.7':
Issue #23839: Various caches now are cleared before running every test file.
https://hg.python.org/cpython/rev/bc81f2137706

New changeset 89776a40e0ec by Serhiy Storchaka in branch '3.5':
Issue #23839: Various caches now are cleared before running every test file.
https://hg.python.org/cpython/rev/89776a40e0ec

New changeset c89f213b21e8 by Serhiy Storchaka in branch '3.6':
Issue #23839: Various caches now are cleared before running every test file.
https://hg.python.org/cpython/rev/c89f213b21e8

New changeset 5d1067e89717 by Serhiy Storchaka in branch 'default':
Issue #23839: Various caches now are cleared before running every test file.
https://hg.python.org/cpython/rev/5d1067e89717

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28663] Higher virtual memory usage on recent Linux versions

[ProgVal]

Changes by ProgVal :


--
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19398] test_trace fails with -S

[Roundup Robot]

Roundup Robot added the comment:

New changeset db220f2df5a9 by Serhiy Storchaka in branch '3.5':
Issue #19398: Extra slash no longer added to sys.path components in case of
https://hg.python.org/cpython/rev/db220f2df5a9

New changeset 1a88baaed7a0 by Serhiy Storchaka in branch '3.6':
Issue #19398: Extra slash no longer added to sys.path components in case of
https://hg.python.org/cpython/rev/1a88baaed7a0

New changeset 82607e7c24c7 by Serhiy Storchaka in branch 'default':
Issue #19398: Extra slash no longer added to sys.path components in case of
https://hg.python.org/cpython/rev/82607e7c24c7

New changeset 237ef36fb1bb by Serhiy Storchaka in branch '2.7':
Issue #19398: Extra slash no longer added to sys.path components in case of
https://hg.python.org/cpython/rev/237ef36fb1bb

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28664] test_bz2 fails with BrokenPipeError when bunzip2 is missing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Wouldn't be better to check once the existence of the bunzip2 executable? 
Proposed patch also simplifies invocation of the bunzip2 command.

--
nosy: +serhiy.storchaka
versions: +Python 3.5
Added file: http://bugs.python.org/file45441/test_bz2-cmdline-bunzip2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19398] test_trace fails with -S

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Raymond Hettinger]

New submission from Raymond Hettinger:

The STORE_FAST, LOAD_FAST, and LOAD_DEREF opcodes all use fast macros for 
variable access.  This patch harmonizes STORE_DEREF to follow the same pattern.

Both the C code and the generated assembly look nicer.  Gives an approx 40% 
speed-up (using both Clang and GCC-6) on the "store_nonlocal" portion of the 
variable access benchmark at http://code.activestate.com/recipes/577834
The eliminates the nonlocal speed penalty, making cell variable updates run 
nearly as fast as updates to locals.

--
assignee: serhiy.storchaka
components: Interpreter Core
files: fastcell.diff
keywords: patch
messages: 280574
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF
type: performance
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45442/fastcell.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28666] Make test.support.rmtree() able to remove non-writable directories

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Some tests create directories with disabled writing or even reading and 
listing. If them fail test.support.rmtree() can't remove such directories. This 
cause failures in other tests.

Proposed patch makes test.support.rmtree() able to remove such directories. If 
some operation fails it try to change the mode of corresponding directory and 
repeat the try.

--
components: Tests
files: test-support-rmtree.patch
keywords: patch
messages: 280575
nosy: ezio.melotti, michael.foord, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Make test.support.rmtree() able to remove non-writable directories
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45443/test-support-rmtree.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM. This saves function call and INCREF/DECREF pair.

What about DELETE_DEREF? PyCell_Set() also is used in unicode_concatenate().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Raymond Hettinger]

Raymond Hettinger added the comment:

Thanks for the quick review.  I'll look at the other two cases when I get a 
chance.

--
assignee: serhiy.storchaka -> rhettinger
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Roundup Robot]

Roundup Robot added the comment:

New changeset d78d45436753 by Raymond Hettinger in branch '3.6':
Issue #28665: Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF giving a 40% 
speedup.
https://hg.python.org/cpython/rev/d78d45436753

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23839] Clear caches after every test

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thanks for the review Raymond.

--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24329] __qualname__ and __slots__

[Xiang Zhang]

Xiang Zhang added the comment:

v3 updates the test cases.

--
Added file: http://bugs.python.org/file45444/slots_special_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28667] FD_SETSIZE is unsigned on FreeBSD

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Seems FreeBSD is the only platforms with unsigned FD_SETSIZE. On NetBSD, Linux, 
and OpenBSD it is defined as just int. [1]

This causes compiler warnings on FreeBSD. [2]

Modules/selectmodule.c:72: warning: comparison between signed and unsigned
Modules/selectmodule.c:112: warning: comparison between signed and unsigned
Modules/selectmodule.c:123: warning: comparison between signed and unsigned
Modules/ossaudiodev.c:476: warning: comparison between signed and unsigned

Proposed patch silences warnings.

[1] https://lists.freebsd.org/pipermail/freebsd-standards/2012-July/002410.html
[2] 
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/5310/steps/compile/logs/warnings%20%2817%29

--
components: FreeBSD
files: FD_SETSIZE.patch
keywords: patch
messages: 280581
nosy: koobs, serhiy.storchaka, skrah
priority: normal
severity: normal
stage: patch review
status: open
title: FD_SETSIZE is unsigned on FreeBSD
type: compile error
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45445/FD_SETSIZE.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28664] test_bz2 fails with BrokenPipeError when bunzip2 is missing

[Xavier de Gaye]

Xavier de Gaye added the comment:

Much better indeed :)
With this patch, the test runs fine on Android.
I left some minor comments on rietveld.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Steve Dower]

Steve Dower added the comment:

I think so. I don't remember if anything else is needed to make it work. Were 
you able to test it? Or I'll give it a quick go on the build machine once 
you've committed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7ec45e7d2194 by Serhiy Storchaka in branch 'default':
Merge from 3.6 (issue #28665).
https://hg.python.org/cpython/rev/7ec45e7d2194

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28661] Fix code example in Python 3.5 telnetlib documentation

[R. David Murray]

R. David Murray added the comment:

Hmm.  I'm surprised that OSX would be different.  I didn't actually experiment 
to confirm it on linux, either.

I could be taken as a "teaching opportunity" to talk about exact match vs 
read_eager, in a comment before or after the example, if someone wants to 
propose a doc patch.  I'm not opposed to changing the example, but an 
explanation would probably be good either way.

Maybe Jack will have an opinion.

--
assignee: docs@python -> 
nosy: +jackdied
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You forgot to merge and created new head.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recent changes in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Guido, here is the new patch with your corrections. I have some questions in 
Rietveld, but those are about mypy, you could apply the patch now.

--
Added file: http://bugs.python.org/file45446/recent-typing-docs-v5.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recent changes in typing.py

[Ivan Levkivskyi]

Changes by Ivan Levkivskyi :


Added file: http://bugs.python.org/file45447/recent-typing-docs-v6.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28664] test_bz2 fails with BrokenPipeError when bunzip2 is missing

[Roundup Robot]

Roundup Robot added the comment:

New changeset 648cd8450f4f by Serhiy Storchaka in branch '3.5':
Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
https://hg.python.org/cpython/rev/648cd8450f4f

New changeset 9184f7f11b30 by Serhiy Storchaka in branch '3.6':
Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
https://hg.python.org/cpython/rev/9184f7f11b30

New changeset 969e85a7a943 by Serhiy Storchaka in branch 'default':
Issue #28664: test_bz2 now works on non-Windows platforms without bunzip2
https://hg.python.org/cpython/rev/969e85a7a943

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28668] instanciation of multiprocessing.Queue raises ImportError in test_logging

[Xavier de Gaye]

New submission from Xavier de Gaye:

Occurs on Android that has a broken shared semaphore implementation (issue 
26924).
Patch attached.

One of the backtraces:

==
ERROR: test_handle_called_with_mp_queue (test.test_logging.QueueListenerTest)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/multiprocessing/synchronize.py", 
line 29, in 
from _multiprocessing import SemLock, sem_unlink
ImportError: cannot import name 'sem_unlink'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/unittest/mock.py", line 1179, 
in patched
return func(*args, **keywargs)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 
3130, in test_handle_c
alled_with_mp_queue
log_queue = multiprocessing.Queue()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/multiprocessing/context.py", 
line 102, in Queue
return Queue(maxsize, ctx=self.get_context())
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/multiprocessing/queues.py", 
line 39, in __init__
from .synchronize import SEM_VALUE_MAX as maxsize
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/multiprocessing/synchronize.py", 
line 34, in 
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, 
therefore, the required sync
hronization primitives needed will not function, see issue 3770.

--
components: Tests
files: test__multiprocessing_queue.patch
keywords: patch
messages: 280589
nosy: vinay.sajip, xdegaye
priority: normal
severity: normal
stage: patch review
status: open
title: instanciation of multiprocessing.Queue raises ImportError in test_logging
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45448/test__multiprocessing_queue.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Honor]

New submission from Honor:

Hello EveryOne,

Payload : 12**62**6
Test script:

import math
math.log10(12**62**6)

Program is looping. I tested apache server and flask web framework.
Result:
Frozen in frost. Cpu usage : %90-99 , system runs but server shutdowns.

Author : Onur TAŞLIOĞLU

--
messages: 280590
nosy: Stone
priority: normal
severity: normal
status: open
title: Math Library Dos Attack

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Mark Dickinson]

Mark Dickinson added the comment:

Please can you give more details about why you consider this a problem?

Yes, some computations take a long time. I fail to see why this is an issue.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Honor]

Honor added the comment:

Very very very long and the server unreachable all path.

On Fri, Nov 11, 2016 at 7:08 PM, Mark Dickinson 
wrote:

>
> Mark Dickinson added the comment:
>
> Please can you give more details about why you consider this a problem?
>
> Yes, some computations take a long time. I fail to see why this is an
> issue.
>
> --
> nosy: +mark.dickinson
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28668] instanciation of multiprocessing.Queue raises ImportError in test_logging

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The decorator doesn't need arguments. I would make it similar to 
skip_unless_symlink.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28664] test_bz2 fails with BrokenPipeError when bunzip2 is missing

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26929] android: test_strptime fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

New patch following Serhiy comments.

--
Added file: http://bugs.python.org/file45449/exclude_ymd_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Mark Dickinson]

Mark Dickinson added the comment:

Thanks. The solution here is "Don't do that, then." That is, don't allow this 
code to execute on your server in the first place.

At a guess, you've got a multithreaded server that's executing the given code 
on one thread, while continuing to listen for connections on another. Now the 
problem is not only that the power computation takes a long time, but also that 
the slow part all happens in a single bytecode instruction, so the GIL never 
gets released while the power operation is in progress, and no other threads 
can run.

In theory it might be possible to rework the power operation to release the GIL 
now and then, but even if we did that there are plenty of other examples in the 
language that are going to have a similar effect (running for a long time 
without releasing the GIL). Changing all those isn't particularly practical.

IOW, I'm afraid this isn't a problem with the core Python language; it's a 
problem with how you're using it: you want to think very carefully before 
allowing arbitrary untrusted code to execute on your server (if that's what 
you're doing), for reasons exactly like this one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28658] MacOsX idle don't run

[Ned Deily]

Changes by Ned Deily :


--
assignee:  -> terry.reedy
components: +IDLE
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Honor]

Honor added the comment:

I will take a video on this subject.
Then I will say the end result.

Thanks a lot.

On Fri, Nov 11, 2016 at 7:27 PM, Mark Dickinson 
wrote:

>
> Mark Dickinson added the comment:
>
> Thanks. The solution here is "Don't do that, then." That is, don't allow
> this code to execute on your server in the first place.
>
> At a guess, you've got a multithreaded server that's executing the given
> code on one thread, while continuing to listen for connections on another.
> Now the problem is not only that the power computation takes a long time,
> but also that the slow part all happens in a single bytecode instruction,
> so the GIL never gets released while the power operation is in progress,
> and no other threads can run.
>
> In theory it might be possible to rework the power operation to release
> the GIL now and then, but even if we did that there are plenty of other
> examples in the language that are going to have a similar effect (running
> for a long time without releasing the GIL). Changing all those isn't
> particularly practical.
>
> IOW, I'm afraid this isn't a problem with the core Python language; it's a
> problem with how you're using it: you want to think very carefully before
> allowing arbitrary untrusted code to execute on your server (if that's what
> you're doing), for reasons exactly like this one.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Nick Coghlan]

Nick Coghlan added the comment:

Carl's patch looks good to me, but my one request in relation to the __slots__ 
situation would be to give it a custom error message better indicating that 
lazy initialization of pre-assigned instance slots isn't supported.

Currently that case just lets the underlying AttributeError escape, which is 
going to be thoroughly cryptic for folks that try it and may look like an 
accidental oversight rather than a deliberate design decision:

>>> class NoDict:
... __slots__ = ()
... 
>>> NoDict().__dict__
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'NoDict' object has no attribute '__dict__'

Suggested error message:

TypeError: No '__dict__' attribute on 'objtype' instance to cache 
'attrname' property.

Essentially, this line:

val = instance.__dict__[self.func.__name__] = self.func(instance)

would become:

attrname = self.func.__name__
try:
cache = instance.__dict__
except AttributeError:
msg = f"No '__dict__' attribute on {type(instance).__name__!r} instance 
to cache {attrname!r} property."
raise TypeError(msg) from None
val = cache[attrname] = self.func(instance)


I believe a future C implementation could potentially be reworked to be 
__slots__ compatible, but I'd have to go read the source code to be sure, and I 
don't think that's necessary.

Note: the class machinery itself already detects actual name conflicts between 
slot and method definitions:

>>> class SlotConflict:
... __slots__ = ("attr")
... @property
... def attr(self):
... return 42
... 
Traceback (most recent call last):
  File "", line 1, in 
ValueError: 'attr' in __slots__ conflicts with class variable

The requested runtime check for the `__dict__` AttributeError covers the case 
where __slots__ is defined, and the cached property *isn't* listed as one of 
the instance attributes, but neither is __dict__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Roundup Robot]

Roundup Robot added the comment:

New changeset 2776720f549c by Terry Jan Reedy in branch '2.7':
Issue #27854: Include idlelib/help.html in 2.7 Windows installer.
https://hg.python.org/cpython/rev/2776720f549c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Terry J. Reedy]

Terry J. Reedy added the comment:

Done.  I plan to test by downloading and installing the release candidate.  If 
you can first test now, that would be great.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Carl Meyer added the comment:

Makes sense, Nick, thanks. The current error message for that situation is 
pretty cryptic indeed. I'll update the patch with your suggestion. Do you think 
a documentation note about slots is also warranted?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Syed Suhail Ahmed]

Syed Suhail Ahmed added the comment:

So from what I have understood, manager.attach_mock must raise an Exception 
when it is called with a wrong attribute, since the patch is called with 
autospec=True and you cannot call a mock with non existing attributes.Is that 
correct?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Please no need to rush. We have 1.5 years to do this right.

I believe supporting __slots__ is very important. But this is not easy task. 
Seems this requires C implementation.

We should also design the behavior in case of setting or deleting cached 
property.

How about just methods without arguments? Some APIs use methods without 
arguments instead of properties. Wouldn't be better to design a decorator that 
memoizes both properties and methods without arguments?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28663] Higher virtual memory usage on recent Linux versions

[James Lu]

Changes by James Lu :


--
nosy: +James Lu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28658] IDLE: catch user cfg file error and improve error message

[Terry J. Reedy]

Terry J. Reedy added the comment:

The exception message says that your problem is a bad byte in one of the config 
files.  The most likely culprit is the one you edited, which I presume is 
~/.idlerc/config-highlight.cfg.  "Invalid start byte' suggests that it tried to 
decode as utf-8, but you used a non-ascii char and saved with some other 
encoding.  I suggest you stick with ascii only for theme names.  The set of 
theme item names should match the all-ascii set used in 
idlelib/config-highlight.def.

For future reference: a crash is a segfault or Mac equivalent, without a python 
traceback.  Uploaded text files should be plain ascii or utf-8 text, 
uncompressed and not wrapped.  "I have a problem.  Help me" requests should 
normally go to python-list or other forums.  The tracker is for bug reports and 
enhancement requests.

In this case, I decided to make this a bug and enhancement issue and changed 
the title accordingly.  The bug is that IDLE stopped instead of continuing 
without the user configuration, the same as it would if there were no file.  I 
propose to catch the exception and replace the traceback with the file name and 
error (the enhancement).  At least for user config files, IDLE should then 
continue (the bugfix).  The revised message will be something like the 
following.

"Unable to read .../.idlerc/config-highlight.cfg.
UnicodeDecodeError: ...
IDLE will continue without this user config file."

--
stage:  -> test needed
title: MacOsX idle don't run -> IDLE: catch user cfg file error and improve 
error message
type: crash -> behavior
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[Raymond Hettinger]

Raymond Hettinger added the comment:

I think this should be marked as "not a bug" as closed.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28669] Math Library Dos Attack

[STINNER Victor]

STINNER Victor added the comment:

> Very very very long and the server unreachable all path.

If a server wants to allow users to run arbitrary code, a sandbox protecting 
the server must be used: limit CPU usage, limit total duration (time), etc.

"while 1: pass" is another simple snippet to eat server resources.

I agree, it's not a bug, it's a feature.

--
nosy: +haypo
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

> - json_loads: 71.4 us +- 0.8 us -> 72.9 us +- 1.4 us: 1.02x slower

Hum, sadly this benchmark is still unstable after my change 59b91b4e9506 ("Mark 
hot functions using __attribute__((hot))", oops, I wanted to write Mark, not 
Make :-/).

This benchmark is around 63.4 us during many months, whereas it reached 72.9 us 
at rev 59b91b4e9506, and the new run (also using hot attribute) gone back to 
63.0 us...

I understand that json_loads depends on the code placement of some other 
functions which are not currently marked with the hot attribute.

https://speed.python.org/timeline/#/?exe=4&ben=json_loads&env=1&revs=50&equid=off&quarts=on&extr=on

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28618] Decorate hot functions using __attribute__((hot)) to optimize Python

[STINNER Victor]

STINNER Victor added the comment:

> - scimark_sparse_mat_mult: 8.71 ms +- 0.19 ms -> 9.28 ms +- 0.12 ms: 1.07x 
> slower

Same issue on this benchmark:

* average on one year: 8.8 ms
* peak at rev 59b91b4e9506: 9.3 ms
* run after rev 59b91b4e9506: 9.0 ms

The benchmark is unstable, but the difference is small, especially compared to 
the difference of call_method without the hot attribute.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28668] instanciation of multiprocessing.Queue raises ImportError in test_logging

[Davin Potts]

Changes by Davin Potts :


--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Davin Potts]

Changes by Davin Potts :


--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Changes by Carl Meyer :


Added file: http://bugs.python.org/file45450/cached_property.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Carl Meyer added the comment:

Uploaded a patch updated per Nick's comment.

Not opposed to waiting to see if someone is motivated to implement a version in 
C that supports __slots__, but if that doesn't happen by the Python 3.7 feature 
deadline, I don't think it should block this proven version.

It also occurred to me that we could probably support __slots__ in pure Python 
without harming the non-slots case by implementing a fallback cache in the 
descriptor itself, keyed by instance in a WeakKeyDictionary. I don't love 
having the behavior differ so much between the slots and non-slots case, but 
maybe it's better than not supporting slots at all.

Re setting and deleting: under the current patch, if you set or delete a cached 
property, you set or delete the cached value. I think this is fine and useful 
behavior, but it could perhaps be added explicitly to the documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Carl Meyer added the comment:

(We wouldn't be able to match the set/delete behavior in a slots-supporting 
fallback implemented in Python.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Daniel Greenfeld]

Daniel Greenfeld added the comment:

I'm delighted to see a patch submitted, but I'm concerned that it isn't thread 
safe. This was implemented in the cached-property package I maintain:

* https://github.com/pydanny/cached-property/issues/6
* https://github.com/pydanny/cached-property/pull/9
* https://github.com/pydanny/cached-property#working-with-threads

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28658] IDLE: catch user cfg file error and improve error message

[Terry J. Reedy]

Terry J. Reedy added the comment:

This is effectively a duplicate of #21973.  In both issues, an error in the 
file causes an exception that is not caught, but should be.

--
resolution:  -> duplicate
stage: test needed -> resolved
status: open -> closed
superseder:  -> Idle should not quit on corrupted user config files

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27542] Segfault in gcmodule.c:360 visit_decref

[Lee Garrett]

Lee Garrett added the comment:

In case someone reaches this bug report via search engine:

apt-get remove python-cffi-backend

fixed this problem for me.

--
nosy: +lgarrett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28670] PEP 235: Implement on every POSIX system, and clean up the C code for `import'

[Michael Witten]

New submission from Michael Witten:

The attached file, `pep-235-on-posix.export', contains 3 patches; the file 
includes the intended commit messages and authorship information. To apply 
these patches, save the file to:

  /path/to/pep-235-on-posix.export

and then run the following from within your Mercurial repository:

  hg import /path/to/pep-235-on-posix.export

I ran `make test' or `./python -m test.regrtest' multiple times, and all but 
one attempt succeeded: There was a transient and probably unrelated error that 
occured whilst running `test_thread'.

Only the initial patch alters functionality; the other patches, though 
structurally useful, are a matter of essentially opportunistic aesthetic 
reconstruction.

Here is the commit message of the initial patch (the formatting of which may be 
mangled here):

8<--8<--8<--8<

PEP 235: Extend to every POSIX system the case-sensitivity semantics for 
`import'

(Note: As per PEP 235, this explicit checking of case may be effectively 
disabled by defining the environment variable `PYTHONCASEOK'.)

>From time to time, even a user of a sane system has been known [to be forced] 
>to use an insane file system; the semantics of PEP 235 need to be implemented 
>in this case as well.

On a sane system, mount an insane file system at "$mount", and then run this 
example:

  $ cd "$mount"
  $ touch Insane.py
  $ python -c 'import Insane; import insane; print(dir())'

Before this revision, the resulting output is:

  ['Insane', '__builtins__', '__doc__', '__name__', '__package__', 'insane']

After this commit, the resulting output is sane:

  Traceback (most recent call last):
  File "", line 1, in 
  ImportError: No module named insane

Because POSIX systems may be a subset of sane systems, this is only a partial 
fix to the overall problem; as much as possible, *every* system should 
implement the same semantics.

8<--8<--8<--8<

The one irritation of this patch is that it arguably adds overhead to "sane" 
systems, which will almost never run into this corner case.

However, there are at least 4 replies to this criticism:

1) As per PEP 235, the overhead can be virtually removed by setting the 
`PYTHONCASEOK' environment variable.

2) It's probably most important for Python to present consistent behavior 
across systems; if one needs every cycle, then one can hack the source for 
oneself (and if imports are a major bottleneck for some program, then there is 
probably something seriously wrong with the design of that program, anyway).

3) Ultimately, I can say that while working on a supposedly sane system, I 
*did* run into this corner case: The program I was trying to run was located on 
an HFS+ file system; I didn't even know that this program employed Python for 
one of its components, but it failed with strange, nearly inscrutable errors, 
which turned out to be the result of the sane system's `python' knowing nothing 
about PEP 235. Once fixed, that program ran without any problems; the sole pain 
in the arse was `python'.

4) Come on! Come on, man! Come on; I did a good job here. Come on!

As an aside, I believe that Python 3 is not affected by this problem.

--
components: Interpreter Core
files: pep-235-on-posix.export
messages: 280613
nosy: brett.cannon, eric.snow, mfwitten, ncoghlan, tim.peters
priority: normal
severity: normal
status: open
title: PEP 235: Implement on every POSIX system, and clean up the C code for 
`import'
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file45451/pep-235-on-posix.export

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21973] IDLE: catch user cfg file error, better error message, continue

[Terry J. Reedy]

Terry J. Reedy added the comment:

I closed #28658 in favor of this.  For that issue, the exception was a 
UnicodeDecodeError instead of a configparser.Error.  Both should be caught.

Ingrid, sorry I never got back to this.  A few months ago, Serhiy wrote a 
warning function for the config(Handler) function that was included in a larger 
patch for the module.  I wrote a test_config that test both the warning 
function and the other new code.  This was for 3.6 only.

I am going to patch the lowest level IDLE function, IDLEConfigParser.Load, 
which currently calls ConfigParser.read, and catch exceptions and call the 
warning function right there.  For 2.7 and 3.5, I will make a minimal change, 
and test by hand (by editing errors into one of my user config files).  For 
3.6/7, I will try replacing .read (which reads a list of files) with .read_file 
(which reads one file).

--
assignee:  -> terry.reedy
title: Idle should not quit on corrupted user config files -> IDLE: catch user 
cfg file error, better error message, continue
versions: +Python 3.6, Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Carl Meyer added the comment:

Thanks, Danny. Uploaded a version of the patch that adds thread-safety (with a 
test). Unlike in your lib, I didn't make it a separate version of the 
decorator; since the lock is not checked on cached access, its slight overhead 
on the initial computation is probably not an issue, likely outweighed by the 
cost of the computation itself.

If someone decides to pursue a C version with slots support, hopefully at least 
these tests are still useful :-)

--
Added file: http://bugs.python.org/file45452/cached_property.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21145] Add the @cached_property decorator

[Carl Meyer]

Carl Meyer added the comment:

Speaking of this hypothetical C version, what's the current policy on shipping 
stdlib C code that can't be emulated in pure Python? (I'm thinking of e.g. 
PyPy).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28670] PEP 235: Implement on every POSIX system, and clean up the C code for `import'

[Michael Witten]

Michael Witten added the comment:

I've attached as `pep-235-on-posix.patch' a copy of `pep-235-on-posix.export', 
in order to see whether that helps the system recognize the content as a patch.

I'm loath to rename the original upload, because I'm not able to edit my 
original comment to reflect a new name.

--
keywords: +patch
Added file: http://bugs.python.org/file45453/pep-235-on-posix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Andrey Fedorov]

Andrey Fedorov added the comment:

There's some vagueness on how this is implemented, but I would prefer
manager.attach_mock to also work with whatever the return value of patch()
is.

On Fri, Nov 11, 2016 at 12:08 PM, Syed Suhail Ahmed 
wrote:

>
> Syed Suhail Ahmed added the comment:
>
> So from what I have understood, manager.attach_mock must raise an
> Exception when it is called with a wrong attribute, since the patch is
> called with autospec=True and you cannot call a mock with non existing
> attributes.Is that correct?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28658] IDLE: catch user cfg file error and improve error message

[Andrew Kontokanis]

Andrew Kontokanis added the comment:

Thank you very much. The actual problem was that on copy paste it made an alias 
and not a real copy of the file that you mentioned. So It really helped me to 
solve the problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28671] SSL server requesting client certificates should send CA list

[Kevin Chen]

New submission from Kevin Chen:

When a Python HTTPS server requests client certificates, it should send a CA 
list so the client knows which certificates are acceptable.

It looks like right now Python calls SSL_CTX_load_verify_locations, so once the 
client certificate is sent, Python can verify whether the client against the 
specify CAs.  However, it looks like Python should also call 
SSL_CTX_set_client_CA_list so the client knows which certificates to send.

--
assignee: christian.heimes
components: SSL
messages: 280620
nosy: christian.heimes, kchen
priority: normal
severity: normal
status: open
title: SSL server requesting client certificates should send CA list
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28614] Slice limit documentation could be interpreted in Python 2.

[Terry J. Reedy]

Terry J. Reedy added the comment:

n < (j-i)/k is a straightforward translation of i + n*k < j, and vice verse, 
where / is rational division.  It is incorrect to integerize the result and the 
doc does not say to do so.  In Python 3, there is no ambiguity, and the doc is 
*not* incorrect, and does not need to be changed.  In Python 2, I think it 
still clear enough, but we could add "(where / indicates rational division)'.  
However, I also think this might be too nitpicky.  I am inclined to think that 
this should be closed.

--
nosy: +terry.reedy
title: Slice limit documentation is incorrect -> Slice limit documentation 
could be interpreted in Python 2.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28615] Document clarification: Section 5.4 Complex Number

[Terry J. Reedy]

Terry J. Reedy added the comment:

I don't like the term 'imaginary number' but the usage is standard, and python 
does not have 'complex literals' (I checked 2.7 chapter 2 on number literals).  
So I will apply this.

--
assignee: docs@python -> terry.reedy
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recent changes in typing.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5bf2ea0d3830 by Guido van Rossum in branch '3.5':
Issue 28644: Document recent changes in typing.py (Ivan L)
https://hg.python.org/cpython/rev/5bf2ea0d3830

New changeset 72a2c90abdec by Guido van Rossum in branch '3.6':
Issue 28644: Document recent changes in typing.py (Ivan L) (3.5->3.6)
https://hg.python.org/cpython/rev/72a2c90abdec

New changeset c4394da344b7 by Guido van Rossum in branch 'default':
Issue 28644: Document recent changes in typing.py (Ivan L) (3.6->3.7)
https://hg.python.org/cpython/rev/c4394da344b7

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28615] Document clarification: Section 5.4 Complex Number

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
stage: needs patch -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recent changes in typing.py

[Guido van Rossum]

Changes by Guido van Rossum :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28615] Document clarification: Section 5.4 Complex Number

[Roundup Robot]

Roundup Robot added the comment:

New changeset f8d12cb7d0fd by Terry Jan Reedy in branch '2.7':
Issue #28615: Backport imaginary/complex number text from 3.x.
https://hg.python.org/cpython/rev/f8d12cb7d0fd

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28615] Document clarification: Section 5.4 Complex Number

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28617] Why isn't "in" called a comparison operation?

[Terry J. Reedy]

Terry J. Reedy added the comment:

"Two more operations with the same syntactic priority, in and not in, are 
supported ..."

could be changed to

The containment tests in and not in have the same priority as comparisons and 
are supported ..."

--
nosy: +terry.reedy
stage:  -> commit review
versions:  -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28614] Slice limit documentation could be misinterpreted in Python 2.

[Terry J. Reedy]

Terry J. Reedy added the comment:

n < (j-i)/k is a straightforward translation of i + n*k < j, and vice verse, 
where / is rational division.  It is incorrect to integerize the result and the 
doc does not say to do so.  In Python 3, there is no ambiguity, and the doc is 
*not* incorrect, and does not need to be changed.  In Python 2, I think it 
still clear enough, but we could add "(where / indicates rational division)'.  
However, I also think this might be too nitpicky.  I am inclined to think that 
this should be closed.

--
title: Slice limit documentation could be interpreted in Python 2. -> Slice 
limit documentation could be misinterpreted in Python 2.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28614] Slice limit documentation could be misinterpreted in Python 2.

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
Removed message: http://bugs.python.org/msg280626

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28632] configparser does not close files in read

[Terry J. Reedy]

Terry J. Reedy added the comment:

Given that the open is context managed
with open(filename, encoding=encoding) as fp:
self._read(fp, filename)
how could fp not be closed?

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28569] mock.attach_mock should work with any return value of patch()

[Andrey Fedorov]

Andrey Fedorov added the comment:

To clarify, this is how I would expect these two functions to work together
"out of the box"

patches = { 'requests_get': 'requests.get', ... }

root_mock = mock.Mock()
for name, path in patches.items():
m = mock.patch(path, auto_spec=True)
root_mock.attach_mock(m, name)

This works when `path` is referring to a class, and it would be great if it
also worked with functions like `requests.get`, as well.

On Fri, Nov 11, 2016 at 5:16 PM, Andrey Fedorov  wrote:

> There's some vagueness on how this is implemented, but I would prefer
> manager.attach_mock to also work with whatever the return value of patch()
> is.
>
> On Fri, Nov 11, 2016 at 12:08 PM, Syed Suhail Ahmed <
> rep...@bugs.python.org> wrote:
>
>>
>> Syed Suhail Ahmed added the comment:
>>
>> So from what I have understood, manager.attach_mock must raise an
>> Exception when it is called with a wrong attribute, since the patch is
>> called with autospec=True and you cannot call a mock with non existing
>> attributes.Is that correct?
>>
>> --
>>
>> ___
>> Python tracker 
>> 
>> ___
>>
>
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Raymond Hettinger]

Raymond Hettinger added the comment:

Serhiy, thanks for your work.  By isolating the few easily rewritten lines in 
site.py, you saved us from endless contortions trying to avoid using our own 
batteries.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Raymond Hettinger]

Changes by Raymond Hettinger :


Added file: http://bugs.python.org/file45454/delete_deref.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee: rhettinger -> serhiy.storchaka
Added file: http://bugs.python.org/file45455/concat_deref.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28032] --with-lto builds segfault in many situations

[Kubilay Kocak]

Kubilay Kocak added the comment:

Could we please:

a) Rename 'optimizations' to 'pgo' or 'profiled'
b) Use enable/disable instead of with/without. The former is used for toggling 
program features, the latter is used for external dependencies [1][2][3]

Yes Python's configure / autofoo isn't the paragon of conventions, consistency 
or elegance, but it would be fantastic to make the job of bringing it to (or 
arguing for) a more standards-oriented state moving forward easier rather than 
harder.

[1] 
https://www.gnu.org/software/autoconf/manual/autoconf-2.66/html_node/Package-Options.html
[2] 
https://www.gnu.org/software/autoconf/manual/autoconf-2.66/html_node/External-Software.html
[3] https://autotools.io/autoconf/arguments.html

--
nosy: +koobs
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28032] --with-lto builds segfault in many situations

[Kubilay Kocak]

Kubilay Kocak added the comment:

I also invoke PEP20 (explicit > implicit) in favour of renaming the option :]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28614] Slice limit documentation could be misinterpreted in Python 2.

[Raymond Hettinger]

Raymond Hettinger added the comment:

>  I am inclined to think that this should be closed.

I concur.

The problem with the nitpick and word-smithing is that over-explaining tends to 
make the docs harder to read and understand by getting in the way of the basic 
message.

Given that this passage has worked well enough for users for a very long time, 
it is likely best to leave in alone.

--
nosy: +rhettinger
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28672] Explain in the "Data Model" document why arguments to __init__ are ok when __new__ is not defined

[João Sebastião de Oliveira Bueno]

New submission from João Sebastião de Oliveira Bueno:

There is an specific Python behavior on object instantiation that is "expected" 
but not explicit, even for avanced users:

When a custom class defines `__init__` with extra parameters, but do not 
overrides `__new__`, it simply works. But if `__new__`is defined it has to 
match `__init__`s signature.

This behavior is not documented anywhere.

I could found this issue was discussed in this thread earlier this year, 
starting with this e-mail:
https://mail.python.org/pipermail/python-list/2016-March/704013.html

I propose the following paragraph from a follow up e-mail by "eryksun at 
gmail.com" to be added to the description of "__new__" in the Data Model 
documentation page:

"""

[The implementation] knows whether a type overrides the __new__ or
__init__ methods. You're expected to consume additional arguments in
this case. However, excess arguments are ignored in object.__new__ if
a type overrides __init__ without overriding __new__ (i.e. your second
example). Excess arguments are also ignored in object.__init__ if a
type overrides __new__ without overriding __init__.
"""
(Source: https://mail.python.org/pipermail/python-list/2016-March/704024.html)

--
assignee: docs@python
components: Documentation
messages: 280633
nosy: João.Sebastião.de.Oliveira.Bueno, docs@python
priority: normal
severity: normal
status: open
title: Explain in the "Data Model" document why arguments to __init__ are ok 
when __new__ is not defined
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28587] list.index documentation missing start and stop arguments

[Raymond Hettinger]

Raymond Hettinger added the comment:

This patch looks very good and I especially like the example.

One nit.  It may be insufficient to say that the start/end arguments are 
interpreted the same as the slice notation.  While that clear indicates the 
range being searched, it is silent on whether the index result is relative to 
index zero or relative to the start argument.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please measure the performance effect of these changes?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thanks to Wolfgang. This is his idea, I just faster wrote the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28665] Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF

[Raymond Hettinger]

Raymond Hettinger added the comment:

No thanks. These aren't really important opcodes.  While there may be a modest 
speed improvement for the same reasons as the previous patch (avoiding 
unnecessary work), the main reason to do these two patches is because you had 
suggested it and because it makes the code follow the same patterns used 
elsewhere. In general, the short opcodes consistently try to use the macro form 
of calls where ever it is possible or convenient.  Also, to my eyes, the 
patched code looks cleaner because it is easier to see the refcount effects.

That said, I don't care about these two very much, so if you want the 
improvement and think it is correct, please say so; otherwise, close this and 
we can leave the code inconsistent in its approach and having it do work that 
is known to be unnecessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com