[issue7874] logging.basicConfig should raise warning/exception on second call
New submission from Tobias : logging.basicConfig should raise warning/eception on second call. Why? logging.basicConfig(filename="/tmp/works.log") logging.basicConfig(filename="/tmp/worksnot.log") what do you think does happen? Right - logging goes to "/tmp/worksnot.log". But does not behave that way. The secound call does nothing. Simply bad coding style, an if without an else. kind regards Tobias -- components: 2to3 (2.x to 3.0 conversion tool) messages: 99003 nosy: tocomo severity: normal status: open title: logging.basicConfig should raise warning/exception on second call type: behavior versions: Python 2.5, Python 2.6 ___ Python tracker <http://bugs.python.org/issue7874> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
tobias added the comment: Actually, I think this whole issue is more complex. For example, consider a (fictious) CGI script where users can upload an image and a description and the script sends a success/error message in return. In this case, one has to: - read the HTTP request header from stdin as US-ASCII - read the image from stdin as raw binary data - read the description from stdin as a string in some encoding - write the HTTP response header to stdout as US-ASCII - write the response message to stdout in some (other) encoding - write error messages to server log via stderr as US-ASCII Also, there are cases when a cgi script should return binary data instead (e.g., images or archive files) or apply a transfer encoding (e.g., gzip). Although FieldStorage only cares about reading, it still has to cope with intermixed textual and binary data. So the only practical way in my opinion is to use raw binary data and have FieldStorage decode strings on demand, since only the programmer knows whether a field should contain text or binary data. FieldStorage should offer two methods for this purpose: one for reading binary data and another for reading and decoding strings on-the-fly (possibly using a default encoding passed to its constructor). -- nosy: +tobias ___ Python tracker <http://bugs.python.org/issue4953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29992] Expose parse_string in JSONDecoder
Tobias Oberstein added the comment: > It's unlikely that you would want to parse every string that looks enough > like a decimal as a decimal, or that you would want to pay the cost of > checking every string in the whole document to see if it's a decimal. fwiw, yes, that's what I do, and yes, it needs to check every string https://github.com/crossbario/autobahn-python/blob/bc98e4ea5a2a81e41209ea22d9acc53258fb96be/autobahn/wamp/serializer.py#L410 > Returning a decimal as a string is becoming quite common in REST APIs to > ensure there is no floating point errors. exactly. it is simply required if money values are involved. since JSON doesn't have a native Decimal, strings need to be used (the only scalar type in JSON that allows one to encode the needed arbitrary precision decimals) CBOR has tagged decimal fraction encoding, as described in RFC7049 section 2.4.3. fwiw, we've added roundtrip and crosstrip testing between CBOR <=> JSON in our hacked Python JSON, and it works https://github.com/crossbario/autobahn-python/blob/bc98e4ea5a2a81e41209ea22d9acc53258fb96be/autobahn/wamp/test/test_wamp_serializer.py#L235 -- ___ Python tracker <https://bugs.python.org/issue29992> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
New submission from Tobias Oberstein : The urlparse module currently does not support the new "ws" and "wss" schemes used for the WebSocket protocol. As a workaround, we currently use the following code (which is a hack of course): import urlparse wsschemes = ["ws", "wss"] urlparse.uses_relative.extend(wsschemes) urlparse.uses_netloc.extend(wsschemes) urlparse.uses_params.extend(wsschemes) urlparse.uses_query.extend(wsschemes) urlparse.uses_fragment.extend(wsschemes) === A WebSocket URL has scheme "ws" or "wss", MUST have a network location and MAY have a resource part with path and query components, but MUST NOT have a fragment component. -- components: Library (Lib) messages: 146167 nosy: oberstet priority: normal severity: normal status: open title: WebSocket schemes in urlparse type: feature request versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment: fragment identifiers: the spec says: "Fragment identifiers are meaningless in the context of WebSocket URIs, and MUST NOT be used on these URIs. The character "#" in URIs MUST be escaped as %23 if used as part of the query component." [see last line of my initial comment] I nevertheless added the ws/wss schemes to urlparse.uses_fragment so that I can detect them being used and throw. Does urllib throw when an URL contains a fragment identifier, but the scheme of the URL is not in urlparse.uses_fragment? If so, thats fine and of course better than putting the burden of checking on the user. == Further, when "#" is to be used in a WS URL, it MUST be encoded, and if so, it's interpreted as part of the query component. So in summary, I think the best would be: urllib throws upon non-encoded "#", and interpret it as part of the query component when encoded. -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment: Well, thinking about it, %23 can also appear in a percent encoded path component. I don't get the conditional "..if used as part of the query component" in the spec. -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment: I see how you interpret that sentence in the spec, but I would have read it differently: invalid: 1. ws://example.com/something#somewhere 2. ws://example.com/something#somewhere/ 3. ws://example.com/something#somewhere/foo 4. ws://example.com/something?query=foo#bar valid: 5. ws://example.com/something%23somewhere 6. ws://example.com/something%23somewhere/ 7. ws://example.com/something%23somewhere/foo 8. ws://example.com/something?query=foo%23bar You would take 2. and 3. as valid, but 1. and 4. as invalid, right? But you are right, the spec does not talk about # in path. If above is a valid summary of the question, I'd better take that to the Hybi list to get feedback before rushing into anything with urllib .. -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment:
I'll ask (to be sure) and link.
However, after rereading the Hybi 17 section, it says
"""
path =
"""
And http://tools.ietf.org/html/rfc3986 says:
"""
The path is terminated by the first question mark ("?") or number sign ("#")
character, or by the end of the URI.
"""
So my reading would be: non-escaped # can never be part of path for a WebSocket
URL by reference of RFC3986.
--
___
Python tracker
<http://bugs.python.org/issue13244>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment: here the links to the question on the Hybi list: http://www.ietf.org/mail-archive/web/hybi/current/msg09257.html and http://www.ietf.org/mail-archive/web/hybi/current/msg09258.html http://www.ietf.org/mail-archive/web/hybi/current/msg09243.html == I'll track those and come back when there is a conclusion .. -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment:
sorry for "throw" .. somewhat bad habit (stemming from wandering between
languages).
uses_fragment extended:
[autobahn@autobahnhub ~/Autobahn]$ python
Python 2.7.1 (r271:86832, Dec 13 2010, 15:52:15)
[GCC 4.2.1 20070719 [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
>>> import urlparse
>>> wsschemes = ["ws", "wss"]
>>> urlparse.uses_relative.extend(wsschemes)
>>> urlparse.uses_netloc.extend(wsschemes)
>>> urlparse.uses_params.extend(wsschemes)
>>> urlparse.uses_query.extend(wsschemes)
>>> urlparse.uses_fragment.extend(wsschemes)
>>> urlparse.urlparse("ws://example.com/something#somewhere/")
ParseResult(scheme='ws', netloc='example.com', path='/something', params='',
query='', fragment='somewhere/')
>>> urlparse.urlparse("ws://example.com/something#somewhere")
ParseResult(scheme='ws', netloc='example.com', path='/something', params='',
query='', fragment='somewhere')
>>>
=> fragment extracted
uses_fragment not extended:
[autobahn@autobahnhub ~/Autobahn]$ python
Python 2.7.1 (r271:86832, Dec 13 2010, 15:52:15)
[GCC 4.2.1 20070719 [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
>>> import urlparse
>>> wsschemes = ["ws", "wss"]
>>> urlparse.uses_relative.extend(wsschemes)
>>> urlparse.uses_netloc.extend(wsschemes)
>>> urlparse.uses_params.extend(wsschemes)
>>> urlparse.uses_query.extend(wsschemes)
>>> urlparse.urlparse("ws://example.com/something#somewhere/")
ParseResult(scheme='ws', netloc='example.com', path='/something#somewhere/',
params='', query='', fragment='')
>>> urlparse.urlparse("ws://example.com/something#somewhere")
ParseResult(scheme='ws', netloc='example.com', path='/something#somewhere',
params='', query='', fragment='')
>>>
=> no fragment extracted, but interpreted as part of path component
=> no exception raised
The answer on Hybi outstanding, but I would interpret Hybi-17: # must always be
escaped, both in path and query components. Fragment components are not
allowed. Thus, unescaped # can never appear in WS URL. Further, it must not be
ignored, but the WS handshake failed.
If this should indeed be the correct reading of the WS spec, then I think
urlparse should raise an exception upon unescaped # within URLs from ws/wss
schemes.
--
___
Python tracker
<http://bugs.python.org/issue13244>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urlparse
Tobias Oberstein added the comment: ok, there was feedback on Hybi list: http://www.ietf.org/mail-archive/web/hybi/current/msg09270.html """ 1. ws://example.com/something#somewhere 2. ws://example.com/something#somewhere/ 3. ws://example.com/something#somewhere/foo 4. ws://example.com/something?query=foo#bar I think all of these are invalid. """ Alexey Melnikov, Co-author of the WS spec. And Julian Reschke: http://www.ietf.org/mail-archive/web/hybi/current/msg09277.html == Thus, I would upload my comment: "# must always be escaped, both in path and query components. Fragment components are not allowed. Thus, unescaped # can never appear in WS URL. Further, it must not be ignored, but the WS handshake failed." And further: urlparse should raise an exception upon unescaped # within URLs from ws/wss schemes. -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urllib.parse
Tobias Oberstein added the comment: > I’d say that urlparse should raise an exception when a ws/wss URI contains a > fragment part. Yep, better. > I’m not sure this will be possible; from a glance at the source and a quick > test, urlparse will happily break the Generic URI Syntax RFC and return a > path including a # character! That's unfortunate. In that case I'd probably prefer the lesser evil, namely that urlparse be set up (falsely) such that ws/wss scheme would falsely allow fragments, so I get back the non-empty fragment as a separate component, and check myself. If urlparse returns the fragment (falsely) within path, then a user could check only by searching for # in the path. Also hacky .. even worse than compare fragment for != "". Essentially, this would be exactly "the hack" that I posted in my very first comment: urlparse.uses_fragment.extend(wsschemes) === Alternative: make this bug dependent on fixing urlparse for fragment rules in generic URI RFC and don't do anything until then? -- ___ Python tracker <http://bugs.python.org/issue13244> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urllib.parse
Tobias Oberstein added the comment:
The patch as it stands will result in wrong behavior:
+self.assertEqual(urllib.parse.urlparse("ws://example.com/stuff#ff"),
+ ('ws', 'example.com', '/stuff#ff', '', '', ''))
The path component returned is invalid for ws/wss and is invalid for any scheme
following the generic URI RFC, since # must be always escaped in path
components.
Is urlparse meant to follow the generic URI RFC?
IMHO, the patch at least should do the equivalent of
urlparse.uses_fragment.extend(wsschemes)
so users of urlparse can do the checking for fragment != "", required for
ws/wss on their own.
--
___
Python tracker
<http://bugs.python.org/issue13244>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urllib.parse
Tobias Oberstein added the comment:
Is that patch supposed to be in Python 2.7.2?
If so, it doesn't work for "ws":
"ws://example.com/somewhere?foo=bar#dgdg"
F:\scm\Autobahn\testsuite\websockets\servers>python
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urlparse import urlparse
>>> urlparse("ws://example.com/somewhere?foo=bar#dgdg")
ParseResult(scheme='ws', netloc='example.com', path='/somewhere?foo=bar#dgdg',
params='', query='', fragment='')
>>> urlparse("ws://example.com/somewhere?foo=bar#dgdg", allow_fragments = True)
ParseResult(scheme='ws', netloc='example.com', path='/somewhere?foo=bar#dgdg',
params='', query='', fragment='')
>>> urlparse("ws://example.com/somewhere?foo=bar#dgdg", allow_fragments = False)
ParseResult(scheme='ws', netloc='example.com', path='/somewhere?foo=bar#dgdg',
params='', query='', fragment='')
>>>
urlparse will neither parse the query nor the (invalid) fragment component for
the "ws" scheme
I would have expected
ParseResult(scheme='ws', netloc='example.com', path='/somewhere', params='',
query='foo=bar', fragment='dgdg')
--
___
Python tracker
<http://bugs.python.org/issue13244>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
Tobias Brink added the comment: I can confirm an overhead of 2 ms to 3 ms using a relatively recent Intel Core i5 CPU. I (personally) believe these 3 ms to be a pretty big overhead on modern computers and I also believe that it would be relatively simple to reduce it. I don't have much time at the moment but I will try to produce a complete proof of concept patch for the futures module in the next weeks. I'd be happy to get some comments when it is finished. -- ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12595] python.h redundant redeclarations
New submission from Tobias Pfaff :
Compiling 'Python.h' with g++ and -Wredundant-decls produces warnings
Testcase:
test.cpp:
#include
int main() { return 0; }
g++ test.cpp -I/usr/include/python3.2mu/ -Wredundant-decls
In file included from /usr/include/python3.2mu/Python.h:106,
from test.cpp:1:
/usr/include/python3.2mu/pyerrors.h:73: warning: redundant redeclaration of
‘void Py_FatalError(const char*)’ in same scope
/usr/include/python3.2mu/pydebug.h:29: warning: previous declaration of ‘void
Py_FatalError(const char*)’
--
components: Build
messages: 140746
nosy: verticalduck
priority: normal
severity: normal
status: open
title: python.h redundant redeclarations
type: compile error
versions: Python 3.2
___
Python tracker
<http://bugs.python.org/issue12595>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10570] curses.tigetstr() returns bytes, but curses.tparm() expects a string
Tobias Klausmann added the comment: This bug is still not fixed and basically makes the curses module unusable except for very narrow use cases. Unfortunately, my C-fu is very weak, otherwise I'd try to make a patch. -- nosy: +klausman ___ Python tracker <http://bugs.python.org/issue10570> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11324] ConfigParser(interpolation=None) doesn't work
New submission from Tobias Brink : The docs for Python 3.2 say that p = configparser.ConfigParser(interpolation=None) disables interpolation. Instead it gives this traceback when calling p.read(): Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.2/configparser.py", line 688, in read self._read(fp, filename) File "/usr/lib/python3.2/configparser.py", line 1081, in _read self._join_multiline_values() File "/usr/lib/python3.2/configparser.py", line 1091, in _join_multiline_values options[name] = self._interpolation.before_read(self, AttributeError: 'NoneType' object has no attribute 'before_read' The attached patch should fix it. -- components: Library (Lib) files: configparser_interpolation.patch keywords: patch messages: 129421 nosy: tbrink priority: normal severity: normal status: open title: ConfigParser(interpolation=None) doesn't work type: behavior versions: Python 3.2, Python 3.3 Added file: http://bugs.python.org/file20895/configparser_interpolation.patch ___ Python tracker <http://bugs.python.org/issue11324> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11324] ConfigParser(interpolation=None) doesn't work
Tobias Brink added the comment: I added a test but I am not too familiar with the Python test suite. Please check if the "test_init_takes_interpolation_none" test is necessary because the test suite also fails without it if my patch is not applied. Feel free to remove it if it is superfluous. -- Added file: http://bugs.python.org/file20957/patch_with_test.diff ___ Python tracker <http://bugs.python.org/issue11324> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
New submission from Tobias Brink : I tested the new concurrent.futures.ProcessPoolExecutor.map() in 3.2 with the is_prime() function from the documentation example. This was significantly slower than using multiprocessing.Pool.map(). Quick look at the source showed that multiprocessing sends the iterable in chunks to the worker process while futures sends always only one entry of the iterable to the worker. Functions like is_prime() which finish relatively fast make the communication overhead (at least I guess that is the culprit) very big in comparison. Attached is a file which demonstrates the problem and a quick workaround. The workaround uses the chunk idea from multiprocessing. The problem is that it requires the iterables passed to map() to have a length and be indexable with a slice. I believe this limitation could be worked around. -- components: Library (Lib) files: map_comparison.py messages: 128963 nosy: tbrink priority: normal severity: normal status: open title: concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument type: performance versions: Python 3.2 Added file: http://bugs.python.org/file20825/map_comparison.py ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
Tobias Brink added the comment: Playing around a bit I wrote the attached implementation which works with all iterables. -- Added file: http://bugs.python.org/file20826/new_processpoolexecutor.py ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
New submission from Tobias Bergkvist :
Python-binaries compiled on either Big Sur or Catalina - and moved to the other
MacOS-version will not work as expected when code depends on
ctypes.util.find_library.
Example symptom of this issue:
https://github.com/jupyterlab/jupyterlab/issues/9863
I have personally faced this when using Python from nixpkgs - since nixpkgs
Python has been built on Catalina - and I'm using Big Sur.
Scenario 1: Compile on Catalina, copy binaries to BigSur, and call
ctypes.util.find_library('c')
Python 3.11.0a0 (heads/main:635bfe8162, Jul 19 2021, 08:09:05) [Clang 12.0.0
(clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library; print(find_library('c'))
None
Scenario 2: Compile on Big Sur, copy binaries to Catalina, and call
ctypes.util.find_library('c'):
Python 3.11.0a0 (heads/main:635bfe8162, Jul 19 2021, 08:28:48) [Clang 12.0.5
(clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library; print(find_library('c'))
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.11/ctypes/__init__.py", line 8, in
from _ctypes import Union, Structure, Array
^^^
ImportError:
dlopen(/usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so, 2):
Symbol not found: __dyld_shared_cache_contains_path
Referenced from:
/usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so (which was
built for Mac OS X 11.4)
Expected in: /usr/lib/libSystem.B.dylib
in /usr/local/lib/python3.11/lib-dynload/_ctypes.cpython-311-darwin.so
--
components: ctypes
messages: 397916
nosy: bergkvist
priority: normal
pull_requests: 25816
severity: normal
status: open
title: MacOS: Python binaries not portable between Catalina and Big Sur
type: behavior
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue44689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
Tobias Bergkvist added the comment: An alternative to using _dyld_shared_cache_contains_path is to use dlopen to check for library existence (which is what Apple recommends in their change notes: https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes). > New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker > cache of all system-provided libraries. As part of this change, copies of > dynamic libraries are no longer present on the filesystem. Code that attempts > to check for dynamic library presence by looking for a file at a path or > enumerating a directory will fail. Instead, check for library presence by > attempting to dlopen() the path, which will correctly check for the library > in the cache. (62986286) I have created a PR which modifies the current find_library from using _dyld_shared_cache_contains_path to dlopen. It passes all of the existing find_library-tests: https://github.com/python/cpython/pull/27251 There might be downsides to using dlopen (performance?) or something else I haven't considered. The huge upside however, is that the function is basically available on all Unix-systems. -- ___ Python tracker <https://bugs.python.org/issue44689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
Tobias Bergkvist added the comment:
You are absolutely right! From the manpage of dlopen(3) on MacOS Big Sur:
> dlopen() examines the mach-o file specified by path. If the file is
> compatible with the current process and has not already been loaded into the
> current process, it is loaded and linked. After being linked, if it contains
> any initializer functions, they are called, before dlopen() returns. dlopen()
> can load dynamic libraries and bundles. It returns a handle that can be used
> with dlsym() and dlclose(). A second call to dlopen() with the same path will
> return the same handle, but the internal reference count for the handle will
> be incremented. Therefore all dlopen() calls should be balanced with a
> dlclose() call.
Essentially, if the shared library contains initializer functions that have
some kind of side-effects, dlopen will also trigger these side-effects.
Basic example:
```
// mylib.c
#include
void myinit(void) {
printf("Hello from mylib\n");
}
__attribute__((section("__DATA,__mod_init_func"))) typeof(myinit) *__init =
myinit;
```
---
```
// main.c
#include
#include
int main(void) {
void* handle = dlopen("./mylib.dyld", RTLD_LAZY);
if (handle == NULL) printf("Failed to load");
}
```
$ clang mylib.c -shared -o mylib.dyld
$ clang main.c -o main
$ ./main
Hello from mylib
--
___
Python tracker
<https://bugs.python.org/issue44689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
Tobias Bergkvist added the comment:
Okay, I decided to look into how I could do dynamic loading as you suggested.
Here is a POC (executable) for using _dyld_shared_cache_contains_path when
available:
```
#include
#include
void* libsystemb_handle;
typedef bool (*_dyld_shared_cache_contains_path_f)(const char* path);
_dyld_shared_cache_contains_path_f _dyld_shared_cache_contains_path;
bool _dyld_shared_cache_contains_path_fallback(const char* name) {
return false;
}
__attribute__((constructor)) void load_libsystemb(void) {
if (
(libsystemb_handle = dlopen("/usr/lib/libSystem.B.dylib", RTLD_LAZY))
== NULL ||
(_dyld_shared_cache_contains_path = dlsym(libsystemb_handle,
"_dyld_shared_cache_contains_path")) == NULL
) {
_dyld_shared_cache_contains_path =
_dyld_shared_cache_contains_path_fallback;
}
}
__attribute__((destructor)) void unload_libsystemb(void) {
if (libsystemb_handle != NULL) {
dlclose(libsystemb_handle);
}
}
int main(int argc, char ** argv) {
printf("Library exists in cache: %d\n",
_dyld_shared_cache_contains_path(argv[1]));
}
```
A fallback function is used when _dyld_shared_cache_contains_path cannot be
loaded, which always returns false. If there is no cache - the (nonexistent)
cache also does not contain whatever path you pass it.
The constructor function is called when the Python extension is loaded -
ensuring that _dyld_shared_cache_contains_path is defined and callable. I've
read that C extension modules cannot be autoreloaded
(https://ipython.org/ipython-doc/3/config/extensions/autoreload.html) - so this
might mean there is no need for a deconstructor? Instead the OS would handle
cleanup once the process exits?
This could be compiled on either MacOS Catalina or Big Sur, and run without
problems on the other MacOS version.
Regarding the "explicit weak linking" when building on MacOS Big Sur and later;
wouldn't this mean that a Big Sur build wouldn't work on Catalina?
Would you be positive towards a PR with the approach I demonstrated here?
--
___
Python tracker
<https://bugs.python.org/issue44689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
Tobias Bergkvist added the comment:
This makes a lot of sense now.
Thank you so much for the thorough explanation Ned - and for highlighting where
my assumptions were wrong!
I didn’t realise I needed to specify MACOSX_DEPLOYMENT_TARGET to enable
backwards compatibility. Is there a reason for not setting this to as old a
version as possible by default when running ./configure? (Bigger binaries? Or
something else?)
If I understand correctly, the following two statements are now equivalent (in
Python >= 3.9, but not in Python == 3.8), after the added Python-support for
weak linking you mention:
```
if (__builtin_available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)) { //
...
If (HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME) { // ...
```
From:
https://github.com/python/cpython/blob/3d315c311676888201f4a3576e4ee3698684a3a2/Modules/_ctypes/callproc.c#L1452
And in order to support the deprecated case of building on an older
MacOS-version, I would want to do something like the following:
```
#ifdef __APPLE__
#ifdef HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH
// leave current implementation as is
#else
// dynamic loading implementation
#endif
#endif
```
That way, dynamic loading will only be used when building on old MacOS, and we
keep the current code path using weak linking when possible.
This means we will still get useful compiler errors if for example Apple
decides to suddenly remove the _dyld_shared_cache_contains_path symbol again in
the future, or change its signature - rather than having to wait for the test
suite to fail. It makes sense to prioritise good error reporting for the latest
version of MacOS, since this will reduce the debugging time for CPython
developers whenever Apple introduces new breaking changes to MacOS.
--
___
Python tracker
<https://bugs.python.org/issue44689>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] MacOS: Python binaries not portable between Catalina and Big Sur
Tobias Bergkvist added the comment: I realised that I needed to define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH_RUNTIME in the source file myself - as this is not defined after running the configure-script. I've updated the PR and its description to only focus on the legacy/deprecated approach on building on Catalina and running on Big Sur. Now a dynamic loading version of _dyld_shared_cache_contains_path is only used when compiling on MacOS < 11 (Catalina or older). And the weak-linking approach is used when HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH is defined (MacOS >= 11). -- ___ Python tracker <https://bugs.python.org/issue44689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44863] Allow TypedDict to inherit from Generics
Change by Tobias Burger : -- nosy: +toburger ___ Python tracker <https://bugs.python.org/issue44863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] ctypes.util.find_library() does not find macOS 11+ system libraries when built on older macOS systems
Tobias Bergkvist added the comment: It seems like _dyld_shared_cache_contains_path exists, while the define flag HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH has not been set. Essentially, the define-flags being used does not seem to agree with the SDK version you are using. Could you try adding HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH like this and see if it builds correctly? clang -Wno-unused-result -Wsign-compare -g -O0 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/uuid -Werror=unguarded-availability-new -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/S DKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/uuid -Werror=unguarded-availability-new -DMACOSX -DUSING_MALLOC_CLOSURE_DOT_C=1 -DHAVE_FFI_PREP_CIF_VAR=1 -DHAVE_FFI_PREP_CLOSURE_LOC=1 -DHAVE_FFI_CLOSURE_ALLOC=1 -HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH=1 -DPy_BUILD_CORE_BUILTIN -I_ctypes/darwin -c ./Modules/_ctypes/callproc.c -o Modules/callproc.o -- ___ Python tracker <https://bugs.python.org/issue44689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44689] ctypes.util.find_library() does not find macOS 11+ system libraries when built on older macOS systems
Tobias Bergkvist added the comment: I made a typo (forgetting the -D): clang -Wno-unused-result -Wsign-compare -g -O0 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/uuid -Werror=unguarded-availability-new -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/S DKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvmgn/T/tmpkfji88v7/tools/deps/include/uuid -Werror=unguarded-availability-new -DMACOSX -DUSING_MALLOC_CLOSURE_DOT_C=1 -DHAVE_FFI_PREP_CIF_VAR=1 -DHAVE_FFI_PREP_CLOSURE_LOC=1 -DHAVE_FFI_CLOSURE_ALLOC=1 -DHAVE_DYLD_SHARED_CACHE_CONTAINS_PATH=1 -DPy_BUILD_CORE_BUILTIN -I_ctypes/darwin -c ./Modules/_ctypes/callproc.c -o Modules/callproc.o -- ___ Python tracker <https://bugs.python.org/issue44689> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12499] textwrap.wrap: add control for fonts with different character widths
Change by Tobias Bengfort : -- nosy: +xi2 ___ Python tracker <https://bugs.python.org/issue12499> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12499] textwrap.wrap: add control for fonts with different character widths
Change by Tobias Bengfort : -- pull_requests: +26574 pull_request: https://github.com/python/cpython/pull/28136 ___ Python tracker <https://bugs.python.org/issue12499> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24665] CJK support for textwrap
Change by Tobias Bengfort : -- nosy: +xi2 nosy_count: 10.0 -> 11.0 pull_requests: +26575 pull_request: https://github.com/python/cpython/pull/28136 ___ Python tracker <https://bugs.python.org/issue24665> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42533] Document encodings.idna limitations
New submission from Tobias Kunze : The documentation for the encodings.idna module contains no indicator that the RFC it supports has been obsoleted by another RFC: https://docs.python.org/3.10/library/codecs.html#module-encodings.idna I'm sure this is obvious when you know your RFCs, but when just looking at the docs, it's easy to miss. In #msg379674, Marc-Andre suggested to fix the situation by deprecating or updating IDNA support. I'd like to propose to add a warning message in the meantime, pointing out the newer RFC and linking to the idna package on PyPI (if links to PyPI packages are alright in the docs?) -- assignee: docs@python components: Documentation, Unicode messages: 382300 nosy: docs@python, ezio.melotti, rixx, vstinner priority: normal severity: normal status: open title: Document encodings.idna limitations type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42533> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42860] Incompatible types in Python grammar
New submission from Tobias Kohn : There seems to be a small type error in the Python grammar in the rule `invalid_parameters`: ``` invalid_parameters: | param_no_default* (slash_with_default | param_with_default+) param_no_default ``` While the `slash_with_default` returns a single element, the `param_with_default` returns a list/sequence. -- messages: 384603 nosy: tobias.kohn priority: normal severity: normal status: open title: Incompatible types in Python grammar versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue42860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42860] Incompatible types in Python grammar
Tobias Kohn added the comment: Great, thanks a lot! -- ___ Python tracker <https://bugs.python.org/issue42860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43104] csv package in class
New submission from Tobias Müllner : Hello I want to use the csv package in a class. If I use the writer instance as local variable in the class it works perfectly but if I use it as class-variable (self.writer) it does not work anymore. Does anyone have the same behaviour? Yours Toby By the way, sorry for my bad English :( -- messages: 386151 nosy: tobias.muellner250502 priority: normal severity: normal status: open title: csv package in class type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue43104> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35853] Extend the functools module with more higher order function combinators
New submission from Tobias Pleyer : The partial function is a typical example of a higher order function: It takes a function as argument and returns a function. As the name of the functools module suggests its purpose is to provide tools for working with functions. This should, in my opinion, include a much bigger set of higher order function combinators as they are known from functional programming. As a start I suggest to add the following functions: identity: The identity function which returns its input compose: Create a function pipeline (threaded computation) sequence: Use compose without binding it to a variable -- components: Library (Lib) messages: 334536 nosy: tpleyer priority: normal severity: normal status: open title: Extend the functools module with more higher order function combinators type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35853> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35853] Extend the functools module with more higher order function combinators
Change by Tobias Pleyer : -- keywords: +patch pull_requests: +11546 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35853> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
New submission from Tobias Däullary : There should be a possibility to change the environment of a process created with multiprocessing. For subprocess this is possible thanks to the "env" attribute. Elaboration: While it is trivial to change os.environ manually, in some cases this is not possible. For instance: creating a COM process on Windows; this process will always inherit the environment of the host process. A workaround is to spawn a python process with a different environment which then will provide this to the child COM process. -- components: Library (Lib) messages: 334591 nosy: r-or priority: normal severity: normal status: open title: Change the environment for a new process type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Change by Tobias Däullary : -- keywords: +patch, patch pull_requests: +11558, 11559 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Change by Tobias Däullary : -- keywords: +patch pull_requests: +11558 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Change by Tobias Däullary : -- keywords: +patch, patch, patch, patch pull_requests: +11558, 11559, 11560, 11561 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Change by Tobias Däullary : -- keywords: +patch, patch, patch pull_requests: +11558, 11559, 11560 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Tobias Däullary added the comment: Because sometimes when a process is implicitly started by some 3rd party library (i.e. COM via pythonwin here) the "old", unchanged environment is retained as the process itself doesn't care if os.environ was changed or not, the original environment cannot be modified. This solution would give a lot of flexibility in that regard. I think pythonwin is actually a special case as the problem here lies somewhere in the dispatch(.) call. The process creation should be handled somewhere within the COM interface, pywin does nothing there. But this solution effectively circumvents that issue. -- components: +Windows nosy: +paul.moore, steve.dower, tim.golden, zach.ware ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35853] Extend the functools module with more higher order function combinators
Tobias Pleyer added the comment: Thank you for your quick and detailed answer. It appears I slightly misunderstood the purpose of the functools module. However kudos for your great effort to push the Python language forward. Best regards Tobias -- ___ Python tracker <https://bugs.python.org/issue35853> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35862] Change the environment for a new process
Tobias Däullary added the comment: Alright, thanks for pointing me in the right direction. I have to conclude that this issue is not present on a current operating system, as I now tried to reproduce with Windows 10 (I came across it on an ancient Windows XP (sic) system - I can just imagine putenv didn't do its job as requested). I suppose this can be closed then! -- ___ Python tracker <https://bugs.python.org/issue35862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29992] Expose parse_string in JSONDecoder
New submission from Tobias Oberstein: Though the JSONDecoder already has all the hooks internally to allow for a custom parse_string (https://github.com/python/cpython/blob/master/Lib/json/decoder.py#L330), this currently is not exposed in the constructor JSONDecoder.__init__. It would be nice to expose it. Currently, I need to do hack it: https://gist.github.com/oberstet/fa8b8e04b8d532912bd616d9db65101a -- messages: 291167 nosy: oberstet priority: normal severity: normal status: open title: Expose parse_string in JSONDecoder type: enhancement ___ Python tracker <http://bugs.python.org/issue29992> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29992] Expose parse_string in JSONDecoder
Tobias Oberstein added the comment: I agree, my use case is probably exotic: transparent roundtripping of binaries over JSON using a beginning \0 byte marker to distinguish plain string and base64 encoded binaries. FWIW, I do think however that adding "parse_string" kw param to the ctor of JSONDecoder would at least fit the current approach: there are parse_xxx parameters for all the other things already. If overriding string parsing would be via subclassing, while all the others stay with the kw parameter approach, that could be slightly confusing too, because it looses on consistency. Switching everything to subclassing/overriding for _all_ parse_XXX is I guess a no go, because it breaks existing stuff? > For me in my situation, it'll be messy anyways, because I need to support Py2 > and 3, and CPy and PyPy .. I just filed the issue for "completeness". -- ___ Python tracker <http://bugs.python.org/issue29992> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22021] shutil.make_archive() root_dir do not work
Tobias Kunze added the comment: I'm similarly confused by this issue. If somebody can help me understand what's going on, I'll put my understanding into a documentation patch. I have created this minimal example to demonstrate what I don't understand: I've created a directory structure within /tmp like this: issue22021 └── root └── structure ├── content │ └── please_add.txt └── do_not_add.txt Now I'd like to create a zip archive that contains the directories "structure" and "content", and the file "please_add.txt", but not the file "do_not_add.txt". My understanding of the documentation was that I'd have to do this: >>> shutil.make_archive(base_name='/tmp/issue22021archive', format='zip', >>> root_dir='/tmp/issue22021/root', >>> base_dir='/tmp/issue22021/root/structure/content') But on my system, the created file (/tmp/issue22021archive.zip) looks like this according to unzip -l: Archive: issue22021archive.zip Length DateTimeName - -- - 0 2018-05-30 00:26 tmp/issue22021/root/structure/content/ 0 2018-05-30 00:26 tmp/issue22021/root/structure/content/please_add.txt - --- 0 2 files This is consistent with my experience that created archives will always contain directories from / on, which is unexpected to me. It appears this happens whenever base_dir and root_dir is set, but to my understanding the documentation does not warn against this. I've confirmed this behaviour with Python 3.6.5 and Python 3.5.3, but I suspect that doesn't really matter, as it seems to be a documentation issue. -- nosy: +rixx ___ Python tracker <https://bugs.python.org/issue22021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22021] shutil.make_archive() root_dir do not work
Tobias Kunze added the comment: Thank you, that's what I figured out later last evening. To my understanding, the docs don't give any indication that base_dir is supposed to be relative to root_dir, so I'd add this information, and maybe add a similar example to the one above, if that's appropriate? -- ___ Python tracker <https://bugs.python.org/issue22021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32392] subprocess.run documentation does not have **kwargs
Change by Tobias Kunze : -- keywords: +patch pull_requests: +6916 stage: needs patch -> patch review ___ Python tracker <https://bugs.python.org/issue32392> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27902] pstats.Stats: strip_dirs() method cannot handle file paths from different OS
Change by Tobias Kunze : -- keywords: +patch pull_requests: +6924 stage: needs patch -> patch review ___ Python tracker <https://bugs.python.org/issue27902> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22021] shutil.make_archive() root_dir do not work
Tobias Kunze added the comment: Yes, this is a documentation issue: A patch clarifying what root_dir and base_dir do, and how they interact (or how they are to be used in combination) would be sufficient to close this issue. -- ___ Python tracker <https://bugs.python.org/issue22021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18427] str.replace causes segfault for long strings
New submission from Tobias Marschall:
Running the following two lines of code causes a segfault:
s = 'A' * 3142044943
t = s.replace('\n','')
My setup:
Python 2.7.5 (default, Jul 11 2013, 11:17:50)
[GCC 4.6.3] on linux2
Linux 3.2.0-45-generic #70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013 x86_64
x86_64 x86_64 GNU/Linux
I could reproduce this behavior on Python 2.6.8 and 2.5.6.
Please let me know if I can provide additional information.
--
components: Interpreter Core
messages: 192855
nosy: tobiasmarschall
priority: normal
severity: normal
status: open
title: str.replace causes segfault for long strings
type: crash
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue18427>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14709] http.client fails sending read()able Object
New submission from Tobias Steinrücken : It seems that http.client's send() function lacks an else/return statement in Line 772. If this method is called with an read()able Object, it jumps into L 750: if hasattr( data,"read"): processes this data correctly, but then falls through (due to missing else ) to L 773: try: L 774: self.socket.sendall(data) where finally an TypeError raises. -- components: None messages: 159845 nosy: Tobias.Steinrücken priority: normal severity: normal status: open title: http.client fails sending read()able Object type: behavior versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue14709> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20344] subprocess.check_output() docs misrepresent what shell=True does
Tobias Klausmann added the comment: Hi! On Tue, 25 Mar 2014, Tuomas Savolainen wrote: > Created a patch that adds notice of using shell=True and iterable to the > documentation. Please do comment if the formatting is wrong (this my first > documentation patch). I'd use articles, i.e. "and a list" and "does not raise an error" Also, s/except/expect/ Regards, Tobias -- ___ Python tracker <http://bugs.python.org/issue20344> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21356] Support LibreSSL (instead of OpenSSL): make RAND_egd optional
Changes by Tobias Oberstein : -- nosy: +oberstet ___ Python tracker <http://bugs.python.org/issue21356> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19277] zlib compressobj: missing parameter doc strings
New submission from Tobias Oberstein: Currently the `zlib` module documents zlib.compressobj([level]) However, there are more parameters present already today: zlib.compressobj([level, method, wbits]) These other parameters are used in at least 2 deployed libraries (in the context of WebSocket compression): https://github.com/tavendo/AutobahnPython/blob/master/autobahn/autobahn/compress_deflate.py#L527 http://code.google.com/p/pywebsocket/source/browse/trunk/src/mod_pywebsocket/util.py#231 -- assignee: docs@python components: Documentation, Library (Lib) messages: 200113 nosy: docs@python, oberstet priority: normal severity: normal status: open title: zlib compressobj: missing parameter doc strings type: enhancement versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue19277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19278] zlib compressobj: expose missing knobs
New submission from Tobias Oberstein: The zlib library provides a couple of knobs to control the behavior and resource consumption of compression: ZEXTERN int ZEXPORT deflateInit2 OF((z_streamp strm, int level, int method, int windowBits, int memLevel, int strategy)); Of these, only `level`, `method` and `windowBits` are exposed on `zlib.compressobj` (and only the first is even documented: see issue #19277). However, as was recently found from emperical evidence in the context of WebSocket compression http://www.ietf.org/mail-archive/web/hybi/current/msg10222.html the `memLevel` parameter in particular is very valuable in controlling memory consumption. For WebSocket compression (with JSON payload), the following parameter set was found to provide a useful resource-consumption/compression-ratio tradeoff: window bits=11 memory level=4 Hence, it would be useful to expose _all_ parameters in Python, that is `memLevel` and `strategy` too. -- components: Library (Lib) messages: 200114 nosy: oberstet priority: normal severity: normal status: open title: zlib compressobj: expose missing knobs type: enhancement versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue19278> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13244] WebSocket schemes in urllib.parse
Tobias Oberstein added the comment:
FWIW, WebSocket URL parsing is still wrong on Python 2.7.6 - in fact, it's
broken in multiple ways:
>>> from urlparse import urlparse
>>> urlparse("ws://example.com/somewhere?foo=bar#dgdg")
ParseResult(scheme='ws', netloc='example.com', path='/somewhere', params='',
query='foo=bar', fragment='dgdg')
>>> urlparse("ws://example.com/somewhere?foo=bar%23dgdg")
ParseResult(scheme='ws', netloc='example.com', path='/somewhere', params='',
query='foo=bar%23dgdg', fragment='')
>>> urlparse("ws://example.com/somewhere?foo#=bar")
ParseResult(scheme='ws', netloc='example.com', path='/somewhere', params='',
query='foo', fragment='=bar')
>>> urlparse("ws://example.com/somewhere?foo%23=bar")
ParseResult(scheme='ws', netloc='example.com', path='/somewhere', params='',
query='foo%23=bar', fragment='')
>>>
--
___
Python tracker
<http://bugs.python.org/issue13244>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19832] XML version is ignored
New submission from Tobias Kuhn: The first line of an XML file should be something like this: The XML parser of xml.sax, however, seems to ignore the value of "version": This should give an error, but it doesn't. It's not a very serious problem, but this should raise an error to be standards-compliant. I experienced this bug in the rdflib package: https://github.com/RDFLib/rdflib/issues/347 -- components: XML messages: 204720 nosy: tkuhn priority: normal severity: normal status: open title: XML version is ignored type: behavior versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue19832> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22129] Please add an equivalent to QString::simplified() to Python strings
New submission from Tobias Leupold: It would be nice if a function equivalent to Qt's QString::simplified() would be added to Python's strings (cf. http://qt-project.org/doc/qt-4.8/qstring.html#simplified ). I'm not sure if my approach is good or fast, but I added this function to my code like so: http://nasauber.de/blog/T/143/QString_simplified_in_Python -- components: Interpreter Core messages: 224633 nosy: l3u priority: normal severity: normal status: open title: Please add an equivalent to QString::simplified() to Python strings type: enhancement versions: Python 3.3 ___ Python tracker <http://bugs.python.org/issue22129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20344] subprocess.check_output() docs misrepresent what shell=True does
New submission from Tobias Klausmann:
The subprocess docs state that the first argument can be either a string or an
iterable that contains the program and arguments to run. It also points out
that using shell=True allows for shell constructs. It does not mention a
subtlety that is introduced by Python's use of sh -c (on Unix):
Using a string that contains the full command line with args and shell=False
breaks as expected:
>>> subprocess.check_output("ls foo", shell=False)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.3/subprocess.py", line 576, in check_output
with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
File "/usr/lib64/python3.3/subprocess.py", line 824, in __init__
restore_signals, start_new_session)
File "/usr/lib64/python3.3/subprocess.py", line 1448, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'ls foo'
Using shell=True instead works as expected (since foo does not exist, ls exits
with non-0 and a different Exception is raised. This, too is to spec and
exactly what the docs describe.
>>> subprocess.check_output("ls foo", shell=True)
ls: cannot access foo: No such file or directory
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.3/subprocess.py", line 589, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'ls foo' returned non-zero exit status 2
Using shell=False and making the command a list of command and args does the
same thing:
>>> subprocess.check_output(["ls", "foo"], shell=False)
ls: cannot access foo: No such file or directory
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.3/subprocess.py", line 589, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['ls', 'foo']' returned non-zero exit
status 2
But using an iterable _and_ requesting a shell results in something very broken:
>>> subprocess.check_output(["ls", "foo"], shell=True)
[contents of my homedir are returned]
>>>
Note that the argument "foo" completely disappears, apparently. strace()
reveals that "foo" is never added to the call to "ls". Instead, it becomes $0
in the shell that ls runs in. This is exactly to spec (i.e. bash is not
broken). "bash -c foo bar baz" will start a shell that sets $0 to "bar", $1 to
"baz" and then executes "foo". Whereas "bash -c 'foo bar baz'" will run 'foo
bar baz'.
I think there are three possible fixes here:
1- Make check_output(list, shell=True) run something like "bash -c '%s'" % "
".join(list)
2- Change the docs to warn of the unintended consequences of combining lists
with shell=True
3- Make check_output() throw an Exception if the first argument is a list and
shell=True
The first option would make it easiest for people to "just use it", but I fear
the string manipulation may become the source of bugs with security
implications in the future.
The second option keeps the status quo, but people tend to not read docs, so it
may not be as effective as one would like, especially since shell-True already
has warnings and people tend to go "yeah I know about unverified input, but
it's not a problem for me" and then ignore both warnings.
Option 3 has the disadvantage that existing scripts that _expect_ the behavior
may break.
--
assignee: docs@python
components: Documentation, Library (Lib)
messages: 208811
nosy: docs@python, klausman
priority: normal
severity: normal
status: open
title: subprocess.check_output() docs misrepresent what shell=True does
type: behavior
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue20344>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6712] sys._getframe is not available on all Python implementations
Tobias Ivarsson added the comment: While it is true that not all Python implementations support sys._getframe() in some or all execution modes, Jython does support it, even IronPython supports it with the proper startup parameters these days. More importantly sys._getframe() [or something equivalent] *is* available as a public API in the inspect module of Python, as inspect.currentframe(). I don't know why the inspect module is not brought up in the discussions about sys._getframe(), but it is a public API. Would this mean that introspecting callframes is something all interpreters are expected to support? Brett: what is your opinion? Perhaps I should move this discussion to the mailing list? -- nosy: +thobe ___ Python tracker <http://bugs.python.org/issue6712> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9447] Python 3.1.2 test suite segfaults on the alpha architecture
New submission from Tobias Klausmann : During testing for inclusion in the Gentoo distribution, we ran into a test failure (actually a SEGV) when running the test suite on the alpha architecture. Log of test failure and backtrace in gdb is here: http://dev.gentoo.org/~klausman/python-3.1.2-testrun.log and also attached to this bug report. gcc is 4.4.3 glibc is 2.11.2 Kernel is 2.6.33 If more info or access to an alpha machine is needed, feel free to contact me or the alpha arch team at gentoo (al...@gentoo.org) -- components: Tests files: python-3.1.2-testrun.log messages: 112319 nosy: klausman priority: normal severity: normal status: open title: Python 3.1.2 test suite segfaults on the alpha architecture type: crash versions: Python 3.2 Added file: http://bugs.python.org/file18308/python-3.1.2-testrun.log ___ Python tracker <http://bugs.python.org/issue9447> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9447] Python 3.1.2 test suite segfaults on the alpha architecture
Tobias Klausmann added the comment: Nevermind, we messed this up ourselves. Sorry for the noise. -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue9447> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5322] Python 2.6 object.__new__ argument calling autodetection faulty
Tobias Hansen added the comment: This change breaks backward compatibility in Python 2.7. This is the example that also broke in #25731. In that case the change was reverted. See https://bugs.python.org/issue25731#msg262922 $ cat foo.pxd cdef class B: cdef object b $ cat foo.pyx cdef class A: pass cdef class B: def __init__(self, b): self.b = b $ cat bar.py from foo import A, B class C(A, B): def __init__(self): B.__init__(self, 1) C() $ cython foo.pyx && gcc -I/usr/include/python2.7 -Wall -shared -fPIC -o foo.so foo.c $ python -c 'import bar' Traceback (most recent call last): File "", line 1, in File "bar.py", line 7, in C() TypeError: foo.A.__new__(C) is not safe, use foo.B.__new__() -- nosy: +thansen ___ Python tracker <http://bugs.python.org/issue5322> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17363] Argument Mixup in PyState_AddModule
New submission from Tobias Becker: http://docs.python.org/3.3/c-api/module.html?highlight=pymodule#PyState_AddModule the arguments of PyState_AddModule are in wrong order: currently: int PyState_AddModule(PyModuleDef *def, PyObject *module) correct: int PyState_AddModule(PyObject *module,PyModuleDef *def) -- assignee: docs@python components: Documentation messages: 183564 nosy: Tobias.Becker, docs@python priority: normal severity: normal status: open title: Argument Mixup in PyState_AddModule versions: Python 3.3 ___ Python tracker <http://bugs.python.org/issue17363> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28355] wsgiref simple_server PATH_INFO treats slashes and %2F the same
New submission from Tobias Dammers: The WSGI reference implementation does not provide any means for application code to distinguish between the following request lines: GET /foo/bar HTTP/1.1 GET /foo%2Fbar HTTP/1.1 Now, the relevant RFC-1945 (https://tools.ietf.org/html/rfc1945#section-3.2) does not explicitly state how these should be handled by application code, but it does clearly distinguish encoded from unencoded forward-slashes in the BNF, which suggests that percent-encoded slashes should be considered part of a path segment, while unencoded slashes should be considere segment separators, and thus that the first URL is supposed to be interpreted as ['foo', 'bar'], but the second one as ['foo/bar']. However, the 'PATH_INFO' WSGI environ variable contains the same string, '/foo/bar', in both cases, making it impossible for application code to handle the difference. I believe the underlying issue is that percent-decoding (and decoding URLs into UTF-8) happens before interpreting the 'PATH_INFO', which is unavoidable because of the design decision to present PATH_INFO as a unicode string - if it were kept as a bytestring, then interpreting it would remain the sole responsibility of the appli cation code; if it were a fully parsed list of unicode path segments, then the splitting could be implemented correctly. Unfortunately, I cannot see a pleasant way of fixing this without breaking a whole lot of stuff, but maybe someone else does. It's also very possible that I interpret the RFC incorrectly, in which case please enlighten me. -- messages: 278032 nosy: tdammers priority: normal severity: normal status: open title: wsgiref simple_server PATH_INFO treats slashes and %2F the same type: behavior versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue28355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18182] xml.dom.createElement() does not take implicit namespaces into account
New submission from Alexander Tobias Heinrich: First of all, I am not sure, if this is a bug in python itself - it could as well be a bug in the py-dom-xpath module (http://code.google.com/p/py-dom-xpath) or not a bug at all (but I find the latter to be highly unlikely). Consider an XML document such as: If one creates a new Chimp-element using the xml.dom.createElement() function and then appends it to the Compound element, then the xpath module will not find the element unless the whole document is saved and then re-parsed. Creating the element with xml.dom.createElementNS() and thus explicitly specifying its namespace works around the problem. I consider this to be a bug, because in both cases, xml.dom will create the same valid XML, so I believe xpath should produce the same results in both cases, too. My believe, that the bug is in xml.dom is just a feeling and I could be wrong. I imagine, that xml.dom.createElement() forgets about adding the new element to its inherited namespace and adds it to the null namespace instead. In consequence xpath doesn't find the new element. I originally posted a more verbose explanation of this issue to StackOverflow (see http://stackoverflow.com/questions/16980521/python-xpath-find-wont-find-new-elements-if-they-were-added-without-namespac ), because I was unsure about whether this is a bug or not - and if it was, then in which module. Because I did not receive any feedback on that post, I have now decided to file it here as a bug report. I attached a sample script that demonstrates the problem if (xpath dependency is installed). I tested it under Windows with Python 2.7.5 and 2.7.4. -- components: XML files: pydomprob.py messages: 190906 nosy: Alexander.Tobias.Heinrich priority: normal severity: normal status: open title: xml.dom.createElement() does not take implicit namespaces into account type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file30527/pydomprob.py ___ Python tracker <http://bugs.python.org/issue18182> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27433] Missing "as err" in Lib/socket.py
New submission from Tobias Burnus (PDF): Cf. https://hg.python.org/cpython/file/tip/Lib/socket.py#l261 In _sendfile_use_sendfile, one has: try: fsize = os.fstat(fileno).st_size except OSError: raise _GiveupOnSendfile(err) # not a regular file I think the "err" in the last line will only work if there is an "as err" in the "except" line. -- components: Library (Lib) messages: 269656 nosy: Tobias Burnus (PDF) priority: normal severity: normal status: open title: Missing "as err" in Lib/socket.py versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue27433> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
