[issue5048] Extending itertools.combinations
New submission from Konrad : The function itertools.combinations might benefit from making the 'r' (length of the combinations) argument optionally a sequence. With that change one could call combinations(sequence, [2, 3]) in order to get all combinations of length 2 and 3. In particular, one could call combinations(sequence, range(len(sequence)) in order to get *all* combinations of given sequence. The change would be backwards compatible as it would check for sequential arguments. -- components: Library (Lib) messages: 80492 nosy: konryd, rhettinger severity: normal status: open title: Extending itertools.combinations type: feature request ___ Python tracker <http://bugs.python.org/issue5048> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5048] Extending itertools.combinations
Konrad added the comment: I'm afraid I don't have any real-world use cases. Originally, I assumed that dropping the length argument will make the function iterate over *all* combinations, which would enable me to write somehow twisted, one- liner for _inefficiently_ solving knapsack problem. max((comb for comb in all_combinations(zip(weights, values)) if sum(map(itemgetter(0), comb)) < LIM), key=lambda comb: sum(map(itemgetter(1), comb))) But unfortunately, this is far from being 'compelling'. Regarding other issues you raised: I think it would be pretty clear for the user, that the length of every combination might vary - that's what he asked for. And the length might be computed by summing the lengths counted using given formula, which is (for me at last) still explicit enough. ___ Python tracker <http://bugs.python.org/issue5048> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8312] Add post/pre hooks for distutils commands
Changes by Konrad : -- nosy: -Konrad.Delong ___ Python tracker <http://bugs.python.org/issue8312> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26425] 'TypeError: object of type 'NoneType' has no len()' in 'splitdrive'
New submission from Konrad:
Hello,
when trying to build the 'bsddb3' extension (from
https://pypi.python.org/pypi/bsddb3/6.1.1) which is necessary for Gramps (
https://gramps-project.org/wiki/index.php?title=Install_latest_BSDDB )I get the
error 'TypeError: object of type 'NoneType' has no len()'. MinGW32 which I've
installed for this purpose seems to work.
Here is the Traceback:
Detected Berkeley DB version 6.0 from db.h
running build
running build_py
running build_ext
building 'bsddb3._pybsddb' extension
c:\temp\mingw\bin\gcc.exe -mdll -O -Wall -IC:/Temp/BerkelyDB/include/ -Ic:\temp\
python34\include -Ic:\temp\python34\include -c Modules/_bsddb.c -o build\temp.wi
n32-3.4\Release\modules\_bsddb.o
writing build\temp.win32-3.4\Release\modules\_pybsddb.def
Traceback (most recent call last):
File "setup3.py", line 527, in
'Programming Language :: Python :: 3.4',
File "c:\temp\python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "c:\temp\python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "c:\temp\python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "c:\temp\python34\lib\distutils\command\build.py", line 126, in run
self.run_command(cmd_name)
File "c:\temp\python34\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "c:\temp\python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "c:\temp\python34\lib\site-packages\setuptools\command\build_ext.py",
line 49, in run
_build_ext.run(self)
File "c:\temp\python34\lib\distutils\command\build_ext.py", line 339, in run
self.build_extensions()
File "c:\temp\python34\lib\distutils\command\build_ext.py", line 448, in
build_extensions
self.build_extension(ext)
File "c:\temp\python34\lib\site-packages\setuptools\command\build_ext.py",
line 174, in build_extension
_build_ext.build_extension(self, ext)
File "c:\temp\python34\lib\distutils\command\build_ext.py", line 535, in
build_extension
target_lang=language)
File "c:\temp\python34\lib\distutils\ccompiler.py", line 717, in
link_shared_object
extra_preargs, extra_postargs, build_temp, target_lang)
File "c:\temp\python34\lib\distutils\cygwinccompiler.py", line 248, in link
target_lang)
File "c:\temp\python34\lib\distutils\unixccompiler.py", line 157, in link
libraries)
File "c:\temp\python34\lib\distutils\ccompiler.py", line 1092, in
gen_lib_options
opt = compiler.runtime_library_dir_option(dir)
File "c:\temp\python34\lib\distutils\unixccompiler.py", line 224, in
runtime_library_dir_option
compiler = os.path.basename(sysconfig.get_config_var("CC"))
File "c:\temp\python34\lib\ntpath.py", line 246, in basename
return split(p)[1]
File "c:\temp\python34\lib\ntpath.py", line 217, in split
d, p = splitdrive(p)
File "c:\temp\python34\lib\ntpath.py", line 159, in splitdrive
if len(p) > 1:
TypeError: object of type 'NoneType' has no len()
This behavior possibly relates to other issues
(http://bugs.python.org/issue21336, http://bugs.python.org/issue22587) but I'm
not sure about it.
Sorry for my poor understanding of these things.
Thanks,
Konrad
--
components: Windows
messages: 260783
nosy: Konrad, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: 'TypeError: object of type 'NoneType' has no len()' in 'splitdrive'
type: compile error
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue26425>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3588] sysconfig variable LINKFORSHARED has wrong value for MacOS X framework build
New submission from Konrad Hinsen <[EMAIL PROTECTED]>: On a MacOS X framework build, the LINKFORSHARED variable obtained from distutils.sysconfig.get_config_vars() has the value -u _PyMac_Error Python.framework/Versions/2.5/Python The last item is incomplete, it needs to be prefixed with the path in which the Python framework is installed. Looking at config/Makefile (from which Distutils takes the variables), I find LINKFORSHARED= -u _PyMac_Error $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK) and PYTHONFRAMEWORKDIR= Python.framework One fix would be to use PYTHONFRAMEWORKINSTALLDIR instead of PYTHONFRAMEWORKDIR in the definition of LINKFORSHARED, but I don't know if this could have undesirable effects on the build process. -- components: Distutils messages: 71324 nosy: khinsen severity: normal status: open title: sysconfig variable LINKFORSHARED has wrong value for MacOS X framework build type: behavior versions: Python 2.5 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3588> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3646] MacOS X framework install to non-standard directory fails
New submission from Konrad Hinsen <[EMAIL PROTECTED]>: The file Mac/README in Python 2.6b3 says: Installing in another place, for instance $HOME/Library/Frameworks if you have no admin privileges on your machine, has only been tested very lightly. This can be done by configuring with --enable- framework=$HOME/Library/Frameworks. The other two directories, "/Applications/MacPython-2.6" and /usr/local/bin, will then also be deposited in $HOME. This is sub-optimal for the unix tools, which you would want in $HOME/bin, but there is no easy way to fix this right now. In reality, the framework is installed in the directory given on the command line, but the installer then wants to write the app to /Applications/MacPython-2.6 and crashes due to lack of write permission: DYLD_FRAMEWORK_PATH=/shared_scratch/Python-2.6b3: ../python.exe ./scripts/Buil\ dApplet.py \ --destroot "" \ -- python=/shared_scratch/Library/Frameworks/Python.framework/Versions/2\ .6/Resources/Python.app/Contents/MacOS/Python`test -f "/shared_scratch/Library/\ Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/M acOS/Py\ thon-32" && echo "-32"` \ --output "/Applications/Python 2.6/Build Applet.app" \ ./scripts/BuildApplet.py kCGErrorRangeCheck : Window Server communications from outside of session allow\ ed for root and console user only ./scripts/BuildApplet.py:34: DeprecationWarning: catching of string exceptions \ is deprecated except buildtools.BuildError, detail: Traceback (most recent call last): File "./scripts/BuildApplet.py", line 149, in main() File "./scripts/BuildApplet.py", line 33, in main buildapplet() File "./scripts/BuildApplet.py", line 116, in buildapplet progress=verbose, destroot=destroot) File "/Volumes/User data/Scratch/Python-2.6b3/Lib/plat- mac/buildtools.py", li\ ne 115, in process copy_codefragment, raw, others, filename, destroot) File "/Volumes/User data/Scratch/Python-2.6b3/Lib/plat- mac/buildtools.py", li\ ne 140, in process_common is_update, raw, others, filename, destroot) File "/Volumes/User data/Scratch/Python-2.6b3/Lib/plat- mac/buildtools.py", li\ ne 327, in process_common_macho builder.build() File "/Volumes/User data/Scratch/Python-2.6b3/Lib/plat- mac/bundlebuilder.py",\ line 147, in build os.mkdir(builddir) OSError: [Errno 13] Permission denied: '/Applications/Python 2.6' make[1]: *** [install_BuildApplet] Error 1 make: *** [frameworkinstallapps] Error 2 -- components: Build messages: 71728 nosy: khinsen severity: normal status: open title: MacOS X framework install to non-standard directory fails versions: Python 2.6 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3646> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7946] Convoy effect with I/O bound threads and New GIL
Changes by Konrad Delong : -- nosy: +konryd ___ Python tracker <http://bugs.python.org/issue7946> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8312] Add post/pre hooks for distutils commands
Changes by Konrad Delong : -- nosy: +konryd ___ Python tracker <http://bugs.python.org/issue8312> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43252] deepcopy of weakref proxies
Konrad Schwarz added the comment: Well, in the example code, the memo dictionary contains the (hard) reference to newly created objects but ultimately, at the close of the deepcopy, the objects are (hard) referenced using the normal "child" attributes. I don't however know how this would work if this was implemented within the weakref code but I expect that their __deepcopy__ method also has access to the memo variable. If, at the close of deepcopy and the destruction of memo, no (hard) references exist to an object, then it would be collected, but that matches standard weak reference semantics. Even if it were not possible to solve this programmatically within Python, I think the problem (and e.g. the workaround given below) should be mentioned in the documentation of deepcopy and/or weakref. -- ___ Python tracker <https://bugs.python.org/issue43252> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45639] Support modern image formats in MIME types
New submission from Adam Konrad : Modern image types webp and avif are not recognized by the mimetypes module. Problem: Many tools are written in Python and running on macOS. Good example is the AWS CLI. Running commands like "s3 sync" will save files with .webp and .avif extensions with incorrect "binary/octet-stream" Content-Type to S3. This creates additional problems with serving these resources over HTTP. The webp and avif image types are supported by most browsers: https://caniuse.com/#feat=webp https://caniuse.com/#feat=avif While webp is fully supported and largely used, it is not officially registered with IANA. Avif is currently less popular, it is fully registered with IANA. https://www.iana.org/assignments/media-types/media-types.xhtml Please consider the attached GitHub PR as a fix to these MIME Content-Type issues. -- components: Library (Lib) messages: 405145 nosy: adamkonrad priority: normal severity: normal status: open title: Support modern image formats in MIME types type: enhancement versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue45639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45639] Support modern image formats in MIME types
Change by Adam Konrad : -- keywords: +patch pull_requests: +27523 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29259 ___ Python tracker <https://bugs.python.org/issue45639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40171] Attempting to import inaccessible package imports an empty package
New submission from Konrad Borowski : The attached shell program returns `AttributeError: module 'mod' has no attribute 'x'`. I would rather expect `ImportError` instead of loading an empty package due to permission error causing Python module to be inaccessible. -- files: demonstration messages: 365699 nosy: xfix priority: normal severity: normal status: open title: Attempting to import inaccessible package imports an empty package type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file49029/demonstration ___ Python tracker <https://bugs.python.org/issue40171> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40171] Attempting to import inaccessible package imports an empty package
Konrad Borowski added the comment: The attached example works on Python 2, but it breaks with Python 3. This may be a regression. -- ___ Python tracker <https://bugs.python.org/issue40171> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43252] deepcopy of weakref proxies
New submission from Konrad Schwarz : copy.deepcopy()-ing a tree structure that has internal parent and cross-reference links implemented as weakref.proxy() objects causes the weak reference proxies themselves to be copied (still refering to their original referents) rather than weak references to deep-copied referents to be created. A workaround is to add the following __deepcopy__ method to affected classes: def __deepcopy__ (self, memo): # taken from Stackoverflow: "How to override the # copy deepcopy operations for a python object" # and "Python: dereferencing weakproxy" cls = self.__class__ result = cls.__new__ (cls) memo [id (self)] = result for k, v in self.__dict__.items (): if isinstance (v, weakref.ProxyType): new_v = weakref.proxy (copy.deepcopy ( v.__repr__.__self__, memo)) else: new_v = copy.deepcopy (v, memo) setattr (result, k, new_v) return result -- components: Library (Lib) messages: 387226 nosy: konrad.schwarz priority: normal severity: normal status: open title: deepcopy of weakref proxies type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43252> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43383] imprecise handling of weakref callbacks
New submission from Konrad Schwarz :
I am seeing the following non-deterministic behavior:
My code processes DeviceTree, a tree-based specification format for hardware
descriptions, that includes cross-references ("phandles"). For all intents and
purposes, this format is similar to XML; phandles are analog to ID/IDREFS.
To prevent reference cycles and avoid the need for garbage collection, my code
uses weakref.proxy for parent pointers and weakref.ref for cross-references.
My goal is to provide a "projection" operation on a DeviceTree: creating
derived DeviceTrees that model subsets of the hardware (this is to partition
the hardware into multiple independent sub-machines). The projection is
specified by newly introduced nodes and attributes (aka properties) in the
tree; phandles are used to indicate which part belongs to which partition.
Python weak references provide a callback interface to indicate the demise of
their referents and my code uses that to prune the tree: e.g., if a node
modeling a partition is deleted, nodes that reference that node (i.e., indicate
they belong to that partition) are deleted in the corresponding weakref
callback. So technically, the code implicitly uses the interpreters list of
weak referrers (__weakref__) to find and execute code on them when the
referent's state changes.
This works exactly as envisioned when single-stepping in PDB.
When running at full speed however, I see that weak reference callbacks are
being triggered after the corresponding weak reference has been deleted with
del (the weak reference is a value of a Python dict holding a node's
attributes.)
I suspect that this is because of some batching or deferred processing in the
Python interpreter.
Ultimately, this is a violation of the semantics and must be classified as a
bug.
However, in my case, it would suffice to have a "memory barrier" type of
operation that flushes the queue of deferred deletions before continuing.
Something like that must exist, because single stepping in PDB is successful.
Initial tests of calling the garbage collector to this end were inconclusive,
unfortunately.
--
components: Interpreter Core
messages: 387999
nosy: konrad.schwarz
priority: normal
severity: normal
status: open
title: imprecise handling of weakref callbacks
type: behavior
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue43383>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43383] imprecise handling of weakref callbacks
Konrad Schwarz added the comment: > If I understand correctly, the reported bug is that you're seeing a weakref > callback being executed > after the weakref.ref instance it's attached to is deleted. Is that correct? Exactly. I del what should be the only reference to the weakref.ref/proxy, then del the weakref.ref's referent. The weakref.ref's callback is executed. > Do you have a minimal example we can use to reproduce the effect? Without > such an example, there's not > much of a realistic path to moving this forward. Unfortunately, not at the moment. > However, my suspicion is that that's not what's actually happening here. I > suspect that when you say > "after the corresponding weak reference has been deleted with del", that the > "del" statement you refer > to is not actually deleting the last reference to the weakref.ref object, so > the weakref still exists > after the "del". One easy way that this could happen is if the weakref is > part of a reference cycle > (and I know from personal experience that it's horribly easy to accidentally > *create* reference cycles > via weakref callbacks, especially if those callbacks refer to instance > methods). I tried to be as punctilious as I could to prevent this sort of thing from happening. The weird thing is that the error does not occur when I single step the code in PDB. It must be something internal to CPython, some sort of optimization. Are local variables/stack frames cleaned lazily? Is the last accessed value of a dictionary cached internally, increasing its reference count? I've worked around the problem by double checking in the callback method that it actually should execute; i.e. in my case, that the object has not been removed (del) from a dictionary (which should be the single reference to the object). If the problem does indeed lie with CPython, but has too much of a performance/maintenance impact, perhaps a caveat could be added to the documentation to the effect of "The lifetime of an object may be longer than what a programmer expects causing a weakref's callback to be invoked at a surprising time; it is good practice to program the callback defensively." -- ___ Python tracker <https://bugs.python.org/issue43383> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43383] imprecise handling of weakref callbacks
Konrad Schwarz added the comment: Unfortunately, my management has impressed other priorities upon me; I can't delve deeper into this subject at the moment. My takeaway is that the error very likely lies on my side; maybe I need to re-check local variables and del them explicitly. In any case, I can work around the situation by being extra careful in the callback. I don't know how well all of this would work in a truly multi-threaded environment, but the application doesn't require that at all. In any case, thank you for the insightful comments and the willingness to understand my problem! I certainly have learned a lot. -- ___ Python tracker <https://bugs.python.org/issue43383> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43383] imprecise handling of weakref callbacks
Change by Konrad Schwarz : -- resolution: -> postponed stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue43383> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42324] Doctest: directives
New submission from Konrad Schwarz : In both PDF and HTML, the examples for Doctest directives don't actually show the directives themselves, perhaps because they syntactically start with a #. -- assignee: docs@python components: Documentation messages: 380752 nosy: docs@python, konrad.schwarz priority: normal severity: normal status: open title: Doctest: directives type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue42324> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42324] Doctest: directives
Konrad Schwarz added the comment: Yes it is. On Wed, Nov 11, 2020 at 1:06 PM Karthikeyan Singaravelan < rep...@bugs.python.org> wrote: > > Karthikeyan Singaravelan added the comment: > > Is this similar to https://bugs.python.org/issue36675 ? > > -- > nosy: +xtreak > > ___ > Python tracker > <https://bugs.python.org/issue42324> > ___ > -- Mit freundlichen Grüßen Konrad Schwarz -- ___ Python tracker <https://bugs.python.org/issue42324> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36244] Lock release fails under windows
New submission from Konrad Ciecierski : In python 3.7.2 when the subprocess releases the acquired lock, the OSError occurs: "OSError: [WinError 6] The handle is invalid". Problem does not occur under Ubuntu 18. File "multip-test.py", line 13, in worker lock.release() OSError: [WinError 6] The handle is invalid -- components: Interpreter Core, Windows files: multip-test.py messages: 337527 nosy: Konrad Ciecierski, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Lock release fails under windows type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48198/multip-test.py ___ Python tracker <https://bugs.python.org/issue36244> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36244] Lock release fails under windows
Konrad Ciecierski added the comment: Yes, it is the same case. Still, the current official (3.7.2) release is affected by this problem. -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue36244> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8312] Add post/pre hooks for distutils commands
Konrad Delong added the comment: I see no reasons to limit the command set to just these two. -- nosy: +Konrad.Delong ___ Python tracker <http://bugs.python.org/issue8312> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
