[issue10890] IDLE Freezing

[James]

New submission from James :

Recently installed Python 2.7.1 on my MacBook running OS X 10.6.6 and have not 
been able to use IDLE without it freezing. When I force quit it doesn't show 
that it's not responding. I can run scripts fine, but if I were to try to 
copy-paste, or save via command-S, it will lock up completely.

--
assignee: ronaldoussoren
components: Macintosh
messages: 126070
nosy: jamgood96, ronaldoussoren
priority: normal
severity: normal
status: open
title: IDLE Freezing
type: crash
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue10890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11122] bdist_rpm fails

[James]

New submission from James :

Hi distutils,

When I run:

./setup.py bdist --formats=rpm

on my source directory, I get the error:

rpm -ba --define _topdir /home/james/code/scantran/build/bdist.linux-x86_64/rpm 
--clean build/bdist.linux-x86_64/rpm/SPECS/scantran.spec
-ba: unknown option
error: command 'rpm' failed with exit status 1

It seems the problem can be fixed by installing the rpm-build package (on 
fedora 14 anyways) and so perhaps something should be done so that this is more 
obvious...

HTH,
James

--
assignee: tarek
components: Distutils
messages: 127942
nosy: eric.araujo, purpleidea, tarek
priority: normal
severity: normal
status: open
title: bdist_rpm fails
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue11122>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11122] bdist_rpm fails

[James]

James  added the comment:

In the source for distutils it seems to attempt to use 'rpmbuild' if it exists, 
but otherwise falls back on regular 'rpm', however in my rpm:

$ rpm --version
RPM version 4.8.1

this fails as there is no -ba option.

James

--

___
Python tracker 
<http://bugs.python.org/issue11122>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11122] bdist_rpm fails

[James]

James  added the comment:

I'll write a docs and script patch for this next week...

I'm happy to do the work,

Thanks for the comments.

James

--

___
Python tracker 
<http://bugs.python.org/issue11122>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

i'm fine with that and willing to contribute patches, however i would feel 
better if whoever upstream was, was more supportive of the idea.
someone let me know.

a thought:
- it's true (as mentioned) that distclean isn't necessarily directly related to 
distributing python scripts, however (in my mind anyways) i see distutils as a 
replacement for common things we do in makefiles. so as a result if we did this 
we could get rid of makefiles and provide useful "action" targets such as 
distclean, test, etc... 100% pure python if needed...

--

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39407] Bitfield Union does not work for bit widths greater than 8 bits

[James]

New submission from James :

Creating a Bitfield from a ctypes union and structure results in unexpected 
behaviour. It seems when you set the bit-width of a structure field to be 
greater than 8 bits it results in the subsequent bits being set to zero.



class BitFieldStruct(ctypes.LittleEndianStructure):
_fields_ = [
("long_field", C_UINT32, 29),
("short_field_0", C_UINT8, 1),
("short_field_1", C_UINT8, 1),
("short_field_2", C_UINT8, 1),
]


class BitField(ctypes.Union):
_anonymous_ = ("fields",)
_fields_ = [
("fields", BitFieldStruct),
("as32bit", C_UINT32)
]

def test_bit_field_union():
f = BitField()
f.as32bit = int.from_bytes([255, 255, 255, 255], byteorder='little')

assert f.long_field == int.from_bytes([255, 255, 255, 31], 
byteorder='little')
assert f.short_field_0 == 1
assert f.short_field_1 == 1
assert f.short_field_2 == 1

test_bit_field_union()  # this call will fail with an assertion error


Equivalent C which does not fail https://rextester.com/FWV78514

I'm running on Ubuntu 16.04 with python3.6 but I have tested on 3.5, 3.7 and on 
repl.it with the same behaviour.

It seems as though setting any of the struct fields to be greater than 8 bit 
width results in any of the following fields being set to zero.

--
components: ctypes
files: python_struct_union_bug.py
messages: 360372
nosy: jschulte
priority: normal
severity: normal
status: open
title: Bitfield Union does not work for bit widths greater than 8 bits
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48856/python_struct_union_bug.py

___
Python tracker 
<https://bugs.python.org/issue39407>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30806] netrc.__repr__() is broken for writing to file

[James]

New submission from James:

Have any valid .netrc file. For testing purposes you can use this:

machine abc.xyz login myusername password mypassword

The documentation for netrc.__repr__() states that it "dumps the class data as 
a string in the format of a netrc file". However, when you try to actually do 
this, you'll encounter a nasty bug. This can be seen by running the follow 
commands:

auth = netrc.netrc(os.path.expanduser(r"~\.netrc"))
print(auth.__repr__())

The expected output is:

machine abc.xyz
login myusername
password mypassword

The actual output is:

machine abc.xyz
login 'myusername'
password 'mypassword'

If you write this back out to the .netrc file, authentication will fail since 
incorrect username/password (with ' character at beginning at end) will be 
passed.

--
components: Library (Lib)
messages: 297296
nosy: Bezier89
priority: normal
pull_requests: 2550
severity: normal
status: open
title: netrc.__repr__() is broken for writing to file
type: behavior
versions: Python 3.6

___
Python tracker 
<http://bugs.python.org/issue30806>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15753] No-argument super in method with variable arguments raises SystemError

[James]

New submission from James:

For example:

Python 3.2.2 (default, Feb 10 2012, 09:23:17) 
[GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A:
... def f(*args):
... print(super().__repr__())
... 
>>> A().f()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in f
SystemError: super(): no arguments

(There is no change in the current development version 3.3.0b2+)

I guess that the problem here is related to the fact that the super call makes 
sense if we call A().f() but not if we call A.f() (which is allowed by the 
method signature)?  I understand that not including self in the signature is 
almost always bad style, but occasionally it is necessary, for example if you 
want to allow arbitrary keyword arguments as dict.update does.

Also, how come using the no-argument form of super outside a method raises 
SystemError - isn't that supposed to be for internal errors?

--
components: Interpreter Core
messages: 168757
nosy: james.sanders
priority: normal
severity: normal
status: open
title: No-argument super in method with variable arguments raises SystemError
type: behavior
versions: Python 3.2, Python 3.3

___
Python tracker 
<http://bugs.python.org/issue15753>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15753] No-argument super in method with variable arguments raises SystemError

[James]

James added the comment:

I've attached a patch that I think fixes the variable arguments problem, and 
changes the SystemErrors that can be obtained by misusing super() into 
RuntimeErrors (I assume that's more appropriate?).  There are three more 
SystemErrors I'm not sure about: "super(): no code object", "super(): bad 
__class__ cell", and "super(): empty __class__ cell" - can they only be caused 
by internal bugs?

--
keywords: +patch
Added file: http://bugs.python.org/file26976/issue15753_varargsuper.patch

___
Python tracker 
<http://bugs.python.org/issue15753>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15753] No-argument super in method with variable arguments raises SystemError

[James]

James added the comment:

It turns out I don't really understand how frame objects work.  My patch can 
crash python if you do this:

>>> class A:
... def f(*args):
... args = 1
... print(super())
... 
>>> A().f()
python: Objects/typeobject.c:6516: super_init: Assertion 
`((PyObject*)(obj))->ob_type))->tp_flags & ((1L<<26))) != 0)' failed.
Aborted

Is there a way of checking if the first argument has been overwritten, or do 
you just have to assume that if it is still a tuple, it hasn't been?  Should 
the super documentation mention that assigning to self (or args in this case) 
can make the no-argument version work incorrectly?

--

___
Python tracker 
<http://bugs.python.org/issue15753>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15753] No-argument super in method with variable arguments raises SystemError

[James]

James added the comment:

Sorry, I wasn't very clear.  super() currently works by assuming that self is 
the first entry in f_localsplus, which is defeated, for example, by doing:

>>> class A:
... def f(self):
... del self
... super()
... 
>>> A().f()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in f
SystemError: super(): arg[0] deleted
>>> class B:
... def f(self):
... self = 1
... super()
... 
>>> B().f()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in f
TypeError: super(type, obj): obj must be an instance or subtype of type

I don't know if this is the intended behaviour (in which case, my assertion 
that obj is a tuple should be replaced by just checking whether it is a tuple 
and raising an exception otherwise; and I suppose the super documentation 
should mention this caveat), or whether this should be fixed somehow.

--

___
Python tracker 
<http://bugs.python.org/issue15753>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6633] No handlers could be found for logger

[James]

New submission from James :

I was trying to suppress the error message as shown in the title, when I
found out (by searching through the source) that there is a NullHandler
for precisely this purpose.

http://svn.python.org/view/python/trunk/Lib/logging/__init__.py?r1=66211&r2=67511

do you think that:
1) this could be documented maybe here (
http://docs.python.org/library/logging.html#handler-objects ) i suppose?
and:
2) this null handler doesn't seem to exist in:
Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49) 
[GCC 4.3.2] on linux2
is this likely to get backported to 2.5? at the moment i've just
included the simple NullHandler class into my code.
3) also not my business really, but should it belong in the handlers.py
file?

thanks for all your hard work, i hope the comments are useful!

--
components: Library (Lib)
messages: 91231
nosy: purpleidea
severity: normal
status: open
title: No handlers could be found for logger
versions: Python 2.5

___
Python tracker 
<http://bugs.python.org/issue6633>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6633] No handlers could be found for logger

[James]

James  added the comment:

very well,
i didn't notice the
http://docs.python.org/library/logging.html#configuring-logging-for-a-library
and i thank you for your time and efforts!

cheers,
_J

--

___
Python tracker 
<http://bugs.python.org/issue6633>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6786] readline and zero based indexing

[James]

New submission from James :

why is it that the zeroth readline history item is seemingly always
none. I would expect this to support zero-based indexing in python, but
perhaps I have missed some detail in readline somewhere. Cheers,
_J


ja...@work:~$ python
Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.get_history_item(0)
>>> readline.get_history_item(0) is None
True
>>> 
ja...@work:~$

--
components: Extension Modules
messages: 91979
nosy: purpleidea
severity: normal
status: open
title: readline and zero based indexing
type: behavior
versions: Python 2.5

___
Python tracker 
<http://bugs.python.org/issue6786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6872] Support system readline on OS X 10.6

[James]

James  added the comment:

it seems to me, that any and all readline interfaces should/could
standardize to the indexing scheme as used by the language; maybe i'm
wrong, but since python is zero based, so could the readline interfaces.

it's definitely more logical for a python programmer to expect
zero-based, and the code will match with the python code.

i would propose that everything be zero-based;
this is duplicate/similar of/to http://bugs.python.org/issue6786

--
nosy: +purpleidea

___
Python tracker 
<http://bugs.python.org/issue6872>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6786] readline and zero based indexing

[James]

James  added the comment:

@mark: thanks for the comment; i suppose we should investigate why and
if c readline is 1 based...

--

___
Python tracker 
<http://bugs.python.org/issue6786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6786] readline and zero based indexing

[James]

James  added the comment:

i found this:
http://tiswww.case.edu/php/chet/readline/history.html
search for: "Variable: int history_base"
perhaps we can set this to 0 in the python bindings.
more so, perhaps someone is using 1 because they made a mistake?

--

___
Python tracker 
<http://bugs.python.org/issue6786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7315] os.path.normpath doesn't normalize ../path/something.py

[James]

New submission from James :

os.path.normpath doesn't normalize paths that start with ../
you would expect the final output line in the secpnd run to read:
"normpath: badnormpath.py" instead of: "normpath: ../tmp/badnormpath.py"

example:

ja...@home:~$ cd tmp/
ja...@home:~/tmp$ cat badnormpath.py 
#!/usr/bin/python

import os.path

print '__file__: %s' % __file__
print 'normpath: %s' % os.path.normpath(__file__)

ja...@home:~/tmp$ ./badnormpath.py 
__file__: ./badnormpath.py
normpath: badnormpath.py
ja...@home:~/tmp$ ./../tmp/badnormpath.py 
__file__: ./../tmp/badnormpath.py
normpath: ../tmp/badnormpath.py
ja...@home:~/tmp$

--
components: Library (Lib)
messages: 95189
nosy: purpleidea
severity: normal
status: open
title: os.path.normpath doesn't normalize ../path/something.py
type: behavior
versions: Python 2.6, Python 2.7, Python 3.0

___
Python tracker 
<http://bugs.python.org/issue7315>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7315] os.path.normpath doesn't normalize ../path/something.py

[James]

James  added the comment:

i looked at the source for normpath.
i know that it doesn't look at the filesystem.
assuming you're not currently sitting at the root directory, in all? 
cases ../xyz brings you back to where you started. we expect normpath to 
clean up a path string, collapsing '../' (as documented), perhaps either 
the docs should state otherwise, or this should collapse dirs as long as 
they are present in the path given to it ?

as an aside is there any reason why normpath shouldn't look at the 
filesystem and decide if it makes sense to collapse a leading '../' when 
this case arises ?

--

___
Python tracker 
<http://bugs.python.org/issue7315>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

New submission from James :

Priority: 4
Keywords: patch, distutils, pyc
Comment:
I posted this on the distutils mailing list, and they said I should post
it here instead.

---
Hi,
I'm certaintly new to distutils and setuptools, however I figured I'd
send in this patch, and either it will get merged because it's a great
idea or someone will perhaps tell me why this doesn't exist already.
(or maybe it does and i can't find it)
In any case, it adds the pyc option to the clean command so that the
.pyc can be removed on request. Personally i'll have a [clean] pyc=1
option in my setup.cfg, but that's for my convenience.
cheers,

_J


---

ps: i wasn't able to pick a priority or keywords, there isn't a box to
type those in, however the titles are there. so either i don't have
permissions or there's a bug ?

--
files: clean.py.patch
keywords: patch
messages: 88512
nosy: purpleidea
severity: normal
status: open
title: Distutils doesn't remove .pyc files
type: behavior
versions: Python 2.5
Added file: http://bugs.python.org/file14112/clean.py.patch

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

Hi, the patch only removes them if one adds the --pyc option.
I think it is a good idea to have some option or target somewhere to
remove the types of files that can be regenerated because often
developers want to "get them out of the way". Example: I'll be working
on a project in my code directory, and it's nice to be able to do an "$
ls" without seeing extra files lying that clutter up the view. Perhaps
different people want to remove them for different reasons. I remember
searching for "remove .pyc" or something like it and it returned some
hits from other people wanting similar functionality. Thanks for your
time, I hope you include something like this patch in upstream.

_J

--

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

I could agree with R. David Murray, and I think that it's fine that this
be included under a dist clean command.
Ultimately I'm writing an application and I'm trying to use distutils
with it. I'll potentially run a: "$ setup.py build_ext -i" or whatever
it may be, and then I'll want to get rid of all the mess. So I'll want
to run a clean. If that clean won't get rid of .pyc for me in one
command, then i have to run a second shell script to clean my dir,
instead of sticking the two together. Which is why i think a clean --pyc
option is useful (off by default) and which I can easily enable in
setup.cfg with [clean] pyc=1
I think this is harmless. Anyone agree?

thanks!
_J

--

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

ps: included is a platform independent version of the code, so that it
doesn't depend on os.system() specific commands.

HTH,
_J

--
Added file: http://bugs.python.org/file14146/clean.py.patch

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

Antoine: Okay sorry not a mess then.
I just figure that if i'm using the distutils tool for doing all the fun
things to my local source directory that I potentially used to do with
say a makefile, then would it not be beneficial to have a useful
-option- (as included in the patch). It won't affect any previous
distutils setups, and can only benefit future users. Why not?

_J

--

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6142] Distutils doesn't remove .pyc files

[James]

James  added the comment:

Currently, I have (had) a make file with a clean target that would
remove these files. Would you recommend keeping this file and it's
associated functionality, or is the idea to be able to integrate this
into distutils and be able to do away with makefiles for python projects?

I'm aware that I can subclass Command and make my own "target"... I've
done this, but it seems to me it's better suited upstream.

--

___
Python tracker 
<http://bugs.python.org/issue6142>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6205] sdist doesn't include data_files

[James]

New submission from James :

Hi, I have shown the output from my terminal below, since it will be
easier to follow for explaining the bug.

ja...@computer:~/testsetup$ ls
helloworld2.py  image1.jpg  setup.py
ja...@computer:~/testsetup$ cat setup.py 
#!/usr/bin/python
import distutils.core   #from distutils.core import setup, Extension
import os

# build a list of modules required for setup function below
py_modules = []

py_modules.append('helloworld2')

distutils.core.setup(
name='helloworld2',
description='distutils test',
version='0.1',
author='James',
author_email='purplei...@gmail.com',
py_modules=py_modules,
# data_files: install directory, 
data_files=[('share/helloworld2', ['image1.jpg'])]
)

ja...@computer:~/testsetup$ ./setup.py sdist
running sdist
warning: sdist: manifest template 'MANIFEST.in' does not exist (using
default file list)
warning: sdist: standard file not found: should have one of README,
README.txt
writing manifest file 'MANIFEST'
creating helloworld2-0.1
making hard links in helloworld2-0.1...
hard linking helloworld2.py -> helloworld2-0.1
hard linking setup.py -> helloworld2-0.1
creating dist
tar -cf dist/helloworld2-0.1.tar helloworld2-0.1
gzip -f9 dist/helloworld2-0.1.tar
removing 'helloworld2-0.1' (and everything under it)
ja...@computer:~/testsetup$ 

as you will notice, the image1.jpg file does not get included in the
source distribution, and if i want to backup the entire dir/code of
everything to send to someone else.
perhaps this is a peculiarity of distutils. i realize i could write my
own manifest.in but then i have to specify *everything* and this isn't
automatic anymore.
this is definitely an issue since a user who downloads the sdist file
and runs an install will see:

error: can't copy 'image1.jpg': doesn't exist or not a regular file

this occurs because it obviously didn't get included in the sdist. the
same thing happens when sdist is run with --no-prune

i thought that perhaps i was using the wrong target so i tried a bdist
(tar.gz). in this case the image1.jpg file gets included, however
unpacking the directory doesn't give me a structure similar to the one
my code is originally maintained in.
so how do i use distutils to share *everything*, (eg: everything
specified in the setup.py directory) with my friends on the tubes?

if you want, i'll write a patch.

_J

--
assignee: tarek
components: Distutils
messages: 88945
nosy: purpleidea, tarek
severity: normal
status: open
title: sdist doesn't include data_files
type: behavior
versions: Python 2.5

___
Python tracker 
<http://bugs.python.org/issue6205>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6260] os.utime should allow None values for ATIME or MTIME

[James]

New submission from James :

Hi, in using os.utime, it's nice that you can specify `None' for the
second argument. However it would be even `nicer' to be able to specify
None for either (or potentially both) values for the argument in the
tuple. to emulate this, i've been using os.stat to read the value
beforehand, and just use os.utime with the new value, and the old one
pulled from stat()

HTH, _J

example:
os.utime('/dev/pts/1', None) WORKS
os.utime('/dev/pts/1') SHOULD WORK (allow second argument to default to
None)
os.utime('/dev/pts/1', (time.time(), None)) SHOULD WORK
os.utime('/dev/pts/1', (None, None)) SHOULD WORK
os.utime('/dev/pts/1', (None, time.time())) I GUESS SHOULD WORK

ps: if this is a feature you'd agree with, let me know any maybe i can
*try* to write a patch?

--
components: IO
messages: 89227
nosy: purpleidea
severity: normal
status: open
title: os.utime should allow None values for ATIME or MTIME
type: feature request
versions: Python 2.5

___
Python tracker 
<http://bugs.python.org/issue6260>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6260] os.utime should allow None values for ATIME or MTIME

[James]

James  added the comment:

very well, this is a good point.
i'm guessing nobody would every accept a patch for upstream utime? (in c)
thanks for your comment.

--

___
Python tracker 
<http://bugs.python.org/issue6260>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6205] sdist doesn't include data_files

[James]

James  added the comment:

great, thanks for the info.

--

___
Python tracker 
<http://bugs.python.org/issue6205>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8955] import doesn't notice changes to working directory

[James]

New submission from James :

Attempting to change the working directory and then import based on that change 
has no effect. Import seems impossible. Attached is tarball example. As seen 
below, bar1.py can import foo from src, however bar2.py bar3.py and bar4.py 
cannot, despite their respective scripts changing their cwd. Below is results 
of a terminal session. Thanks in advance.

ja...@hostname:~$ tree import_bug/
import_bug/
|-- bar1.py
|-- __init__.py
|-- src
|   |-- foo.py
|   `-- __init__.py
`-- test
|-- bar2.py
|-- bar3.py
|-- bar4.py
|-- bar5.py
`-- __init__.py

2 directories, 9 files
ja...@hostname:~$ cat import_bug/src/foo.py 
# this is foo.py
print('this is foo.py')

ja...@hostname:~$ cat import_bug/bar1.py 
# this is bar1.py
import os



print(os.getcwd())
from src import foo
print('this is bar1.py')

ja...@hostname:~$ cat import_bug/test/bar2.py 
# this is bar2.py
import os

os.chdir(os.path.join(os.path.dirname(__file__), os.pardir))

print(os.getcwd())
from src import foo
print('this is bar2.py')

ja...@hostname:~$ python import_bug/bar1.py 
/home/james
this is foo.py
this is bar1.py
ja...@hostname:~$ python import_bug/test/bar2.py 
/home/james/import_bug
Traceback (most recent call last):
  File "import_bug/test/bar2.py", line 7, in 
from src import foo
ImportError: No module named src
ja...@hostname:~$

--
components: Library (Lib)
files: import_bug.tar
messages: 107403
nosy: purpleidea
priority: normal
severity: normal
status: open
title: import doesn't notice changes to working directory
versions: Python 2.6, Python 3.1
Added file: http://bugs.python.org/file17600/import_bug.tar

___
Python tracker 
<http://bugs.python.org/issue8955>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6786] readline and zero based indexing

[James]

James  added the comment:

It's an incompatible change; it would definitely break my code, however I think 
it should be wishlisted for an API-break release like 3.5 or 4.0 or something 
like that. IMHO, the bindings should be "pythonic", even if the underlying 
library isn't.

In addition, maybe we could add a readline.set_zero_based_indexing() function? 
I could write a patch for this perhaps...

--

___
Python tracker 
<http://bugs.python.org/issue6786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6786] readline and zero based indexing

[James]

James  added the comment:

I'd be writing a patch which would allow a programmer the option to explicitly 
use/instantiate the library in a zero-based way. This way throughout their 
particular program, the indexing of elements could be consistent. Not having 
this causes you to have to reason differently about which parts should be 
list[i+1] versions list[i].

Would this be reasonable?
Thanks,

_J

--

___
Python tracker 
<http://bugs.python.org/issue6786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22691] A Better Help File

[James]

New submission from James:

Just the General Help that is in Python, doesn't really help.  Here's what 
would help, if every Module, had an example in code of how it was used instead 
of the Trees.  I mean, word trees, well that's what the writing reminds me of, 
is word trees like you'd produce in an English class.  But, a short working 
piece of code, does allot more for me, than a lecture, you'd have to take a 
course to understand because of it's use of it's own writing style.  It doesn't 
help to type help() and hit enter.

--
assignee: docs@python
components: Documentation
messages: 229796
nosy: FCK, docs@python
priority: normal
severity: normal
status: open
title: A Better Help File
type: enhancement
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue22691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22691] A Better Help File

[James]

James added the comment:

Just the General Help that is in Python, doesn't really help.  Here's what 
would help, if every Module, had an example in code of how it was used instead 
of the Trees.  I mean, word trees, well that's what the writing reminds me of, 
is word trees like you'd produce in an English class.  But, a short working 
piece of code, does allot more for me, than a lecture, you'd have to take a 
course to understand because of it's use of it's own writing style.  It doesn't 
help to type help() and hit enter.

Honestly, the old old version of Quick Basic, that once came with every version 
of windows back in the 90s.  It's help, is a template that I think Python, 
should follow.

--

___
Python tracker 
<http://bugs.python.org/issue22691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22692] Problems with Python's help()

[James]

New submission from James:

Hello,

I really think that Microsoft’s last release of Quick Basic 4.5 really had 
the ultimate of all help files.  Here’s why, you could cut and copy the code to 
the program you were working on, and then alter it to your program.  It was one 
of the nicer things you could do with the help file.  Because, tearing down the 
working example, usually just means customizing it to your needs, or wants.

You have several randomizing routines right?  How do you use these modules? 
 I have to say, looking at the help, should work for anyone that can’t leave it 
alone and learn to program if that’s what they want to do.  I was always a hunt 
an pick programmer.  I rather read through the help files and see what it can 
do, and then figure out what I want to do.  There’s no reason anyone couldn’t 
be self taught via reading the help file but, we should at least want it to 
work that way.

It’s fun to be on a ship with men.

James.

--
messages: 229798
nosy: FCK
priority: normal
severity: normal
status: open
title: Problems with Python's help()

___
Python tracker 
<http://bugs.python.org/issue22692>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22694] The help file issue I'm having.

[James]

New submission from James:

Hello,

Now, I really want you to think about the hunt and pick method of 
programming and learning how to program.  Being self taught, isn’t something 
that can happen unless, the authors of the software want people to learn how to 
use it.  Help files, are not typically designed that way but, there’s an easy 
way to design them that way.  Some of the strange notation that requires a 
course in computer science, is not what anybody needs to program a computer.  
The computer science method of explaining computer programming languages is all 
literally politics, or legal terms.  Let’s not make it harder than it really 
is, and keep both politics and legal terms out of this when in reality we just 
need a few terms to be define a function subroutine or suite of utilities and 
tools.  There’s science the right way, and we name things because they’ve never 
been named before, and then there’s politics, were we just talk too much about 
one dumb thing, or legal terms, and just re-invent the language around how many 
different ways we are capable of explaining the same damn thing differently, or 
making a new word out of lecture of human social situations and by defining one 
human social situation, come up with one legal term.

People work best one way, they learn by example.

James.

--
messages: 229800
nosy: FCK
priority: normal
severity: normal
status: open
title: The help file issue I'm having.

___
Python tracker 
<http://bugs.python.org/issue22694>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22691] A Better Help File

[James]

James added the comment:

I've written several languages, I'm no novice but, I also know when to brush 
up.Its just how I started, it looks like an opening for others.

-Original Message- 
From: R. David Murray
Sent: Wednesday, October 22, 2014 6:25 AM
To: geek.mo...@gmail.com
Subject: [issue22691] A Better Help File

R. David Murray added the comment:

The help isn't targeted at teaching you to use the module.  The help is 
targeted at *reminding* you how to use the module after you've read the full 
documentation, which usually does contain examples (though generally not at 
the top of the page...they are usually at the bottom or interspersed...it is 
a *reference* guide after all, the tutorial is a separate thing with yet a 
different target).  I'm afraid, though, that if you find the help to be a 
wall of words, you'll find the library reference worse.  You might be best 
serve by checking out the book/website "Python Module of the Week" 
(pymotw.com), which has a more tutorial style and more examples.

I've never see the Quick Basic style docs.  I don't know if that style would 
be applicable to Python modules.

Bottom line right now, though, is that this isn't really a useful issue for 
the bug tracker.  If you want to discuss strategies for making overall 
improvements in the documentation, that's something that should be done with 
the group of people who focus on documentation.  Their mailing list is 
d...@python.org if you want to join the team and advocate for a change (your 
suggestion has already been posted to that mailing list by the bug tracker, 
FYI).

--
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<http://bugs.python.org/issue22691>
___

--

___
Python tracker 
<http://bugs.python.org/issue22691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20825] containment test for "ip_network in ip_network"

[James]

James added the comment:

What is the status of these changes? Apparently they were slated for inclusion 
in 3.5 but it looks as though they haven't hit yet - is there a reason for 
this, or was it just forgotten?

--
nosy: +JamesGuthrie

___
Python tracker 
<http://bugs.python.org/issue20825>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25167] THE SCORCH TRIALS OF MAZE RUNNER DISFACTION LOGGIC

[james]

New submission from james:

http://www.thebigidea.co.nz/profile/james/65456
http://www.cyclefish.com/hebucoho/
https://soundation.com/user/MazeRunnerTheScorchTrials
https://issuu.com/mazerunnerthescorchtrials
http://poputka.ua/user-profile-39591.aspx
http://www.pikore.com/mazerunnerthescorch
https://instagram.com/mazerunnerthescorch/
http://iconosquare.com/mazerunnerthescorch
https://tofo.me/mazerunnerthescorch
http://buzzsta.com/mazerunnerthescorch
https://bitbucket.org/hebucoho/

--
components: asyncio
messages: 250965
nosy: gvanrossum, haypo, leo, yselivanov
priority: normal
severity: normal
status: open
title: THE SCORCH TRIALS OF MAZE RUNNER DISFACTION LOGGIC
versions: Python 3.4

___
Python tracker 
<http://bugs.python.org/issue25167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26752] Mock(2.0.0).assert_has_calls() raise AssertionError in two same calls

[James]

New submission from James:

>>> import mock
>>> print mock.__version__
2.0.0
>>> 

=
test.py
from mock import Mock,call

class BB(object):
def __init__(self):pass
def print_b(self):pass
def print_bb(self,tsk_id):pass


bMock = Mock(return_value=Mock(spec=BB))
bMock().print_bb(20)
bMock().assert_has_calls([call.print_bb(20)])


===
Traceback (most recent call last):
  File "test.py", line 11, in 
bMock().assert_has_calls([call.print_bb(20)])
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 969, in 
assert_has_calls
), cause)
  File "/usr/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Calls not found.
Expected: [call.print_bb(20)]
Actual: [call.print_bb(20)]

===
print expected in mock.py assert_has_calls()

result is:
[TypeError('too many positional arguments',)]

--
files: test.py
messages: 263375
nosy: jekin000, rbcollins
priority: normal
severity: normal
status: open
title: Mock(2.0.0).assert_has_calls() raise AssertionError in two same calls
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file42460/test.py

___
Python tracker 
<http://bugs.python.org/issue26752>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24035] When Caps Locked, + alpha-character still displayed as uppercase

[James]

New submission from James:

Referring to Python 2.7 running on Windows (7/8): 

At the interactive interpreter, if either 1) Caps are Locked OR 2)  is 
held while an alpha-character is selected, the character is output and 
displayed as uppercase, as one would expect. However, in Python 2.7, when Caps 
are Locked AND  is held while an alpha-character is selected, the output 
is still uppercase, when it should be lowercase.

This behavior seems to be limited to Python 2(.7), and only when Python is run 
interactively using the Windows console. When Python 2.7 is started through 
Cygwin using mintty on Windows 7/8, the behavior is not reproducible. The same 
goes for IDLE, as well as when running python interactively on Ubuntu. The 
behavior is reproducible using IPython when run in the standard Windows 
console, but is not reproducible when Python is run using the IPython QtConsole.

To summarize:

-   ---
AFFECTED|  |NOT AFFECTED |
--  ---
python 2 via cmd (Windows 7/8) python 3 via cmd(Windows 7/8)
ipython via cmd  (Windows 7/8) IDLE python 2 & 3   (Windows 7/8) 
   python 2 via mintty (Windows 7/8)
   python 2 via bash  (Ubuntu 14.04)
   ipython qtconsole   (Windows 7/8)

--
components: IO
messages: 241845
nosy: principia1687
priority: normal
severity: normal
status: open
title: When Caps Locked,  + alpha-character still displayed as uppercase
type: behavior
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue24035>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24035] When Caps Locked, + alpha-character still displayed as uppercase

[James]

James added the comment:

When I start the interpreter with the -S switch, the problem goes away. 
Thanks for looking into it, and I apologize for the false alarm!

--

___
Python tracker 
<http://bugs.python.org/issue24035>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24035] When Caps Locked, + alpha-character still displayed as uppercase

[James]

Changes by James :


--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 
<http://bugs.python.org/issue24035>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46116] _asyncio_backend.py datagram_received doesn't handle Future cancelled, throws Exception

[James Lawrie]

New submission from James Lawrie :

The datagram_received:

def datagram_received(self, data, addr):
if self.recvfrom:
self.recvfrom.set_result((data, addr))
self.recvfrom = None

Throws an exception if self.recvfrom is a Future Cancelled:

Exception in callback _SelectorDatagramTransport._read_ready()
handle: 
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 1021, in 
_read_ready
self._protocol.datagram_received(data, addr)
  File "/usr/local/lib/python3.8/dist-packages/dns/_asyncio_backend.py", line 
30, in datagram_received
self.recvfrom.set_result((data, addr))
asyncio.exceptions.InvalidStateError: invalid state

In my test code (attached), this wasn't caught in a try: catch:

--
components: asyncio
files: asynctest.py
messages: 408778
nosy: asvetlov, james2, yselivanov
priority: normal
severity: normal
status: open
title: _asyncio_backend.py datagram_received doesn't handle Future cancelled, 
throws Exception
versions: Python 3.8
Added file: https://bugs.python.org/file50500/asynctest.py

___
Python tracker 
<https://bugs.python.org/issue46116>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] print() docs do not indicate its return value

[James Gerity]

Change by James Gerity :


--
assignee: docs@python
components: Documentation
nosy: SnoopJeDi, docs@python
priority: normal
severity: normal
status: open
title: print() docs do not indicate its return value
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] print() docs do not indicate its return value

[James Gerity]

Change by James Gerity :


--
keywords: +patch
pull_requests: +28642
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30435

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] print() docs do not indicate its return value

[James Gerity]

James Gerity  added the comment:

I opened this ticket on behalf of a user who asked about print() specifically 
in #python on the Libera IRC network, who I assume does not find this obvious.

I don't think it would be tenable to add this note to every built-in, but 
that's not the intended scope of this issue. I do think it's worth mentioning 
for print(), though.

--

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] print() docs do not indicate its return value

[James Gerity]

James Gerity  added the comment:

The original question was closer to the related issue of "indicate return types 
for all built-ins," conversation log follows (UTC-5):

```
09:33:50 ringo__ | is there a stdlib api docs which actually has *full* 
functions signature?
   
09:34:27 ringo__ | for example, 
https://docs.python.org/3/library/functions.html, function  

   
 | abs(x), it returns what, int? I need to read the 
whole sentence to figure
   
 | out the return value of a function?  

   
09:34:48 ringo__ | (or argument for that matter)

   
09:35:51 bjs | ringo__: well like it says it doesn't just support 
int 
 
09:36:00 bjs | int, float, or any type that supports it 

   
09:37:01 bjs | in general you can find actual type annotations for 
the functions in the

 | typeshed 

   
 | 
https://github.com/python/typeshed/blob/master/stdlib/builtins.pyi  


09:37:32 bjs | I wonder if it would be useful to include the 
typeshed annotation in the  
  
 | docs, or whether it would be more confusing  

   
09:37:49 ringo__ | Thanks bjs ! I'll bookmark this typeshed 

   
09:38:13  SnoopJ | abs() will do whatever __abs__() on the type does, 
which can be different  
 
 | for any given type. You'd expect T -> T but it's not 
guaranteed. 
   
09:38:18 ringo__ | I used abs() as an example. In fact I was wondering 
what does print()   

 | return. I *guessed* it returns None, but the docs 
won't say   
  
09:39:05 ringo__ | I could do a try-it-yourself approach but I was 
puzzled why the docs

 | simply won't give you full fn signature, ie 
print(..) -> None   

09:39:17  SnoopJ | that one is just an omission :)
```

--

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46292] Add microseconds to logging.LogRecord

[James Casbon]

New submission from James Casbon :

LogRecord makes microseconds available via the msecs attribute.  This patch 
adds microseconds via usecs attribute.

Some regulators (eg MIFID II) require accuracy greater than 1ms in some 
industries.

This patch calls time_ns rather than time so that the usecs and msecs are 
calculated from int types and then gets the original ct attribute via division.

--
files: usecs.patch
keywords: patch
messages: 409963
nosy: jamescasbon
priority: normal
severity: normal
status: open
title: Add microseconds to logging.LogRecord
Added file: https://bugs.python.org/file50546/usecs.patch

___
Python tracker 
<https://bugs.python.org/issue46292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46292] Add microseconds to logging.LogRecord

[James Casbon]

Change by James Casbon :


--
components: +Library (Lib)
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue46292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] return value of builtins is not clearly indicated

[James Gerity]

Change by James Gerity :


--
title: print() docs do not indicate its return value -> return value of 
builtins is not clearly indicated

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] return value of builtins is not clearly indicated

[James Gerity]

James Gerity  added the comment:

> advertising that all functions default to returning None

This is already communicated in § 4.7 ("Defining Functions") of the official 
tutorial. 

I think it would be a good idea to revise that section so that this property of 
functions is a little more clear, but that isn't the scope of this ticket.

The title change reflects my intent to submit a PR that adds a hint to the 
builtins doc.

--

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46282] return value of builtins is not clearly indicated

[James Gerity]

James Gerity  added the comment:

My thought was to add something like this to the top of functions.rst:

```
Note that some of the functions listed here have the :ref:`default return value 
 of ``None``.
```

For reference, the builtins this applies to are:

* breakpoint()
* delattr()
* exec()
* help()
* print()
* setattr()

Which makes me wonder if the hint is even worth having, since it's so few of 
them.

Note that of these, exec() does what this ticket originally proposed for 
print() - i.e. it explicitly says that the function returns None.

--

___
Python tracker 
<https://bugs.python.org/issue46282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46555] Unicode-mangled names refer inconsistently to constants

[James Gerity]

Change by James Gerity :


--
nosy: +SnoopJeDi

___
Python tracker 
<https://bugs.python.org/issue46555>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46555] Unicode-mangled names refer inconsistently to constants

[James Gerity]

James Gerity  added the comment:

> Why was it decided to not raise a syntax error...

I'm not sure if such a decision was even ever made, the error happens before 
normalization is applied. I.e. the parser is doing two things here: (1) 
validating the syntax against the grammar and (2) building the AST. 
Normalization happens after (1), and `𝕋𝕣𝕦𝕖 = 0` is valid syntax because the 
grammar is NOT defined in terms of normalized identifiers, it's describing the 
valid (but confusing!) assignment that Carl described.

I agree that this doesn't seem like bug, but it IS my new favorite quirk of 
identifier normalization.

--

___
Python tracker 
<https://bugs.python.org/issue46555>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46690] create_autospec() doesn't respect configure_mock style kwargs

[James Marchant]

New submission from James Marchant :

When using `create_autospec()` to create a mock object, it doesn't respect 
values passed through in the style described for passing mock configurations in 
the Mock constructor 
(https://docs.python.org/3.8/library/unittest.mock.html#unittest.mock.Mock.configure_mock).
 Instead, they seem to get discarded somewhere here 
(https://github.com/python/cpython/blob/77bab59c8a1f04922bb975cc4f11e5323d1d379d/Lib/unittest/mock.py#L2693-L2741).

Here's a simple test case:
```
from unittest.mock import create_autospec

class Test:
def test_method(self):
pass

autospec_mock = create_autospec(Test, instance=True, 
**{"test_method.side_effect": ValueError})

# Should throw a ValueError exception
autospec_mock.test_method()

# Assign manually
autospec_mock.test_method.side_effect = ValueError
# Throws as expected
autospec_mock.test_method()
```

--
components: Tests
messages: 412898
nosy: marchant.jm
priority: normal
severity: normal
status: open
title: create_autospec() doesn't respect configure_mock style kwargs
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue46690>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46726] Thread spuriously marked dead after interrupting a join call

[James Gerity]

Change by James Gerity :


--
nosy: +SnoopJeDi

___
Python tracker 
<https://bugs.python.org/issue46726>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1179] [CVE-2007-4965] Integer overflow in imageop module

[James Antill]

James Antill added the comment:

So I think this is all the places integer overflow checking is needed
in imageop.c and rbgimgmodule.c.
 There might be checks here which can't be exploited anyway, and I
haven't checked any other files yet.

 Feel free to comment.

 Ps. This is against the 2.5 in Fedora-7, but it should apply to upstream.

--
nosy: +nevyn

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1179>
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1179] [CVE-2007-4965] Integer overflow in imageop module

[James Antill]

Changes by James Antill:


__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1179>
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1179] [CVE-2007-4965] Integer overflow in imageop module

[James Antill]

James Antill added the comment:

And now the obvious typo fix, *sigh*.

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1179>
__diff -rup Python-2.5-orig/Modules/imageop.c Python-2.5/Modules/imageop.c
--- Python-2.5-orig/Modules/imageop.c	2006-01-19 01:09:39.0 -0500
+++ Python-2.5/Modules/imageop.c	2007-09-19 18:01:05.0 -0400
@@ -78,7 +78,7 @@ imageop_crop(PyObject *self, PyObject *a
 	char *cp, *ncp;
 	short *nsp;
 	Py_Int32 *nlp;
-	int len, size, x, y, newx1, newx2, newy1, newy2;
+	int len, size, x, y, newx1, newx2, newy1, newy2, nlen;
 	int ix, iy, xstep, ystep;
 	PyObject *rv;
 
@@ -90,13 +90,19 @@ imageop_crop(PyObject *self, PyObject *a
 		PyErr_SetString(ImageopError, "Size should be 1, 2 or 4");
 		return 0;
 	}
-	if ( len != size*x*y ) {
+	/* ( len != size*x*y ) */
+	if ( size != ((len / x) / y) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
 	xstep = (newx1 < newx2)? 1 : -1;
 	ystep = (newy1 < newy2)? 1 : -1;
 
+nlen = (abs(newx2-newx1)+1)*(abs(newy2-newy1)+1)*size;
+if ( size != ((nlen / (abs(newx2-newx1)+1)) / (abs(newy2-newy1)+1)) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
 	rv = PyString_FromStringAndSize(NULL,
 			 (abs(newx2-newx1)+1)*(abs(newy2-newy1)+1)*size);
 	if ( rv == 0 )
@@ -132,7 +138,7 @@ imageop_scale(PyObject *self, PyObject *
 	char *cp, *ncp;
 	short *nsp;
 	Py_Int32 *nlp;
-	int len, size, x, y, newx, newy;
+	int len, size, x, y, newx, newy, nlen;
 	int ix, iy;
 	int oix, oiy;
 	PyObject *rv;
@@ -145,12 +151,18 @@ imageop_scale(PyObject *self, PyObject *
 		PyErr_SetString(ImageopError, "Size should be 1, 2 or 4");
 		return 0;
 	}
-	if ( len != size*x*y ) {
+	/* ( len != size*x*y ) */
+	if ( size != ((len / x) / y) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
+nlen = newx*newy*size;
+	if ( size != ((nlen / newx) / newy) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
 
-	rv = PyString_FromStringAndSize(NULL, newx*newy*size);
+	rv = PyString_FromStringAndSize(NULL, nlen);
 	if ( rv == 0 )
 		return 0;
 	ncp = (char *)PyString_AsString(rv);
@@ -190,7 +202,8 @@ imageop_tovideo(PyObject *self, PyObject
 		PyErr_SetString(ImageopError, "Size should be 1 or 4");
 		return 0;
 	}
-	if ( maxx*maxy*width != len ) {
+	/* if ( maxx*maxy*width != len ) */
+	if ( maxx != ((len / maxy) / width) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -240,7 +253,8 @@ imageop_grey2mono(PyObject *self, PyObje
 	if ( !PyArg_ParseTuple(args, "s#iii", &cp, &len, &x, &y, &tres) )
 		return 0;
 
-	if ( x*y != len ) {
+	/* ( x*y != len ) */
+	if ( x != len / y ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -281,7 +295,8 @@ imageop_grey2grey4(PyObject *self, PyObj
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	/* ( x*y != len ) */
+	if ( x != len / y ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -320,7 +335,8 @@ imageop_grey2grey2(PyObject *self, PyObj
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	/* ( x*y != len ) */
+	if ( x != len / y ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -358,7 +374,8 @@ imageop_dither2mono(PyObject *self, PyOb
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	/* ( x*y != len ) */
+	if ( x != len / y ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -404,7 +421,8 @@ imageop_dither2grey2(PyObject *self, PyO
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	/* ( x*y != len ) */
+	if ( x != len / y ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -443,7 +461,11 @@ imageop_mono2grey(PyObject *self, PyObje
 	if ( !PyArg_ParseTuple(args, "s#", &cp, &len, &x, &y, &v0, &v1) )
 		return 0;
 
-	nlen = x*y;
+nlen = x*y;
+	if ( x != (nlen / y) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
 	if ( (nlen+7)/8 != len ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
@@ -481,6 +503,10 @@ imageop_grey22grey(PyObject *self, PyObj
 		return 0;
 
 	nlen = x*y;
+	if ( x != (nlen / y) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length&quo

[issue1179] [CVE-2007-4965] Integer overflow in imageop module

[James Antill]

James Antill added the comment:

Guido: It's true that that len can be slightly bigger than x*y, the big
thing is that it can't be smaller so we can malloc(len) and use upto x*y
(which was my main focus).
 I first looked at any of this code today, but I didn't see any reason
that having len be slightly larger would be a problem ... and in pretty
much all cases it'll be len == x*y.

 However we could have both cases covered by doing:

 if ( (len != x*y) || (x != (len / y)) )

...but esp. at that point it seems like we'd want some interface so that
we could just do something like:

 if ( check_mutliplies2(len, x, y) )

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1179>
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1179] [CVE-2007-4965] Integer overflow in imageop module

[James Antill]

James Antill added the comment:

Not sure who Neal is, and this probably isn't a final upstream fix ...
but it's what I've applied to Fedora's python. It's basically the same
patch as before, but it keeps the original * tests instead of just
replacing them with / tests. So given:

 if x * y != len

...the first patch did:

 if len / x != y

...and this patch does:

 if x * y != len ||
len / x != y

Added file: 
http://bugs.python.org/file8592/python-2.5.CVE-2007-4965-int-overflow.patch

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1179>
__diff -ru Python-2.5-orig/Modules/imageop.c Python-2.5/Modules/imageop.c
--- Python-2.5-orig/Modules/imageop.c	2006-01-19 01:09:39.0 -0500
+++ Python-2.5/Modules/imageop.c	2007-10-19 01:11:33.0 -0400
@@ -78,7 +78,7 @@
 	char *cp, *ncp;
 	short *nsp;
 	Py_Int32 *nlp;
-	int len, size, x, y, newx1, newx2, newy1, newy2;
+	int len, size, x, y, newx1, newx2, newy1, newy2, nlen;
 	int ix, iy, xstep, ystep;
 	PyObject *rv;
 
@@ -90,13 +90,19 @@
 		PyErr_SetString(ImageopError, "Size should be 1, 2 or 4");
 		return 0;
 	}
-	if ( len != size*x*y ) {
+	if (( len != size*x*y ) ||
+( size != ((len / x) / y) )) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
 	xstep = (newx1 < newx2)? 1 : -1;
 	ystep = (newy1 < newy2)? 1 : -1;
 
+nlen = (abs(newx2-newx1)+1)*(abs(newy2-newy1)+1)*size;
+if ( size != ((nlen / (abs(newx2-newx1)+1)) / (abs(newy2-newy1)+1)) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
 	rv = PyString_FromStringAndSize(NULL,
 			 (abs(newx2-newx1)+1)*(abs(newy2-newy1)+1)*size);
 	if ( rv == 0 )
@@ -132,7 +138,7 @@
 	char *cp, *ncp;
 	short *nsp;
 	Py_Int32 *nlp;
-	int len, size, x, y, newx, newy;
+	int len, size, x, y, newx, newy, nlen;
 	int ix, iy;
 	int oix, oiy;
 	PyObject *rv;
@@ -145,12 +151,18 @@
 		PyErr_SetString(ImageopError, "Size should be 1, 2 or 4");
 		return 0;
 	}
-	if ( len != size*x*y ) {
+	if ( ( len != size*x*y ) ||
+ ( size != ((len / x) / y) ) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
+nlen = newx*newy*size;
+	if ( size != ((nlen / newx) / newy) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
 
-	rv = PyString_FromStringAndSize(NULL, newx*newy*size);
+	rv = PyString_FromStringAndSize(NULL, nlen);
 	if ( rv == 0 )
 		return 0;
 	ncp = (char *)PyString_AsString(rv);
@@ -190,7 +202,8 @@
 		PyErr_SetString(ImageopError, "Size should be 1 or 4");
 		return 0;
 	}
-	if ( maxx*maxy*width != len ) {
+	if ( ( maxx*maxy*width != len ) ||
+ ( maxx != ((len / maxy) / width) ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -240,7 +253,8 @@
 	if ( !PyArg_ParseTuple(args, "s#iii", &cp, &len, &x, &y, &tres) )
 		return 0;
 
-	if ( x*y != len ) {
+	if ( ( x*y != len ) ||
+ ( x != len / y ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -281,7 +295,8 @@
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	if ( ( x*y != len ) ||
+ ( x != len / y ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -320,7 +335,8 @@
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	if ( ( x*y != len ) ||
+ ( x != len / y ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -358,7 +374,8 @@
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	if ( ( x*y != len ) ||
+ ( x != len / y ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -404,7 +421,8 @@
 	if ( !PyArg_ParseTuple(args, "s#ii", &cp, &len, &x, &y) )
 		return 0;
 
-	if ( x*y != len ) {
+	if ( ( x*y != len ) ||
+ ( x != len / y ) ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
 	}
@@ -443,7 +461,11 @@
 	if ( !PyArg_ParseTuple(args, "s#", &cp, &len, &x, &y, &v0, &v1) )
 		return 0;
 
-	nlen = x*y;
+nlen = x*y;
+	if ( x != (nlen / y) ) {
+		PyErr_SetString(ImageopError, "String has incorrect length");
+		return 0;
+	}
 	if ( (nlen+7)/8 != len ) {
 		PyErr_SetString(ImageopError, "String has incorrect length");
 		return 0;
@@ -481,6 +503,10 @@
 		return 0;
 
 	nlen = x*y;
+	if ( x != (nlen / y) ) {
+		PyErr_Se

[issue1681674] subprocess.Popen fails with socket._fileobject on Windows

[James Burgess]

James Burgess  added the comment:

"Can't Fix" that is not true. 

I've just fixed this in 2.7 with a trivial change to subprocesss.py, I think 
it'd work in over versions too.

Note that type shenanigans are already in play in _get_handles, it's looking at 
the types of the parameters being passed in to decide how to get a hold of the 
"handle". The socket module makes a duck type of the file object. The fileno() 
method of the socket object returns a handle not a CRT file descriptor. This is 
exactly the kind of handle that _get_handles() is looking for.

So all that is needed is one more "if" to the sequence of how to get the 
"handle" which for a socket object would be just stdin.fileno() etc.

I've just tested this in a fairly complicated remote job queuing software (a 
commercial product) that has the ability to connect the spooler (on one 
machine) with an arbitrary server machine (linux, osx and now windows) via a 
socket. The job is launched with subprocess.Popen and sockets are wired into 
the stdin,stdout and stderr. Works beautifully now.

I've attached a patch file made with:
$ diff -c subprocess.py.ORIG subprocess.py  > subprocess.py.patch

Apply with:
$ cd  Python-2.7.1/Lib ; patch -p0 < c:/temp/subprocess.py.patch

Cheers,
- James

--
keywords: +patch
nosy: +James.Burgess
Added file: http://bugs.python.org/file23545/subprocess.py.patch

___
Python tracker 
<http://bugs.python.org/issue1681674>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5364] documentation in epub format

[James Polley]

James Polley  added the comment:

So http://bitbucket.org/birkenfeld/sphinx/issue/140/ has now been closed; 
sphinx happily builds epub.

However, the python docs are still not available for download in epub format 
from http://docs.python.org/download.html, which was the original request in 
this issue.

--
nosy: +James.Polley

___
Python tracker 
<http://bugs.python.org/issue5364>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5364] documentation in epub format

[James Polley]

James Polley  added the comment:

It looks like the first release that had epub support was 1.0; docs.python.org 
is still using 0.6.7, according to the footer on the bottom of the page.

I suspect that this is (A) pending the upgrade to 1.0.0, which is (B) more 
difficult than it sounds like, or at least, needs someone to make a concerted 
effort, and (C) should automatically just happen (or at least, be fairly 
trivial) once the move to the new version happens.

--

___
Python tracker 
<http://bugs.python.org/issue5364>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13602] format string '%b' doesn't work as expected

[James Classen]

New submission from James Classen :

I notice that, in versions 2.7 and 3.2 on Windows XP (haven't tested any other 
versions or platforms), the following statements in the interpreter work as 
documented:

'%x' % 17
'%o' % 17

and output '11' and '21' respectively, as I expect. However,

'%b' % 17

throws "ValueError: unsupported format character 'b' (0x62) at index 1" instead 
of returning '10001'.

--
components: None
messages: 149471
nosy: James.Classen
priority: normal
severity: normal
status: open
title: format string '%b' doesn't work as expected
type: behavior
versions: Python 2.7, Python 3.2

___
Python tracker 
<http://bugs.python.org/issue13602>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13602] format string '%b' doesn't work as expected

[James Classen]

James Classen  added the comment:

I didn't see section 4.6.2 of the library for 3.2 documentation, only section 
5.6.2 of the 2.7 docs. So this is an invalid issue.

--
resolution:  -> invalid
status: open -> closed

___
Python tracker 
<http://bugs.python.org/issue13602>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)

[James B]

James B  added the comment:

I have encountered this issue(python 2.7) with respect to positional arguments 
that begin with a dash (linux/ bash).

In the following example, the parser requires three positional arguments. I 
attempted to encase the arguments in single-quotes as that is expected in 
general to result in strings to be correctly handled (these args are API keys, 
so they could contain shell-unfriendly chars like - and &).

./tool.py  arg1 'arg2' '-arg3&otherstuff' 

You'll note there are no optional arguments in this example, it just boils down 
to a positional argument being broken up on parse.

Needless to say it was quite confusing to see the script complain after passing 
in what would typically be perfectly valid strings in most other apps / scripts.

Is it possible to get argparse to correctly notice and handle shell-appropriate 
single-quoting methods(dont break down a string that has been implied as a 
complete token via ' ')

As it stands, it appears I have two workaround options: 1) adopt the ./tool.py 
--  convention mentioned in this thread, or 2) escape leading 
dashes in positional argument strings to avoid this issue.

--
nosy: +skilletaudio

___
Python tracker 
<http://bugs.python.org/issue9334>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13678] way to prevent accidental variable overriding

[James Hutchison]

New submission from James Hutchison :

In python is currently there a way to elegantly throw an error if a variable is 
already in the current scope?

For example:

def longfunc(self, filename):
FILE = open(filename);
header = FILE.readline();
... bunch of code ...
childfiles = self.children;
for child in childfiles:
 FILE = open(child);
 header = FILE.readline();
 ... do something with header ...
for line in FILE:
   ... etc ...

In this case, I'm accidentally overriding the old values of FILE and header, 
resulting in a bug. But I'm not going to catch this. I've had a couple of real 
life bugs due to this that were a lot more subtle and lived for months without 
anyone noticing the output data was slightly wrong.

This situation could be prevented if there was a way to say something along the 
lines of "new FILE = open(child)" or "new header = FILE.readline()" and have 
python throw an error to let me know that it already exists. This would also 
make code clearer because it allows the intended scope of a variable to become 
more apparent. Since "new var = something" is a syntax error, adding this 
functionality wouldn't break old code, as long as python would allow for 'new' 
(or whatever the keyword would end up being) to also be a variable name (like 
"new new = 1" or "new = 1")

--
components: Interpreter Core
messages: 150344
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: way to prevent accidental variable overriding
type: enhancement
versions: Python 3.2, Python 3.3, Python 3.4

___
Python tracker 
<http://bugs.python.org/issue13678>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13678] way to prevent accidental variable overriding

[James Hutchison]

James Hutchison  added the comment:

For starters, this would be most efficient implementation:

def unique(varname, value, scope):
assert(not varname in scope);
scope[varname] = value;

Usage:
unique('b', 1, locals());
print(b);

But you can't put that in a loop else it will false trigger. Ideally this 
wouldn't false trigger. This could be done by having python internally 
associate a line number with each explicit variable declaration.

Anyways, an external python function would be too slow for my use case. 
Likewise, since it would be something you could use a lot, it should be 
implemented in the underlying C code to give it minimal overhead.

Keeping functions small is very impractical at times. I shouldn't create 50 
randomly named one use functions in my class as a safeguard against accidental 
overwriting when I have a legitimately complicated piece of code that can't be 
dissected without becoming unreadable. In many cases I might need 8 or 9 locals 
at a time in a single line in each loop section.

I don't see how this is the area of pylint/pyflakes at all. The idea is to make 
it so the human doesn't have to carefully inspect their code in order to decide 
if they made a mistake or not. Inspecting a long list of warnings is no better, 
and arguably I could pull up a bunch of python language design decisions and 
ask why they were made if pylint/pyflakes exists.

If such a change would have be implemented after much consideration and 
discussion, I don't see how closing my post helps accomplish that.

--

___
Python tracker 
<http://bugs.python.org/issue13678>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7501] python -m unittest path_to_suite_function errors

[James Westby]

James Westby  added the comment:

Hi,

I think this was misdiagnosed:

from unittest.py in 2.6, loadTestFromName:

elif hasattr(obj, '__call__'):
test = obj()
if isinstance(test, TestSuite):
return test
elif isinstance(test, TestCase):
return TestSuite([test])
else:
raise TypeError("calling %s returned %s, not a test" %
(obj, test))

so it supports callables, such as the test_suite function that
Rob is passing. Therefore I don't think this is a feature request,
and "use load_tests" isn't an appropriate resolution.

I haven't checked what 2.7 and later do, but going on the above
my diagnosis of this is the following.

  1. test_suite is correctly identified as a callable
  2. It is called an returns a unittest.TestSuite
  3. It is not matched as being a a TestSuite or a TestCase, and so the error 
is raised.

The reason it is not matched is that when run as -m unittest, the unittest 
module is __main__, and so the TestSuite in the isinstance check is a 
unittest.TestSuite against a __main__.TestSuite, which won't match.

Therefore I think this is a legitimate bug, in at least 2.6.

Thanks,

James

--
nosy: +james-w

___
Python tracker 
<http://bugs.python.org/issue7501>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9801] Can not use append/extend to lists in a multiprocessing manager dict

[James Hutchison]

New submission from James Hutchison :

tested python 3.1.2

Man = multiprocessing.Manager();
d = man.dict();
d['l'] = list();
d['l'].append("hey");
print(d['l']);
>>> []

using debugger reveals a KeyError. Extend also does not work. Only thing that 
works is += which means you can't insert actual tuples or lists into the list. 
This was all done on a single process

--
components: Library (Lib)
messages: 115891
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: Can not use append/extend to lists in a multiprocessing manager dict
type: behavior

___
Python tracker 
<http://bugs.python.org/issue9801>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9803] IDLE closes with save while breakpoint open

[James Hutchison]

New submission from James Hutchison :

I have multiple versions of python - 2.6.1 and 3.1.2. 2.6.1 is the primary 
install (i.e., right click on a file and "edit with IDLE" brings up 2.6), and 
was installed first. This issue occurs on 3.1.2, Windows XP 32-bit

If I highlight a breakpoint, run with the debugger, make a change, then save, 
IDLE will close all windows after saving without an error message. If I clear 
the breakpoint and then save, IDLE will not close. Reopening the file reveals 
that the save was successful. I do not need to actually be stopped at a 
highlighted breakpoint for this to occur (i.e. this will occur if I save at the 
beginning before the script has ran). If I run with 2.6.1 I do not see this 
issue. The save difference could be something simple, such as adding a space 
somewhere random.

--
components: IDLE
messages: 115894
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: IDLE closes with save while breakpoint open
type: crash
versions: Python 3.1

___
Python tracker 
<http://bugs.python.org/issue9803>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9847] Binary strings never compare equal to raw/normal strings

[James Hutchison]

New submission from James Hutchison :

Tested on Python 3.1.2 Windows XP 32-bit

Binary strings (such as what is returned by filereader.readline()) are never 
equal to raw or normal strings, even when both strings are empty

if(b"" == ""):
print("Strings are equal");
else:
if(b"" == r""):
print("raw and binary equal, normal isn't");
else:
print("they aren't equal");

output: they aren't equal

--
components: Interpreter Core
messages: 116331
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: Binary strings never compare equal to raw/normal strings
type: behavior
versions: Python 3.1

___
Python tracker 
<http://bugs.python.org/issue9847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9801] Can not use append/extend to lists in a multiprocessing manager dict

[James Hutchison]

James Hutchison  added the comment:

Is there a way to get this so it behaves more intuitively? You'd think adding a 
managed list to a managed dictionary (or another managed list) or making a deep 
copy would work but it still doesn't. When you get an item from a managed data 
structure, it seems to be returning a data-only copy of the object instead of a 
handle to the manager of the object. The fact that += (extend) works but 
.extend() doesn't work also seems to raise a flag for me (although I do 
understand why this is). I don't think it should behave this way.

i.e.:
currently:
d['l'] -> return copy.deepcopy(d['l'])

should be:
d['l'] -> return managerObject(d['l'])
where managerObject is a managed object that runs on the same process as the 
manager it came from

Problem: Currently there is no easy way to do random access without copying out 
and copying back in. I'd think that would be a real efficiency problem.

--

___
Python tracker 
<http://bugs.python.org/issue9801>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10150] Local references not released after exception

[James Bowman]

New submission from James Bowman :

import sys

def foo():
x = [o] * 100
raise ArithmeticError

o = "something"
print sys.getrefcount(o)
try:
foo()
except ArithmeticError:
pass
print sys.getrefcount(o)

---

Gives:

4
104


Looking at the CPython source, FrameObject's deallocator does actually 
decrement refcounts of its locals and arguments.  Guessing that the FrameObject 
is not being deallocated.

--
components: Interpreter Core
messages: 119187
nosy: James.Bowman
priority: normal
severity: normal
status: open
title: Local references not released after exception
type: resource usage
versions: Python 2.6

___
Python tracker 
<http://bugs.python.org/issue10150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10332] Multiprocessing maxtasksperchild results in hang

[James Hutchison]

New submission from James Hutchison :

v.3.2a3

If the maxtasksperchild argument is used, the program will just hang after 
whatever that value is rather than working as expected. Tested in Windows XP 
32-bit

test code:

import multiprocessing

def f(x):
return 0;

if __name__ == '__main__':
pool = multiprocessing.Pool(processes=2,maxtasksperchild=1);
results = list();
for i in range(10):
results.append(pool.apply_async(f, (i)));
pool.close();
pool.join();
for r in results:
print(r);
print("Done");

--
components: Library (Lib)
messages: 120547
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: Multiprocessing maxtasksperchild results in hang
versions: Python 3.2

___
Python tracker 
<http://bugs.python.org/issue10332>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10376] ZipFile unzip is unbuffered

[James Hutchison]

New submission from James Hutchison :

The Unzip module is always unbuffered (tested v.3.1.2 Windows XP, 32-bit). This 
means that if one has to do many small reads it is a lot slower than reading a 
chunk of data to a buffer and then reading from that buffer. It seems logical 
that the unzip module should default to buffered reading and/or have a buffered 
argument. Likewise, the documentation should clarify that there is no buffering 
involved when doing a read, which runs contrary to the default behavior of a 
normal read.

start Zipfile read
done
27432 reads done
took 0.859 seconds
start buffered Zipfile read
done
27432 reads done
took 0.072 seconds
start normal read (default buffer)
done
27432 reads done
took 0.139 seconds
start buffered normal read
done
27432
took 0.137 seconds

--
assignee: d...@python
components: Documentation, IO, Library (Lib)
messages: 120871
nosy: Jimbofbx, d...@python
priority: normal
severity: normal
status: open
title: ZipFile unzip is unbuffered
type: performance
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1

___
Python tracker 
<http://bugs.python.org/issue10376>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10376] ZipFile unzip is unbuffered

[James Hutchison]

James Hutchison  added the comment:

I should clarify that this is the zipfile constructor I am using:

zipfile.ZipFile(filename, mode='r', allowZip64=True);

--

___
Python tracker 
<http://bugs.python.org/issue10376>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11652] urlib{, 2} returns a pair of integers as the content-length value

[James Whisnant]

James Whisnant  added the comment:

Varnish on the sourceforge server has been upgraded and/or reconfigured 
(yesterday) to fix the issue that was happening with this file (and others).

Just an FYI that you will no longer be able to re-create the triggering error.

'content-length': '289519',
'via': '1.1 varnish'

--
nosy: +jwhisnant

___
Python tracker 
<http://bugs.python.org/issue11652>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11990] redirected output - stdout writes newline as \n in windows

[James Hutchison]

New submission from James Hutchison :

In windows, 64-bit, python *mostly* writes only a \n to stdout even though it's 
mode is 'w'. However it sometimes writes a \r\n on certain print statements and 
erratically when I have multiple processes writing to stdout.

Output looks fine in console, in IDLE, and using v.3.1

Example with multiple processes writing to stdout out using the same code: 
print("TESTCODE"); (note that in windows, the naked \n is ignored):

TESTCODETESTCODE
TESTCODE
TESTCODE
TESTCODETESTCODETESTCODE
TESTCODE

Windows program that calls python and receives its piped output is a C# .NET 
program.

--
components: IO, Windows
messages: 135076
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: redirected output - stdout writes newline as \n in windows
type: behavior
versions: Python 3.2

___
Python tracker 
<http://bugs.python.org/issue11990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11990] redirected output - stdout writes newline as \n in windows

[James Hutchison]

James Hutchison  added the comment:

Sorry there isn't more info but I'm really busy right now

In fact a workaround would be appreciated if known.

--

___
Python tracker 
<http://bugs.python.org/issue11990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11990] redirected output - stdout writes newline as \n in windows

[James Hutchison]

James Hutchison  added the comment:

Nevermind, I have a workaround that didn't require rewriting all the print 
statements but its in the C# code not the python code

--

___
Python tracker 
<http://bugs.python.org/issue11990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11990] redirected output - stdout writes newline as \n in windows

[James Hutchison]

James Hutchison  added the comment:

Yes and no, I can give you a single process single child example that just 
shows that python 3.2 uses binary output while python 3.1 used system default 
when piping, but trying to reproduce the multiprocessing output inconsistencies 
would be... difficult. Unfortunately the software I used to spot the \n, \r\n 
inconsistency with is proprietary. After creating a sample case in only python 
I couldn't reproduce the inconsistent \r\n issue in python 3.2 so I cannot say 
for certain that it wasn't caused by my C# program. I wouldn't worry about the 
inconsistent endlines for now.

Note that I don't see in the "what's new" documentation it mentioning that 3.2 
changed the behavior of piped output. Kind of a big deal.

Sample code to compare 3.1 and 3.2 piped output:

import sys;
import os;
import subprocess;
from time import sleep;

python31loc = r"C:\python31\python.exe";
python32loc = r"C:\python32\python.exe";

myname = "IOPipetest.py";

def main(args):
if(len(args) == 1):
# main code
print("Testing python 3.1 endlines.", end='\r\n');
output = subprocess.check_output("%s %s -output" % (python31loc, 
myname));
print(output);
print("Testing python 3.2 endlines.", end='\r\n');
output = subprocess.check_output("%s %s -output" % (python32loc, 
myname));
print(output);
sleep(7);
else:
for i in range(4):
print("TESTING DEFAULT"); # endline is supposed to be automatic
print("TESTING SLASH-EN\n", end='');
print("TESTING WINDOW-RETURN\r\n", end='');


if __name__ == "__main__":
main(sys.argv);

--

sample output:

Testing python 3.1 endlines.

b'TESTING DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING 
DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING 
DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\nTESTING 
DEFAULT\r\nTESTING SLASH-EN\r\nTESTING WINDOW-RETURN\r\r\n'
Testing python 3.2 endlines.

b'TESTING DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING 
DEFAULT\nTESTING SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING DEFAULT\nTESTING 
SLASH-EN\nTESTING WINDOW-RETURN\r\nTESTING DEFAULT\nTESTING SLASH-EN\nTESTING 
WINDOW-RETURN\r\n'

--

___
Python tracker 
<http://bugs.python.org/issue11990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12020] Attribute error with flush on stdout,stderr

[James Hutchison]

New submission from James Hutchison :

When upgrading from Python 3.1 to Python 3.2 I noticed that when my program 
closed it printed out a non-consequential AttributeError Exception. My program 
had a custom class that replaced stdout and stderr for use in a piped program 
(it flushed the buffer after every print statement)

I was able to reduce my code down to this simple test case that will reproduce 
the issue. Note that this doesn't show up in idle.

code:
import sys
from time import sleep
import subprocess

python31loc = r"C:\python31\python.exe";
python32loc = r"C:\python32\python.exe";
myname = "attributeError.py";

class FlushFile(object):
#"""Write-only flushing wrapper for file-type objects."""
def __init__(self, f):
self.f = f
try:
self.encoding = f.encoding;
except:
pass;
def write(self, x):
self.f.write(x)
self.f.flush()

# sets stdout and stderr to autoflush
def setAutoFlush():
if sys.__stdout__ != None: # will be None in IDLE
sys.stdout = FlushFile(sys.__stdout__);
sys.stderr = FlushFile(sys.__stderr__);

if __name__ == "__main__":
setAutoFlush();
if(len(sys.argv) == 1):
print("Testing python 3.1");
output = subprocess.check_output("%s %s -output" % (python31loc, 
myname));
print("Should see no error");
print("Testing python 3.2");
output = subprocess.check_output("%s %s -output" % (python32loc, 
myname));
print("Should see no error");
sleep(16);


Output:
Testing python 3.1
Should see no error
Testing python 3.2
Exception AttributeError: 'flush' in <__main__.FlushFile object at 0x00C347F0> i
gnored
Should see no error

--
components: IO, Windows
messages: 135347
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: Attribute error with flush on stdout,stderr
type: behavior
versions: Python 3.2

___
Python tracker 
<http://bugs.python.org/issue12020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12020] Attribute error with flush on stdout,stderr

[James Hutchison]

James Hutchison  added the comment:

You are right, when I add:

def flush(self):
pass;

the error goes away.

When I have this:

def flush():
pass;

I get:

Exception TypeError: 'flush() takes no arguments (1 given)' in 
<__main__.FlushFile object at 0x00C2AB70> ignored

This leads me to believe that sys.stdout.flush() is being called on program 
close

So this would be the correct implementation of my flushfile override:

class FlushFile(object):
#"""Write-only flushing wrapper for file-type objects."""
def __init__(self, f):
self.f = f;
self.flush = f.flush;
try:
self.encoding = f.encoding;
except:
pass;
def write(self, x):
self.f.write(x)
self.f.flush()

--

___
Python tracker 
<http://bugs.python.org/issue12020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11623] Distutils is reporting OSX 10.6 w/ XCode 4 as "universal"

[James Tatum]

Changes by James Tatum :


--
nosy: +James.Tatum

___
Python tracker 
<http://bugs.python.org/issue11623>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11990] redirected output - stdout writes newline as \n in windows

[James Hutchison]

James Hutchison  added the comment:

I would like to add in windows, "input" now adds a \r at the end which wasn't 
in 3.1. It doesn't do it in idle. This is using just the regular console window 
that opens up when you double click. I'm guessing this is related to the issue 
I saw earlier:

code:
from time import sleep

item = input("Input something\n");
print("%s %i" % (item, len(item)));
sleep(15);

in Idle:
Input something
something
something 9

in console window:
Input something
something
 10ething

ouch

--

___
Python tracker 
<http://bugs.python.org/issue11990>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1731717] race condition in subprocess module

[James Lamanna]

James Lamanna  added the comment:

stubbing out subprocess._cleanup does not work around the problem from this 
example on 2.6.5:

import subprocess, signal
subprocess._cleanup = lambda: None

signal.signal(signal.SIGCLD, signal.SIG_IGN)
subprocess.Popen(['echo','foo']).wait()

ja...@hyla:~$ python tt.py
foo
Traceback (most recent call last):
  File "tt.py", line 5, in 
subprocess.Popen(['echo','foo']).wait()
  File "/usr/lib/python2.6/subprocess.py", line 1170, in wait
pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.6/subprocess.py", line 465, in _eintr_retry_call
return func(*args)
OSError: [Errno 10] No child processes

This bug still prevents subprocess from being used inside of a daemon where 
SIGCLD is being caught to reap zombie processes.

--
nosy: +jlamanna

___
Python tracker 
<http://bugs.python.org/issue1731717>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

I can work on this task.

--
nosy: +jjt009

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

i'm looking at the source and there doesn't appear to be a function
uname within os.py. are we just considering the uname function in
platform.py?

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

much handling code already seems to exist under the line
except AttributeError:
in platform.py (function uname(), lines 1101-1161 platform.py)
i'm not too familiar with the Python codebase (i just began developing
with Python a few days back)

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

Alright, that makes things much clearer. 
I'm looking at this code snippet in platform.py:
if system == 'unknown':
system = ''
if node == 'unknown':
node = ''
if release == 'unknown':
release = ''
if version == 'unknown':
version = ''
if machine == 'unknown':
machine = ''
if processor == 'unknown':
processor = ''

So essentially what you want me to do is add code that tries to replace
the ''s with meaningful information. From what I understand, os.uname()
is only defined in posix-based systems (it is imported from the module
posix, after all). That means that 'unknown' is returned because the
uname command is unable to retrieve the necessary information. Are there
any alternatives to uname that could provide me with system info?

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

Ah, ok, the code under except AttributeError: gives me some good ideas.
Should I use the methods utilized there to extract information from the
system?

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

Here is the patch (apply to platform.py)

--
keywords: +patch
Added file: http://bugs.python.org/file10594/platform.patch

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2912] let platform.uname try harder

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

Your patch works perfectly on windows. Thanks for your help.

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue2912>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3095] multiprocessing initializes flags dict unsafely

[James Thomas]

James Thomas <[EMAIL PROTECTED]> added the comment:

I believe this patch solves the problem. 
I added the line
Py_DECREF(temp)
after the code block shown above. 
I also changed the line
if (PyDict_SetItemString(temp, #name, Py_BuildValue("i", name)) < 0) return
to
if (PyDict_SetItemString(PyObject_GetAttrString(module, "flags"), #name,
Py_BuildValue("i", name)) < 0) return

--
keywords: +patch
nosy: +jjt009
Added file: http://bugs.python.org/file10609/multiprocessing.diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3095>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1503502] Pdb doesn't call flush on its stdout file descriptor

[James Dominy]

James Dominy <[EMAIL PROTECTED]> added the comment:

I've been working on a patch that allows pdb when run as a script to
split it's output such that the program being debugged uses a specified
tty for stdin/stdout, and leave the pdb.py IO on the original
stdin/stdout. I think perhaps these efforts should be merged. Certainly
your suggested patch would make my work much easier.

--
nosy: +sirlark

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1503502>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3066] FD leak in urllib2

[James Antill]

James Antill <[EMAIL PROTECTED]> added the comment:

So if I add a:

class _WrapForRecv:
def __init__(self, obj):
self.__obj = obj

def __getattr__(self, name):
if name == "recv": name = "read"
return getattr(self.__obj, name)

...and then change:

r.recv = r.read

...into:

r = _WrapForRecv(r)

...it stops the leak, and afaics nothing bad happens.

--
nosy: +nevyn

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3066>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3978] ZipFileExt.read() can be incredibly slow

[James Athey]

New submission from James Athey <[EMAIL PROTECTED]>:

I've created a patch that improves the decompression performance of
zipfile.py by up to two orders of magnitude.

In ZipFileExt.read(), decompressed bytes waiting to be read() sit in a
string buffer, self.readbuffer.  When a piece of that string is read,
the string is split in two, with the first piece being returned, and the
second piece becoming the new self.readbuffer.  Each of these two pieces
must be allocated space and have their contents copied into them.  When
the length of the readbuffer far exceeds the number of bytes requested,
allocating space for the two substrings and copying in their contents
becomes very expensive.

The attached zeroes.zip demonstrates a worst-case scenario for this
problem.  It contains one 100 MiB file filled with zeroes.  This file
compresses to just 100 KiB, however, because it is so repetitive.  This
repetitive data means that the zlib decompressor returns many MiBs of
uncompressed data when fed just 64 KiB of compressed data.  Each call to
read() requests only 16 KiB, so each call must reallocate and copy many
MiBs.

The attached patch makes the read buffer a StringIO instead of a string.
  Each call to the decompressor creates a new StringIO buffer.  Reading
from the StringIO does not create a new string for the unread data. 
When the buffer has been exhausted, a new StringIO is created with the
next batch of uncompressed bytes.

The patch also fixes the behavior of zipfile.py when called as a script
with the -e flag.  Before, to extract a file, it decompressed the entire
file to memory, and then wrote the entire file to disk.  This behavior
is undesirable if the decompressed file is even remotely large.  Now, it
uses ZipFile.extractall(), which correctly streams the decompressed data
to disk.

unzip vs. Python's zipfile.py vs. patched zipfile.py:

$ time unzip -e zeroes.zip
Archive:  zeroes.zip
  inflating: zeroes_unzip/zeroes

real0m0.707s
user0m0.463s
sys 0m0.244s

$ time python zipfileold.py -e zeroes.zip zeroes_old

real3m42.012s
user0m57.670s
sys 2m43.510s

$ time python zipfile.py -e zeroes.zip zeroes_patched

real0m0.986s
user0m0.409s
sys 0m0.490s

In this test, the patched version is 246x faster than the unpatched
version, and is not far off the pace of the C version.

Incidentally, this patch also improves performance when the data is not
repetitive.  I tested a ZIP containing a single compressed file filled
with random data, created by running
$ dd if=/dev/urandom of=random bs=1024 count=1024
$ zip random.zip random
This archive demonstrates the opposite scenario - where compression has
next to no impact on file size, and the read buffer will never be
dramatically larger than the amount of data fed to the zlib decompressor.

$ time python zipfileold.py -e random.zip random_old

real0m0.063s
user0m0.053s
sys 0m0.010s

$ time python zipfile.py -e random.zip random_patched

real0m0.059s
user0m0.047s
sys 0m0.012s

--
components: Extension Modules
files: zipfile_read_perf.patch
keywords: patch
messages: 73880
nosy: lightstruk
severity: normal
status: open
title: ZipFileExt.read() can be incredibly slow
type: performance
versions: Python 2.6, Python 2.7
Added file: http://bugs.python.org/file11621/zipfile_read_perf.patch

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3978>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com