[issue1502] itertools should grow chunkify

[Chris Mellon]

New submission from Chris Mellon:

One of the most common requests in c.l.p and #python is a way to break
an iterable up into some particular size chunks. For example, "abcdef"
-> "ab", "cd", "ef". It's pretty easy to write one, but there are a few
subtleties to it (like if you want padding or partial results) and it's
so common that having it in the stdlib would be nice.

Attached is a patch which implements itertools.chunkify. It can
optionally discard, pad, or return any leftovers in the source iterable.
Tests and docstrings are included, but it needs to be documented in the
manual. One thing it does not do, but maybe it should, is guess what
type the yielded values should have based on the input sequence - it
always returns lists.

Patch is against trunk, r59186.

--
components: Library (Lib)
files: chunkify.patch
messages: 57861
nosy: arkanes
severity: normal
status: open
title: itertools should grow chunkify
type: behavior
versions: Python 2.6
Added file: http://bugs.python.org/file8811/chunkify.patch

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1502>
__

chunkify.patch
Description: Binary data
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1518] Fast globals/builtins access (patch)

[Chris Mellon]

Chris Mellon added the comment:

In funcobject.c:PyFunction_New, the declarations of op and __name__ need
to be moved to the top of the function to compile in MSVC, and if
accepted the fastglobals.c file needs to be added to PCBuild.

In the test script, the use of import * generates syntax warnings that
make the output awkward to read, and the benchmark for dict_get is
actually running dict_set. I'm attaching the fixed copy of the test
script I used.

I see roughly the same speed ups (MSVC 7.1, Windows XP, Intel Core2 Duo
@ 2.13Ghz), but I see a 10% slowdown in the dict-insert/delete and
dict-set benchmarks which I find very troubling.

TestTrunk  fastglobalsTime difference
--
Dict insert/del 2.0024954952.2074091251.102329134
Dict get0.7502532050.7455766620.993766714
Dict set0.9826959211.1149977661.13463152
Local get   0.5333870290.51337118 0.96247406
Local set   0.5965657740.6141249141.029433703
Global get  0.9356050730.7311365840.78145855
Global set  1.48638532 1.03868462 0.69879903
Builtin get 1.3926063670.7351806730.52791707
Function call   1.9387057811.7162330040.885246756
List comp   1.5477801051.1882157560.767690289


PyBench shows an overall slowdown - String mappings in particular,
string/unicode predicates, and new instance creation all show
significant slowdowns. The results are fairly verbose so I've uploaded
them as a google docs spreadsheet at
http://spreadsheets.google.com/ccc?key=p7g0z40g_NpvH5UpPTpr-Ag&hl=en

I notice that you create a new PyFastGlobals object in every call to
PyEval_EvalCode. This might explain some of the general case slowdown,
is this really what you want to do?

--
nosy: +arkanes
Added file: http://bugs.python.org/file8823/fastglobals_test.py

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1518>
__import timeit, new, time, dis

MULTIPLIER = 1


def test_local_get():
	x = 0
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x
	x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x; x


def test_local_set():
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0; x = 0
	x = 0; x = 0; 

[issue1518] Fast globals/builtins access (patch)

[Chris Mellon]

Chris Mellon added the comment:

I may have had some old code in the build or something - I did a clean
rebuild to look at some of the slowdowns and the fastglobals_test
benchmarks are now even better than Neils. Pybench is still overall
slower but it's mostly in float operations, which seems odd and I'd
discount it unless someone else can recreate it. I've updated the
spreadsheet I linked before with the updated timings, along with my
microbenchmark and pystone results. Very impressive speedup on pystone,
no doubt because of the heavy use of globals.

__
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1518>
__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1757072] Zipfile robustness

[Chris Mellon]

Chris Mellon added the comment:

I agree that the zipfile is out of spec. Here are my arguments in favor
of making the change anyway:

Existing zip tools like 7zip, pkzip, and winzip handle these files "as
expected"

As far as I know, it won't break any valid zipfiles.

Because the fix necessary is buried inside a private function in the
zipfile module, it's difficult to monkey-patch the behavior into place
and it's quite hard to figure out what needs to be done.

_
Tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue1757072>
_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com