New submission from Serhiy Storchaka:
Proposed patch makes a number of classes produce more compact pickle data in
common case. This includes iterators of list, tuple, str, bytes, bytearray,
enumerate, array, deque, iterator object for classes with __getitem__, some
itertools iterators, and non-iterator objects: slice, bytearray, deque. This is
achieved by omitting default constructor arguments or state.
Exhausted iterators are pickled as iter(()). This is not new, exhausted bytes,
and bytearray iterators, and reversed list iterator are already pickled as
iter('') or iter([]) correspondingly. iter(()) is just the simplest way to
create an empty iterator and it has the most compact pickle representation.
An example.
Unpatched:
>>> import pickle, pickletools, itertools
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
80
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
66
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh',
>>> 4), 3)))
0: \x80 PROTO 3
2: c GLOBAL 'itertools islice'
20: ( MARK
21: c GLOBAL 'builtins iter'
36: X BINUNICODE 'abcdefgh'
49: \x85 TUPLE1
50: R REDUCE
51: K BININT1 0
53: b BUILD
54: K BININT1 0
56: K BININT1 4
58: K BININT1 1
60: t TUPLE (MARK at 20)
61: R REDUCE
62: K BININT1 0
64: b BUILD
65: . STOP
highest protocol among opcodes = 2
Patched:
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
69
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
55
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh',
>>> 4), 3)))
0: \x80 PROTO 3
2: c GLOBAL 'itertools islice'
20: c GLOBAL 'builtins iter'
35: X BINUNICODE 'abcdefgh'
48: \x85 TUPLE1
49: R REDUCE
50: K BININT1 4
52: \x86 TUPLE2
53: R REDUCE
54: . STOP
highest protocol among opcodes = 2
----------
components: Extension Modules, Interpreter Core
files: iterators_pickle.diff
keywords: patch
messages: 255699
nosy: alexandre.vassalotti, pitrou, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: More compact pickle of iterators etc
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file41207/iterators_pickle.diff
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25776>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
