[issue19256] Optimize marshal format and add version token.

Serhiy Storchaka added the comment: Largest modules in the stdlib have up to 800 references. So it would be worth to have 4 opcodes for short (10-bit) ref indices. -- ___ Python tracker ___

[issue19256] Optimize marshal format and add version token.

Changes by Stefan Krah : -- nosy: -skrah ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.o

[issue19256] Optimize marshal format and add version token.

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue19256] Optimize marshal format and add version token.

Kristján Valur Jónsson added the comment: Never put off to tomorrow what you can do today :) -- ___ Python tracker ___ ___ Python-bugs

[issue19256] Optimize marshal format and add version token.

Serhiy Storchaka added the comment: What is difference between introducing the version token right now or only when (if?) we will need it? -- ___ Python tracker ___

[issue19256] Optimize marshal format and add version token.

Kristján Valur Jónsson added the comment: Quoting myself: "for prudence." We probably should have had this from the beginning. Adding this now makes it easier to make such changes in future, because then you don't have to re-invent the versioning mechanism. -- ___

[issue19256] Optimize marshal format and add version token.

Kristján Valur Jónsson added the comment: Only output TYPE_SHORT_REF for version >= 4 -- Added file: http://bugs.python.org/file32115/marshal2.patch ___ Python tracker ___ __

[issue19256] Optimize marshal format and add version token.

Serhiy Storchaka added the comment: > Right, the idea of the version token is to introduce it now, as early as > possible, even if it is not needed, for prudence. How it will help? If we will break backward compatibility in version 10, then we will introduce the version token for version 10 an

[issue19256] Optimize marshal format and add version token.

Kristján Valur Jónsson added the comment: We have done so previously and should continue to do that. Thanks for pointing out that the new SHORT_REF needs that fix :) -- ___ Python tracker

[issue19256] Optimize marshal format and add version token.

Antoine Pitrou added the comment: > Should we support serialization in an older version? For example, > disable the new SMALL and ASCII tokens? It is officially supported with the "version" parameter: http://docs.python.org/3.4/library/marshal.html#marshal.dump -- _

[issue19256] Optimize marshal format and add version token.

STINNER Victor added the comment: Should we support serialization in an older version? For example, disable the new SMALL and ASCII tokens? -- ___ Python tracker ___

[issue19256] Optimize marshal format and add version token.

Kristján Valur Jónsson added the comment: Right, the idea of the version token is to introduce it now, as early as possible, even if it is not needed, for prudence. For example, if version 5 decides to change the semantics of some of the opcodes, we can then support both kinds, in the future.

[issue19256] Optimize marshal format and add version token.

Antoine Pitrou added the comment: > The version token needed only when we want break backward > compatibility (change the meaning of existing opcodes). This is true. For example, my first hunch was to make w_long() emit variable-length data, (0-254: one byte integer, 255: four bytes integer foll

[issue19256] Optimize marshal format and add version token.

Serhiy Storchaka added the comment: The version token needed only when we want break backward compatibility (change the meaning of existing opcodes). -- ___ Python tracker ___ _

[issue19256] Optimize marshal format and add version token.

Raymond Hettinger added the comment: Unlike pickle, the marshal module makes no promises about keeping the format consistent between Python versions. -- nosy: +rhettinger ___ Python tracker ___

[issue19256] Optimize marshal format and add version token.

Antoine Pitrou added the comment: > "I actually agree with Kristjan that an opcode is the least disruptive > choice here." > > Does you mean that data serialized with python 3.3 can be read with > python 3.4? But not the opposite (version token unknown in Python > 3.3)? Yes, indeed. I'm not sur

[issue19256] Optimize marshal format and add version token.

STINNER Victor added the comment: "I actually agree with Kristjan that an opcode is the least disruptive choice here." Does you mean that data serialized with python 3.3 can be read with python 3.4? But not the opposite (version token unknown in Python 3.3)? --

[issue19256] Optimize marshal format and add version token.

Antoine Pitrou added the comment: I actually agree with Kristjan that an opcode is the least disruptive choice here. I also agree it's useful if we want to be able to evolve the protocol without being tied by backwards compatibility constraints. -- __

[issue19256] Optimize marshal format and add version token.

STINNER Victor added the comment: Good idea to add a version. But it should not be a token, but a mandatory header at the beginning. For example, 16 unsigned bits at the beginning. Many file formats use a "magic" key, like "MZ" for Windows executable or "GIF" for GIF pictures. What do you thin

[issue19256] Optimize marshal format and add version token.

Serhiy Storchaka added the comment: Let defer version token until it be needed. -- nosy: +serhiy.storchaka ___ Python tracker ___ ___

[issue19256] Optimize marshal format and add version token.

Changes by Antoine Pitrou : -- nosy: +skrah ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue19256] Optimize marshal format and add version token.

New submission from Kristján Valur Jónsson: Issue $19219 added new tokens making marshal format smaller and faster. This patch adds two new tokens: TYPE_SHORT_REF for which the ref index is a byte and TYPE_VERSION for which the operand is the protocol version. The former helps because it catches