New submission from Olivier Grisel <olivier.gri...@ensta.org>: I noticed that both pickle.Pickler (C version) and pickle._Pickler (Python version) make unnecessary memory copies when dumping large str, bytes and bytearray objects.
This is caused by unnecessary concatenation of the opcode and size header with the large bytes payload prior to calling self.write. For protocol 4, an additional copy is caused by the framing mechanism. I will submit a pull request to fix the issue for the Python version. I am not sure how to test this properly. The BigmemPickleTests seems to be skipped on my 16 GB laptop. ---------- components: Library (Lib) messages: 305975 nosy: Olivier.Grisel, pitrou priority: normal severity: normal status: open title: pickle.dump allocates unnecessary temporary bytes / str type: resource usage versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31993> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
