[issue39681] pickle.load expects an object that implements readinto

Nathan Goldbaum added the comment: Thank you for the fix! Yes I'm planning to file an issue with flair about this and patch this use case in PyTorch itself. -- ___ Python tracker ___

[issue39681] pickle.load expects an object that implements readinto

Antoine Pitrou added the comment: The regression is fixed in git master and in the 3.8 branch. However, I would still advice against the idiom of handling a mmap object as a regular file when using pickle. -- resolution: -> fixed stage: patch review -> resolved status: open -> close

[issue39681] pickle.load expects an object that implements readinto

Łukasz Langa added the comment: New changeset b19f7ecfa3adc6ba1544225317b9473649815b38 by Miss Islington (bot) in branch '3.8': bpo-39681: Fix C pickle regression with minimal file-like objects (GH-18592) (#18630) https://github.com/python/cpython/commit/b19f7ecfa3adc6ba1544225317b9473649815b

[issue39681] pickle.load expects an object that implements readinto

Łukasz Langa added the comment: New changeset 9f37872e307734666a7169f7be6e3370d3068282 by Antoine Pitrou in branch 'master': bpo-39681: Fix C pickle regression with minimal file-like objects (#18592) https://github.com/python/cpython/commit/9f37872e307734666a7169f7be6e3370d3068282 --

[issue39681] pickle.load expects an object that implements readinto

Change by miss-islington : -- nosy: +miss-islington nosy_count: 3.0 -> 4.0 pull_requests: +17995 pull_request: https://github.com/python/cpython/pull/18630 ___ Python tracker _

[issue39681] pickle.load expects an object that implements readinto

Change by Antoine Pitrou : -- assignee: -> pitrou type: -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue39681] pickle.load expects an object that implements readinto

Change by Antoine Pitrou : -- keywords: +patch pull_requests: +17960 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18592 ___ Python tracker ___ _

[issue39681] pickle.load expects an object that implements readinto

Antoine Pitrou added the comment: Well, in the mmap case, this is trivially fixed by writing: ``` with open("my_data.pkl", "r+b") as f_in: mm = mmap.mmap(f_in.fileno(), 0) print(pickle.loads(memoryview(mm))) ``` It will also be faster this way, since the pickle will be read directly from

[issue39681] pickle.load expects an object that implements readinto

Nathan Goldbaum added the comment: So I *think* I've pieced together what caused the user crash that originated in the flair library. It turns out that pickle.load, via torch.load, is getting passed an mmap.mmap. https://github.com/flairNLP/flair/blob/1d44abf35f1122d1e146c9a219b2cb5f98b41b4

[issue39681] pickle.load expects an object that implements readinto

Nathan Goldbaum added the comment: In this case the tests are explicitly testing that a file-like object that does not implement readinto works with torch.load (which is using pickles under the hood). See https://github.com/pytorch/pytorch/blob/master/test/test_serialization.py#L416-L429, i

[issue39681] pickle.load expects an object that implements readinto

Antoine Pitrou added the comment: Right, I think at some point during the PEP 574 implementation review, we decided that the input object had to be a proper file implementation (for example, deriving from io.BufferedIOBase). I think it should be easy for PyTorch to fix this internally (by der

[issue39681] pickle.load expects an object that implements readinto

Change by Karthikeyan Singaravelan : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:/

[issue39681] pickle.load expects an object that implements readinto

New submission from Nathan Goldbaum : As of https://github.com/python/cpython/pull/7076, it looks like at least the C implementation of pickle.load expects the file argument to implement readinto: https://github.com/python/cpython/blob/ffd9753a944916ced659b2c77aebe66a6c9fbab5/Modules/_pickle.c