[issue32034] Impossible to unpickle IncompleteReadError

[badouxn]

New submission from badouxn :

When using asyncio in combination with multiprocessing, a TypeError occur when 
readuntil() encounter an EOF instead of the delimiter.
readuntil return a IncompleteReadError exception which is pickled by the 
multiprocessing package.
The unpickling failed due to an argument being None. As noted in the following 
links, it fails:
https://stackoverflow.com/questions/16244923/how-to-make-a-custom-exception-class-with-multiple-init-args-pickleable

The error trace is: 
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
  File "/usr/lib64/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 463, in 
_handle_results
task = get()
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 252, in recv
return _ForkingPickler.loads(r)
TypeError: __init__() missing 1 required positional argument: 'expected'


Fix proposed: 
Make the "expected" parameter of the IncompleteReadError exception class 
optional.

--
components: asyncio
files: Impossible_to_unpickle_IncompleteReadError.py
messages: 306268
nosy: badouxn, yselivanov
priority: normal
severity: normal
status: open
title: Impossible to unpickle IncompleteReadError
Added file: 
https://bugs.python.org/file47266/Impossible_to_unpickle_IncompleteReadError.py

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32034] Error when unpickling IncompleteReadError

[badouxn]

Change by badouxn :


--
title: Impossible to unpickle IncompleteReadError -> Error when unpickling 
IncompleteReadError

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32034] Error when unpickling IncompleteReadError

[badouxn]

Change by badouxn :


--
versions: +Python 3.7

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32034] Error when unpickling IncompleteReadError

[badouxn]

Change by badouxn :


--
keywords: +patch
pull_requests: +4353
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32034] Error when unpickling IncompleteReadError

[badouxn]

badouxn  added the comment:

My first fix was indeed wrong. 
Upon looking further into it, it seems to me that the problem come from the 
fact that the call to super is not done with the right argument. 
Upon unpickling, the argument that will be passed to the __init__ is the string 
representation built on line 33-34.
That's why, when leaving expected as optional the number returned is 44, the 
length of the string. 

I went looking for similar Exception class in the code base and found the 
MaybeEncodingError in multiprocessing/pool.py 

By replacing the Error content with this one I don't have any error anymore. 
The message is kept identical. 
'''
 def __init__(self, partial, expected):
 super().__init__(partial, expected)
 self.partial = partial
 self.expected = expected
 def __str__(self):
 return ("%d bytes read on a total of %r expected bytes" % 
(len(partial), expected))
'''

Does such a fix looks correct to you ?

--

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32034] Error when unpickling IncompleteReadError

[badouxn]

badouxn  added the comment:

Typo in the last comment. The code should be:

'''
 def __init__(self, partial, expected):
 super().__init__(partial, expected)
 self.partial = partial
 self.expected = expected
 def __str__(self):
 return ("%d bytes read on a total of %r expected bytes" % 
(len(self.partial), self.expected))
'''

--

___
Python tracker 
<https://bugs.python.org/issue32034>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com