New submission from Kristiyan :
Hello,
Last several days I'm trying to implement an async "opener" object that can be
used as Coroutine as well as an AsyncContextManager (eg. work with `await
obj.open()` and `async with obj.open()`). I've researched several
implementations from various python packages such as:
1. aiofiles:
https://github.com/Tinche/aiofiles/blob/master/src/aiofiles/base.py#L28
2. aiohttp:
https://github.com/aio-libs/aiohttp/blob/master/aiohttp/client.py#L1082
Unlike these libs though, I want my implementation to return a custom object
that is a wrapper around the object returned from the underlying module I'm
hiding.
Example:
I want to implement a DataFeeder interface that has a single method `open()`.
Sub-classes of this interface will support, for example, opening an file using
aiofiles package. So,
AsyncFileDataFeeder.open() will call `aiofiles.open()`, but instead of
returning "file-handle" from aiofiles, I want to return a custom Feed class
that implements some more methods for reading -- for example:
async with async_data_feeder.open() as feed:
async for chunk in feed.iter_chunked():
...
To support that I'm returning an instance of the following class from
DataFeeder.open():
class ContextOpener(
Coroutine[Any, Any, Feed],
AbstractAsyncContextManager[Feed],
):
__slots__ = ("_wrapped_coro", "_feed_cls", "_feed")
def __init__(self, opener_coro: Coroutine, feed_cls: Type[Feed]):
self._wrapped_coro = opener_coro
self._feed_cls = feed_cls
self._feed: Any = None
def __await__(self) -> Generator[Any, Any, Feed]:
print("in await", locals())
handle = yield from self._wrapped_coro.__await__()
return self._feed_cls(handle)
def send(self, value: Any) -> Any:
print("in send", locals())
return self._wrapped_coro.send(value)
def throw(self, *args, **kwargs) -> Any:
print("in throw", locals())
return self._wrapped_coro.throw(*args, **kwargs)
def close(self) -> None:
print("in close", locals())
self._wrapped_coro.close()
async def __aenter__(self) -> feeds.Feed:
handle = await self._wrapped_coro
self._feed = self._feed_cls(handle)
return self._feed
async def __aexit__(
self,
exc_type: Optional[Type[BaseException]],
exc: Optional[BaseException],
tb: Optional[TracebackType],
) -> None:
await self._feed.close()
self._feed = None
This code actually works! But I've noticed that when calling `await
DataFeeder.open()` the event loop never calls my `send()` method.
if __name__ == "__main__":
async def open_test():
await asyncio.sleep(1)
return 1
async def main():
c = ContextOpener(open_test(), feeds.AsyncFileFeed)
ret = await c
print("Finish:", ret, ret._handle)
The output:
in await {'self': <__main__.ContextOpener object at 0x11099cd10>}
Finish: 1
>From then on a great thinking and reading on the Internet happened, trying to
>explain to myself how exactly coroutines are working. I suspect that the
>ContextOpener.__await__ is returning a generator instance and from then on,
>outer coroutines (eg. main in this case) are calling send()/throw()/close() on
>THAT generator, not on the ContextOpener "coroutine".
The only way to make Python call ContextOpener send() method (and friends) is
when ContextOpener is the outermost coroutine that is communicating directly
with the event loop:
ret = asyncio.run(ContextOpener(open_test(), feeds.AsyncFileFeed))
print("Finish:", ret)
Output:
in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None}
in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None}
Finish: 1
However, now I see that I have an error in my implementation that was hidden
before: my send() method implementation is not complete because StopIteration
case is not handled and returns 1, instead of Feed object.
Since __await__() should return iterator (by PEP492) I can't figure out a way
to implement what I want unless making my coroutine class an iterator itself
(actually generator) by returning `self` from __await__ and add __iter__ and
__next__ methods:
def __await__(self):
return self
def __iter__(self):
return self
def __next__(self):
return self.send(None)
Is this the proper way to make a Coroutine out of a collections.abc.Coroutine?
Why is then the documentation not explicitly saying that a Coroutine should
inherit from collections.abc.Generator?
I see this as very common misconception since every such "ContextManager"
similar to Conte
