New submission from Kristiyan <skre...@gmail.com>:
Hello, Last several days I'm trying to implement an async "opener" object that can be used as Coroutine as well as an AsyncContextManager (eg. work with `await obj.open()` and `async with obj.open()`). I've researched several implementations from various python packages such as: 1. aiofiles: https://github.com/Tinche/aiofiles/blob/master/src/aiofiles/base.py#L28 2. aiohttp: https://github.com/aio-libs/aiohttp/blob/master/aiohttp/client.py#L1082 Unlike these libs though, I want my implementation to return a custom object that is a wrapper around the object returned from the underlying module I'm hiding. Example: I want to implement a DataFeeder interface that has a single method `open()`. Sub-classes of this interface will support, for example, opening an file using aiofiles package. So, AsyncFileDataFeeder.open() will call `aiofiles.open()`, but instead of returning "file-handle" from aiofiles, I want to return a custom Feed class that implements some more methods for reading -- for example: async with async_data_feeder.open() as feed: async for chunk in feed.iter_chunked(): ... To support that I'm returning an instance of the following class from DataFeeder.open(): class ContextOpener( Coroutine[Any, Any, Feed], AbstractAsyncContextManager[Feed], ): __slots__ = ("_wrapped_coro", "_feed_cls", "_feed") def __init__(self, opener_coro: Coroutine, feed_cls: Type[Feed]): self._wrapped_coro = opener_coro self._feed_cls = feed_cls self._feed: Any = None def __await__(self) -> Generator[Any, Any, Feed]: print("in await", locals()) handle = yield from self._wrapped_coro.__await__() return self._feed_cls(handle) def send(self, value: Any) -> Any: print("in send", locals()) return self._wrapped_coro.send(value) def throw(self, *args, **kwargs) -> Any: print("in throw", locals()) return self._wrapped_coro.throw(*args, **kwargs) def close(self) -> None: print("in close", locals()) self._wrapped_coro.close() async def __aenter__(self) -> feeds.Feed: handle = await self._wrapped_coro self._feed = self._feed_cls(handle) return self._feed async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc: Optional[BaseException], tb: Optional[TracebackType], ) -> None: await self._feed.close() self._feed = None This code actually works! But I've noticed that when calling `await DataFeeder.open()` the event loop never calls my `send()` method. if __name__ == "__main__": async def open_test(): await asyncio.sleep(1) return 1 async def main(): c = ContextOpener(open_test(), feeds.AsyncFileFeed) ret = await c print("Finish:", ret, ret._handle) The output: in await {'self': <__main__.ContextOpener object at 0x11099cd10>} Finish: <feeds.AsyncFileFeed object at 0x1109a9a80> 1 >From then on a great thinking and reading on the Internet happened, trying to >explain to myself how exactly coroutines are working. I suspect that the >ContextOpener.__await__ is returning a generator instance and from then on, >outer coroutines (eg. main in this case) are calling send()/throw()/close() on >THAT generator, not on the ContextOpener "coroutine". The only way to make Python call ContextOpener send() method (and friends) is when ContextOpener is the outermost coroutine that is communicating directly with the event loop: ret = asyncio.run(ContextOpener(open_test(), feeds.AsyncFileFeed)) print("Finish:", ret) Output: in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None} in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None} Finish: 1 However, now I see that I have an error in my implementation that was hidden before: my send() method implementation is not complete because StopIteration case is not handled and returns 1, instead of Feed object. Since __await__() should return iterator (by PEP492) I can't figure out a way to implement what I want unless making my coroutine class an iterator itself (actually generator) by returning `self` from __await__ and add __iter__ and __next__ methods: def __await__(self): return self def __iter__(self): return self def __next__(self): return self.send(None) Is this the proper way to make a Coroutine out of a collections.abc.Coroutine? Why is then the documentation not explicitly saying that a Coroutine should inherit from collections.abc.Generator? I see this as very common misconception since every such "ContextManager" similar to ContextOpener from 3rd party packages (like the aforementioned two, aiofiles and aiohttp, but there are others as well) is subclassing collections.abc.Coroutine and implements send(), throw() and close() methods that are not actually being called. I suspect, the authors of these libraries haven't noticed that because the returned value from the __await__() and send() methods is the same in their case. ---------- components: asyncio messages: 413652 nosy: asvetlov, skrech, yselivanov priority: normal severity: normal status: open title: Proper way to inherit from collections.abc.Coroutine versions: Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue46818> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
