Nathanael England created FLINK-31069:
-----------------------------------------
Summary: Pyflink 1.16.1 has unclosed resources at the end of unit
tests
Key: FLINK-31069
URL: https://issues.apache.org/jira/browse/FLINK-31069
Project: Flink
Issue Type: Bug
Environment: Ubuntu 20.04
Python 3.8.10
Pyflink 1.16.1
Reporter: Nathanael England
A simple pyflink unit test has unclosed resources at the end of the testing. A
minimally reproducable example of this can be seen with the following in 1.16.1
{code:python}
from pyflink.testing import test_case_utils
class InputBroadcastProcessFunctionTests(
test_case_utils.PyFlinkStreamingTestCase):
def test_nothing(self):
pass
{code}
When `pytest.ini`is instructed to have `filterwarnings = errors`, the user is
met with errors like the following
{code:bash}
$ pytest example_test.py
================================================================================================================================================
test session starts
================================================================================================================================================
platform linux -- Python 3.8.10, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/my_repo, configfile: pytest.ini
plugins: forked-1.6.0, anyio-3.6.2, timeout-2.1.0, typeguard-2.13.3,
xdist-2.5.0, rabbitmq-2.2.1, cov-4.0.0
timeout: 60.0s
timeout method: signal
timeout func_only: False
collected 1 item
example_test.py E
[100%]
======================================================================================================================================================
ERRORS
=======================================================================================================================================================
_________________________________________________________________________________________________________________________
ERROR at setup of InputBroadcastProcessFunctionTests.test_nothing
_________________________________________________________________________________________________________________________
cls = <class '_pytest.runner.CallInfo'>, func = <function
call_runtest_hook.<locals>.<lambda> at 0x7f42530f0af0>, when = 'setup', reraise
= (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
@classmethod
def from_call(
cls,
func: "Callable[[], TResult]",
when: "Literal['collect', 'setup', 'call', 'teardown']",
reraise: Optional[
Union[Type[BaseException], Tuple[Type[BaseException], ...]]
] = None,
) -> "CallInfo[TResult]":
"""Call func, wrapping the result in a CallInfo.
:param func:
The function to call. Called without arguments.
:param when:
The phase in which the function is called.
:param reraise:
Exception or exceptions that shall propagate if raised by the
function, instead of being wrapped in the CallInfo.
"""
excinfo = None
start = timing.time()
precise_start = timing.perf_counter()
try:
> result: Optional[TResult] = func()
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/runner.py:339:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/runner.py:260:
in <lambda>
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pluggy/_hooks.py:265:
in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pluggy/_manager.py:80:
in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/unraisableexception.py:83:
in pytest_runtest_setup
yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
with catch_unraisable_exception() as cm:
yield
if cm.unraisable:
if cm.unraisable.err_msg is not None:
err_msg = cm.unraisable.err_msg
else:
err_msg = "Exception ignored in"
msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
msg += "".join(
traceback.format_exception(
cm.unraisable.exc_type,
cm.unraisable.exc_value,
cm.unraisable.exc_traceback,
)
)
> warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E pytest.PytestUnraisableExceptionWarning: Exception ignored in:
<function Popen.__del__ at 0x7f427ce715e0>
E
E Traceback (most recent call last):
E File "/usr/lib/python3.8/subprocess.py", line 946, in __del__
E _warn("subprocess %s is still running" % self.pid,
E ResourceWarning: subprocess 1906862 is still running
dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/unraisableexception.py:78:
PytestUnraisableExceptionWarning
-----------------------------------------------------------------------------------------------------------------------------------------------
Captured stderr setup
-----------------------------------------------------------------------------------------------------------------------------------------------
13:11:22.743 I test_case_utils.py@91: Using
/home/my_repo/dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pyflink
as FLINK_HOME...
------------------------------------------------------------------------------------------------------------------------------------------------
Captured log setup
-------------------------------------------------------------------------------------------------------------------------------------------------
INFO root:test_case_utils.py:91 Using
/home/my_repo/dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pyflink
as FLINK_HOME...
==============================================================================================================================================
short test summary info
==============================================================================================================================================
ERROR example_test.py::InputBroadcastProcessFunctionTests::test_nothing -
pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function
Popen.__del__ at 0x7f427ce715e0>
=================================================================================================================================================
1 error in 1.22s
==================================================================================================================================================
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
