Hi Xingbo, Yes, there are a number of temporary views being created, where each is being created using SQL (CREATE TEMPORARY VIEW ...) rather than explicit calls to the Table and DataStream APIs.
Is this a good pattern or are there caveats I should be aware of please? Many thanks, John ________________________________ From: Xingbo Huang <hxbks...@gmail.com> Sent: 16 June 2022 12:34 To: John Tipper <john_tip...@hotmail.com> Cc: user@flink.apache.org <user@flink.apache.org> Subject: Re: The configured managed memory fraction for Python worker process must be within (0, 1], was: %s Hi John, Does your job logic include conversion between Table and DataStream? For example, methods such as `create_temporary_view(path: str, data_stream: DataStream): -> Table` are used. Best, Xingbo John Tipper <john_tip...@hotmail.com<mailto:john_tip...@hotmail.com>> 于2022年6月16日周四 18:31写道： Hi Xingbo, I’m afraid I can’t share my code but Flink is 1.13. The main Flink code is running inside Kinesis on AWS so I cannot change the version. Many thanks, John Sent from my iPhone On 16 Jun 2022, at 10:37, Xingbo Huang <hxbks...@gmail.com<mailto:hxbks...@gmail.com>> wrote: Hi John, Could you provide the code snippet and the version of pyflink you used? Best, Xingbo John Tipper <john_tip...@hotmail.com<mailto:john_tip...@hotmail.com>> 于2022年6月16日周四 17:05写道： Hi all, I'm trying to run a PyFlink unit test to test some PyFlink SQL and where my code uses a Python UDF. I can't share my code but the test case is similar to the code here: https://github.com/apache/flink/blob/f8172cdbbc27344896d961be4b0b9cdbf000b5cd/flink-python/pyflink/testing/test_case_utils.py When I have some simple SQL everything is fine. When I add a more complex query I get an error, which looks like it's memory related. java.lang.IllegalArgumentException: The configured managed memory fraction for Python worker process must be within (0, 1], was: %s. It may be because the consumer type "Python" was missing or set to 0 for the config option "taskmanager.memory.managed.consumer-weights".0.0 In my test case setUp(), I try to set that value like this, but it seems to have no effect: self.t_env.get_config().get_configuration().set_string("taskmanager.memory.managed.consumer-weights", "PYTHON:30") Am I not setting it correctly, or is there something else I need to do to fix this error? Many thanks, John
