Re: python Table API problem with pandas

2023-10-27 Thread Alexey Sergeev
# problem_3.py # call to .where() after .map() with pandas type function # also resets column names # and doesn't really filter values import pandas as pd t_env = TableEnvironment.create(EnvironmentSettings.in_streaming_mode()) table = t_env.from_elements( elements=[ (1, 'China'),

Re: python Table API problem with pandas

2023-10-27 Thread Alexey Sergeev
I'll copy the problems here if your prefer that. # problem_1.py # add_columns() resets column names to default names f0, f1, ..., fN t_env = TableEnvironment.create(EnvironmentSettings.in_streaming_mode()) table = t_env.from_elements( elements=[ (1, '{"name": "Flink"}'), (2,

Re: python Table API problem with pandas

2023-10-27 Thread Alexey Sergeev
# problem_2.py # .alias() does not work either import json t_env = TableEnvironment.create(EnvironmentSettings.in_streaming_mode()) table = t_env.from_elements( elements=[ (1, '{"name": "Flink"}'), (2, '{"name": "hello"}'), (3, '{"name": "world"}'), (4, '{"na

python Table API problem with pandas

2023-10-27 Thread Alexey Sergeev
Hi everyone, Python Table API seems to be a little bit buggy. Some minimal examples of strange behaviors here: https://gist.github.com/nrdhm/88322a68fc3e9a14a5f4ab6ec13403cf Was testing in pyflink-shell in our small cluster with Flink 1.17. Docker image: flink:1.17.1-scala_2.12-java11 The