godfreyhe opened a new pull request #11296: [FLINK-16363] [table] Correct the execution behavior of TableEnvironment and StreamTableEnvironment URL: https://github.com/apache/flink/pull/11296 ## What is the purpose of the change *Both TableEnvironment.execute() and StreamExecutionEnvironment.execute can trigger a Flink table program execution. However if you use TableEnvironment to build a Flink table program, you must use TableEnvironment.execute() to trigger execution, because you can’t get the StreamExecutionEnvironment instance. If you use StreamTableEnvironment to build a Flink table program, you can use both to trigger execution. If you convert a table program to a DataStream program (using StreamExecutionEnvironment.toAppendStream/toRetractStream), you also can use both to trigger execution. So it’s hard to explain which `execute` method should be used. For example: StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); StreamTableEnvironment tEnv = StreamTableEnvironment.create(env); tEnv.sqlUpdate("INSERT INTO sink1 SELECT a, b FROM MyTable1") Table table = tEnv.sqlQuery("SELECT c, d from MyTable2") DataStream dataStream = tEnv.toAppendStream(table, Row.class) dataStream… env.execute("job name") ; // or tEnv.execute("job name"); // both `env.execute` and `tEnv.execute` can trigger the execution To correct current messy trigger point, we propose that: for TableEnvironment and StreamTableEnvironment, you must use TableEnvironment.execute() to trigger table program execution, once you convert the table program to a DataStream program (through toAppendStream or toRetractStream method), you must use StreamExecutionEnvironment.execute to trigger the DataStream program. StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); StreamTableEnvironment tEnv = StreamTableEnvironment.create(env); tEnv.sqlUpdate("INSERT INTO sink1 SELECT a, b FROM MyTable1") Table table = tEnv.sqlQuery("SELECT c, d from MyTable2") DataStream dataStream = tEnv.toAppendStream(table, Row.class) dataStream… env.execute("job name") ; // only trigger the DataStream program tEnv.execute("job name"); // only trigger the sql update query program * ## Brief change log - *remove isEagerOperationTranslation method, and the buffer behavior is same for both TableEnvironmentImpl and StreamTableEnvironmentImpl* - *Use DummyStreamExecutionEnvironment to buffer the transformation generated during translation in planner* - *use StreamGraphGenerator to generate StreamGraph in Executor* _ *add createPipeline method in Executor and getPipeline method in StreamTableEnvironmentImpl for sql client* ## Verifying this change This change added tests and can be verified as follows: - *Added TableEnvironmentITCase to verify different TableEnvironment methods combination* - *update the existing test cases based on new execution behavior * ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no)** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (**yes** / no) - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
