One comment to `executeMultilineSql`, I'm afraid sometimes user might forget to iterate the returned iterators, e.g. user submits a bunch of DDLs and expect the framework will execute them one by one. But it didn't.
Best, Kurt On Wed, Apr 1, 2020 at 5:10 PM Aljoscha Krettek <aljos...@apache.org> wrote: > Agreed to what Dawid and Timo said. > > To answer your question about multi line SQL: no, we don't think we need > this in Flink 1.11, we only wanted to make sure that the interfaces that > we now put in place will potentially allow this in the future. > > Best, > Aljoscha > > On 01.04.20 09:31, godfrey he wrote: > > Hi, Timo & Dawid, > > > > Thanks so much for the effort of `multiline statements supporting`, > > I have a few questions about this method: > > > > 1. users can well control the execution logic through the proposed method > > if they know what the statements are (a statement is a DDL, a DML or > > others). > > but if a statement is from a file, that means users do not know what the > > statements are, > > the execution behavior is unclear. > > As a platform user, I think this method is hard to use, unless the > platform > > defines > > a set of rule about the statements order, such as: no select in the > middle, > > dml must be at tail of sql file (which may be the most case in product > > env). > > Otherwise the platform must parse the sql first, then know what the > > statements are. > > If do like that, the platform can handle all cases through `executeSql` > and > > `StatementSet`. > > > > 2. SQL client can't also use `executeMultilineSql` to supports multiline > > statements, > > because there are some special commands introduced in SQL client, > > such as `quit`, `source`, `load jar` (not exist now, but maybe we need > this > > command > > to support dynamic table source and udf). > > Does TableEnvironment also supports those commands? > > > > 3. btw, we must have this feature in release-1.11? I find there are few > > user cases > > in the feedback document which behavior is unclear now. > > > > regarding to "change the return value from `Iterable<Row` to > > `Iterator<Row`", > > I couldn't agree more with this change. Just as Dawid mentioned > > "The contract of the Iterable#iterator is that it returns a new iterator > > each time, > > which effectively means we can iterate the results multiple times.", > > we does not provide iterate the results multiple times. > > If we want do that, the client must buffer all results. but it's > impossible > > for streaming job. > > > > Best, > > Godfrey > > > > Dawid Wysakowicz <dwysakow...@apache.org> 于2020年4月1日周三 上午3:14写道： > > > >> Thank you Timo for the great summary! It covers (almost) all the topics. > >> Even though in the end we are not suggesting much changes to the current > >> state of FLIP I think it is important to lay out all possible use cases > >> so that we do not change the execution model every release. > >> > >> There is one additional thing we discussed. Could we change the result > >> type of TableResult#collect to Iterator<Row>? Even though those > >> interfaces do not differ much. I think Iterator better describes that > >> the results might not be materialized on the client side, but can be > >> retrieved on a per record basis. The contract of the Iterable#iterator > >> is that it returns a new iterator each time, which effectively means we > >> can iterate the results multiple times. Iterating the results is not > >> possible when we don't retrieve all the results from the cluster at > once. > >> > >> I think we should also use Iterator for > >> TableEnvironment#executeMultilineSql(String statements): > >> Iterator<TableResult>. > >> > >> Best, > >> > >> Dawid > >> > >> On 31/03/2020 19:27, Timo Walther wrote: > >>> Hi Godfrey, > >>> > >>> Aljoscha, Dawid, Klou, and I had another discussion around FLIP-84. In > >>> particular, we discussed how the current status of the FLIP and the > >>> future requirements around multiline statements, async/sync, collect() > >>> fit together. > >>> > >>> We also updated the FLIP-84 Feedback Summary document [1] with some > >>> use cases. > >>> > >>> We believe that we found a good solution that also fits to what is in > >>> the current FLIP. So no bigger changes necessary, which is great! > >>> > >>> Our findings were: > >>> > >>> 1. Async vs sync submission of Flink jobs: > >>> > >>> Having a blocking `execute()` in DataStream API was rather a mistake. > >>> Instead all submissions should be async because this allows supporting > >>> both modes if necessary. Thus, submitting all queries async sounds > >>> good to us. If users want to run a job sync, they can use the > >>> JobClient and wait for completion (or collect() in case of batch jobs). > >>> > >>> 2. Multi-statement execution: > >>> > >>> For the multi-statement execution, we don't see a contradication with > >>> the async execution behavior. We imagine a method like: > >>> > >>> TableEnvironment#executeMultilineSql(String statements): > >>> Iterable<TableResult> > >>> > >>> Where the `Iterator#next()` method would trigger the next statement > >>> submission. This allows a caller to decide synchronously when to > >>> submit statements async to the cluster. Thus, a service such as the > >>> SQL Client can handle the result of each statement individually and > >>> process statement by statement sequentially. > >>> > >>> 3. The role of TableResult and result retrieval in general > >>> > >>> `TableResult` is similar to `JobClient`. Instead of returning a > >>> `CompletableFuture` of something, it is a concrete util class where > >>> some methods have the behavior of completable future (e.g. collect(), > >>> print()) and some are already completed (getTableSchema(), > >>> getResultKind()). > >>> > >>> `StatementSet#execute()` returns a single `TableResult` because the > >>> order is undefined in a set and all statements have the same schema. > >>> Its `collect()` will return a row for each executed `INSERT INTO` in > >>> the order of statement definition. > >>> > >>> For simple `SELECT * FROM ...`, the query execution might block until > >>> `collect()` is called to pull buffered rows from the job (from > >>> socket/REST API what ever we will use in the future). We can say that > >>> a statement finished successfully, when the `collect#Iterator#hasNext` > >>> has returned false. > >>> > >>> I hope this summarizes our discussion @Dawid/Aljoscha/Klou? > >>> > >>> It would be great if we can add these findings to the FLIP before we > >>> start voting. > >>> > >>> One minor thing: some `execute()` methods still throw a checked > >>> exception; can we remove that from the FLIP? Also the above mentioned > >>> `Iterator#next()` would trigger an execution without throwing a > >>> checked exception. > >>> > >>> Thanks, > >>> Timo > >>> > >>> [1] > >>> > >> > https://docs.google.com/document/d/1ueLjQWRPdLTFB_TReAyhseAX-1N3j4WYWD0F02Uau0E/edit# > >>> > >>> On 31.03.20 06:28, godfrey he wrote: > >>>> Hi, Timo & Jark > >>>> > >>>> Thanks for your explanation. > >>>> Agree with you that async execution should always be async, > >>>> and sync execution scenario can be covered by async execution. > >>>> It helps provide an unified entry point for batch and streaming. > >>>> I think we can also use sync execution for some testing. > >>>> So, I agree with you that we provide `executeSql` method and it's > async > >>>> method. > >>>> If we want sync method in the future, we can add method named > >>>> `executeSqlSync`. > >>>> > >>>> I think we've reached an agreement. I will update the document, and > >>>> start > >>>> voting process. > >>>> > >>>> Best, > >>>> Godfrey > >>>> > >>>> > >>>> Jark Wu <imj...@gmail.com> 于2020年3月31日周二 上午12:46写道： > >>>> > >>>>> Hi, > >>>>> > >>>>> I didn't follow the full discussion. > >>>>> But I share the same concern with Timo that streaming queries should > >>>>> always > >>>>> be async. > >>>>> Otherwise, I can image it will cause a lot of confusion and problems > if > >>>>> users don't deeply keep the "sync" in mind (e.g. client hangs). > >>>>> Besides, the streaming mode is still the majority use cases of Flink > >>>>> and > >>>>> Flink SQL. We should put the usability at a high priority. > >>>>> > >>>>> Best, > >>>>> Jark > >>>>> > >>>>> > >>>>> On Mon, 30 Mar 2020 at 23:27, Timo Walther <twal...@apache.org> > wrote: > >>>>> > >>>>>> Hi Godfrey, > >>>>>> > >>>>>> maybe I wasn't expressing my biggest concern enough in my last mail. > >>>>>> Even in a singleline and sync execution, I think that streaming > >>>>>> queries > >>>>>> should not block the execution. Otherwise it is not possible to call > >>>>>> collect() or print() on them afterwards. > >>>>>> > >>>>>> "there are too many things need to discuss for multiline": > >>>>>> > >>>>>> True, I don't want to solve all of them right now. But what I know > is > >>>>>> that our newly introduced methods should fit into a multiline > >>>>>> execution. > >>>>>> There is no big difference of calling `executeSql(A), > >>>>>> executeSql(B)` and > >>>>>> processing a multiline file `A;\nB;`. > >>>>>> > >>>>>> I think the example that you mentioned can simply be undefined for > >>>>>> now. > >>>>>> Currently, no catalog is modifying data but just metadata. This is a > >>>>>> separate discussion. > >>>>>> > >>>>>> "result of the second statement is indeterministic": > >>>>>> > >>>>>> Sure this is indeterministic. But this is the implementers fault > >>>>>> and we > >>>>>> cannot forbid such pipelines. > >>>>>> > >>>>>> How about we always execute streaming queries async? It would > unblock > >>>>>> executeSql() and multiline statements. > >>>>>> > >>>>>> Having a `executeSqlAsync()` is useful for batch. However, I don't > >>>>>> want > >>>>>> `sync/async` be the new batch/stream flag. The execution behavior > >>>>>> should > >>>>>> come from the query itself. > >>>>>> > >>>>>> Regards, > >>>>>> Timo > >>>>>> > >>>>>> > >>>>>> On 30.03.20 11:12, godfrey he wrote: > >>>>>>> Hi Timo, > >>>>>>> > >>>>>>> Agree with you that streaming queries is our top priority, > >>>>>>> but I think there are too many things need to discuss for multiline > >>>>>>> statements: > >>>>>>> e.g. > >>>>>>> 1. what's the behaivor of DDL and DML mixing for async execution: > >>>>>>> create table t1 xxx; > >>>>>>> create table t2 xxx; > >>>>>>> insert into t2 select * from t1 where xxx; > >>>>>>> drop table t1; // t1 may be a MySQL table, the data will also be > >>>>> deleted. > >>>>>>> > >>>>>>> t1 is dropped when "insert" job is running. > >>>>>>> > >>>>>>> 2. what's the behaivor of unified scenario for async execution: > >>>>>>> (as you > >>>>>>> mentioned) > >>>>>>> INSERT INTO t1 SELECT * FROM s; > >>>>>>> INSERT INTO t2 SELECT * FROM s JOIN t1 EMIT STREAM; > >>>>>>> > >>>>>>> The result of the second statement is indeterministic, because the > >>>>> first > >>>>>>> statement maybe is running. > >>>>>>> I think we need to put a lot of effort to define the behavior of > >>>>>> logically > >>>>>>> related queries. > >>>>>>> > >>>>>>> In this FLIP, I suggest we only handle single statement, and we > also > >>>>>>> introduce an async execute method > >>>>>>> which is more important and more often used for users. > >>>>>>> > >>>>>>> Dor the sync methods (like `TableEnvironment.executeSql` and > >>>>>>> `StatementSet.execute`), > >>>>>>> the result will be returned until the job is finished. The > following > >>>>>>> methods will be introduced in this FLIP: > >>>>>>> > >>>>>>> /** > >>>>>>> * Asynchronously execute the given single statement > >>>>>>> */ > >>>>>>> TableEnvironment.executeSqlAsync(String statement): TableResult > >>>>>>> > >>>>>>> /** > >>>>>>> * Asynchronously execute the dml statements as a batch > >>>>>>> */ > >>>>>>> StatementSet.executeAsync(): TableResult > >>>>>>> > >>>>>>> public interface TableResult { > >>>>>>> /** > >>>>>>> * return JobClient for DQL and DML in async mode, else > return > >>>>>>> Optional.empty > >>>>>>> */ > >>>>>>> Optional<JobClient> getJobClient(); > >>>>>>> } > >>>>>>> > >>>>>>> what do you think? > >>>>>>> > >>>>>>> Best, > >>>>>>> Godfrey > >>>>>>> > >>>>>>> Timo Walther <twal...@apache.org> 于2020年3月26日周四 下午9:15写道： > >>>>>>> > >>>>>>>> Hi Godfrey, > >>>>>>>> > >>>>>>>> executing streaming queries must be our top priority because this > is > >>>>>>>> what distinguishes Flink from competitors. If we change the > >>>>>>>> execution > >>>>>>>> behavior, we should think about the other cases as well to not > break > >>>>> the > >>>>>>>> API a third time. > >>>>>>>> > >>>>>>>> I fear that just having an async execute method will not be enough > >>>>>>>> because users should be able to mix streaming and batch queries > in a > >>>>>>>> unified scenario. > >>>>>>>> > >>>>>>>> If I remember it correctly, we had some discussions in the past > >>>>>>>> about > >>>>>>>> what decides about the execution mode of a query. Currently, we > >>>>>>>> would > >>>>>>>> like to let the query decide, not derive it from the sources. > >>>>>>>> > >>>>>>>> So I could image a multiline pipeline as: > >>>>>>>> > >>>>>>>> USE CATALOG 'mycat'; > >>>>>>>> INSERT INTO t1 SELECT * FROM s; > >>>>>>>> INSERT INTO t2 SELECT * FROM s JOIN t1 EMIT STREAM; > >>>>>>>> > >>>>>>>> For executeMultilineSql(): > >>>>>>>> > >>>>>>>> sync because regular SQL > >>>>>>>> sync because regular Batch SQL > >>>>>>>> async because Streaming SQL > >>>>>>>> > >>>>>>>> For executeAsyncMultilineSql(): > >>>>>>>> > >>>>>>>> async because everything should be async > >>>>>>>> async because everything should be async > >>>>>>>> async because everything should be async > >>>>>>>> > >>>>>>>> What we should not start for executeAsyncMultilineSql(): > >>>>>>>> > >>>>>>>> sync because DDL > >>>>>>>> async because everything should be async > >>>>>>>> async because everything should be async > >>>>>>>> > >>>>>>>> What are you thoughts here? > >>>>>>>> > >>>>>>>> Regards, > >>>>>>>> Timo > >>>>>>>> > >>>>>>>> > >>>>>>>> On 26.03.20 12:50, godfrey he wrote: > >>>>>>>>> Hi Timo, > >>>>>>>>> > >>>>>>>>> I agree with you that streaming queries mostly need async > >>>>>>>>> execution. > >>>>>>>>> In fact, our original plan is only introducing sync methods in > this > >>>>>> FLIP, > >>>>>>>>> and async methods (like "executeSqlAsync") will be introduced in > >>>>>>>>> the > >>>>>>>> future > >>>>>>>>> which is mentioned in the appendix. > >>>>>>>>> > >>>>>>>>> Maybe the async methods also need to be considered in this FLIP. > >>>>>>>>> > >>>>>>>>> I think sync methods is also useful for streaming which can be > used > >>>>> to > >>>>>>>> run > >>>>>>>>> bounded source. > >>>>>>>>> Maybe we should check whether all sources are bounded in sync > >>>>> execution > >>>>>>>>> mode. > >>>>>>>>> > >>>>>>>>>> Also, if we block for streaming queries, we could never support > >>>>>>>>>> multiline files. Because the first INSERT INTO would block the > >>>>> further > >>>>>>>>>> execution. > >>>>>>>>> agree with you, we need async method to submit multiline files, > >>>>>>>>> and files should be limited that the DQL and DML should be > >>>>>>>>> always in > >>>>>> the > >>>>>>>>> end for streaming. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Godfrey > >>>>>>>>> > >>>>>>>>> Timo Walther <twal...@apache.org> 于2020年3月26日周四 下午4:29写道： > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Hi Godfrey, > >>>>>>>>>> > >>>>>>>>>> having control over the job after submission is a requirement > that > >>>>> was > >>>>>>>>>> requested frequently (some examples are [1], [2]). Users would > >>>>>>>>>> like > >>>>> to > >>>>>>>>>> get insights about the running or completed job. Including the > >>>>> jobId, > >>>>>>>>>> jobGraph etc., the JobClient summarizes these properties. > >>>>>>>>>> > >>>>>>>>>> It is good to have a discussion about synchronous/asynchronous > >>>>>>>>>> submission now to have a complete execution picture. > >>>>>>>>>> > >>>>>>>>>> I thought we submit streaming queries mostly async and just > >>>>>>>>>> wait for > >>>>>> the > >>>>>>>>>> successful submission. If we block for streaming queries, how > >>>>>>>>>> can we > >>>>>>>>>> collect() or print() results? > >>>>>>>>>> > >>>>>>>>>> Also, if we block for streaming queries, we could never support > >>>>>>>>>> multiline files. Because the first INSERT INTO would block the > >>>>> further > >>>>>>>>>> execution. > >>>>>>>>>> > >>>>>>>>>> If we decide to block entirely on streaming queries, we need the > >>>>> async > >>>>>>>>>> execution methods in the design already. However, I would > >>>>>>>>>> rather go > >>>>>> for > >>>>>>>>>> non-blocking streaming queries. Also with the `EMIT STREAM` key > >>>>>>>>>> word > >>>>>> in > >>>>>>>>>> mind that we might add to SQL statements soon. > >>>>>>>>>> > >>>>>>>>>> Regards, > >>>>>>>>>> Timo > >>>>>>>>>> > >>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-16761 > >>>>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-12214 > >>>>>>>>>> > >>>>>>>>>> On 25.03.20 16:30, godfrey he wrote: > >>>>>>>>>>> Hi Timo, > >>>>>>>>>>> > >>>>>>>>>>> Thanks for the updating. > >>>>>>>>>>> > >>>>>>>>>>> Regarding to "multiline statement support", I'm also fine that > >>>>>>>>>>> `TableEnvironment.executeSql()` only supports single line > >>>>> statement, > >>>>>>>> and > >>>>>>>>>> we > >>>>>>>>>>> can support multiline statement later (needs more discussion > >>>>>>>>>>> about > >>>>>>>> this). > >>>>>>>>>>> > >>>>>>>>>>> Regarding to "StatementSet.explian()", I don't have strong > >>>>>>>>>>> opinions > >>>>>>>> about > >>>>>>>>>>> that. > >>>>>>>>>>> > >>>>>>>>>>> Regarding to "TableResult.getJobClient()", I think it's > >>>>> unnecessary. > >>>>>>>> The > >>>>>>>>>>> reason is: first, many statements (e.g. DDL, show xx, use xx) > >>>>>>>>>>> will > >>>>>> not > >>>>>>>>>>> submit a Flink job. second, `TableEnvironment.executeSql()` and > >>>>>>>>>>> `StatementSet.execute()` are synchronous method, `TableResult` > >>>>>>>>>>> will > >>>>>> be > >>>>>>>>>>> returned only after the job is finished or failed. > >>>>>>>>>>> > >>>>>>>>>>> Regarding to "whether StatementSet.execute() needs to throw > >>>>>>>> exception", I > >>>>>>>>>>> think we should choose a unified way to tell whether the > >>>>>>>>>>> execution > >>>>> is > >>>>>>>>>>> successful. If `TableResult` contains ERROR kind (non-runtime > >>>>>>>> exception), > >>>>>>>>>>> users need to not only check the result but also catch the > >>>>>>>>>>> runtime > >>>>>>>>>>> exception in their code. or `StatementSet.execute()` does not > >>>>>>>>>>> throw > >>>>>> any > >>>>>>>>>>> exception (including runtime exception), all exception > >>>>>>>>>>> messages are > >>>>>> in > >>>>>>>>>> the > >>>>>>>>>>> result. I prefer "StatementSet.execute() needs to throw > >>>>> exception". > >>>>>> cc > >>>>>>>>>> @Jark > >>>>>>>>>>> Wu <imj...@gmail.com> > >>>>>>>>>>> > >>>>>>>>>>> I will update the agreed parts to the document first. > >>>>>>>>>>> > >>>>>>>>>>> Best, > >>>>>>>>>>> Godfrey > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Timo Walther <twal...@apache.org> 于2020年3月25日周三 > >>>>>>>>>>> 下午6:51写道： > >>>>>>>>>>> > >>>>>>>>>>>> Hi Godfrey, > >>>>>>>>>>>> > >>>>>>>>>>>> thanks for starting the discussion on the mailing list. And > >>>>>>>>>>>> sorry > >>>>>>>> again > >>>>>>>>>>>> for the late reply to FLIP-84. I have updated the Google doc > one > >>>>>> more > >>>>>>>>>>>> time to incorporate the offline discussions. > >>>>>>>>>>>> > >>>>>>>>>>>> From Dawid's and my view, it is fine to postpone the > >>>>>>>>>>>> multiline > >>>>>>>> support > >>>>>>>>>>>> to a separate method. This can be future work even though we > >>>>>>>>>>>> will > >>>>>> need > >>>>>>>>>>>> it rather soon. > >>>>>>>>>>>> > >>>>>>>>>>>> If there are no objections, I suggest to update the FLIP-84 > >>>>>>>>>>>> again > >>>>>> and > >>>>>>>>>>>> have another voting process. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Timo > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On 25.03.20 11:17, godfrey he wrote: > >>>>>>>>>>>>> Hi community, > >>>>>>>>>>>>> Timo, Fabian and Dawid have some feedbacks about FLIP-84[1]. > >>>>>>>>>>>>> The > >>>>>>>>>>>> feedbacks > >>>>>>>>>>>>> are all about new introduced methods. We had a discussion > >>>>>> yesterday, > >>>>>>>>>> and > >>>>>>>>>>>>> most of feedbacks have been agreed upon. Here is the > >>>>>>>>>>>>> conclusions: > >>>>>>>>>>>>> > >>>>>>>>>>>>> *1. about proposed methods in `TableEnvironment`:* > >>>>>>>>>>>>> > >>>>>>>>>>>>> the original proposed methods: > >>>>>>>>>>>>> > >>>>>>>>>>>>> TableEnvironment.createDmlBatch(): DmlBatch > >>>>>>>>>>>>> TableEnvironment.executeStatement(String statement): > >>>>>>>>>>>>> ResultTable > >>>>>>>>>>>>> > >>>>>>>>>>>>> the new proposed methods: > >>>>>>>>>>>>> > >>>>>>>>>>>>> // we should not use abbreviations in the API, and the term > >>>>> "Batch" > >>>>>>>> is > >>>>>>>>>>>>> easily confused with batch/streaming processing > >>>>>>>>>>>>> TableEnvironment.createStatementSet(): StatementSet > >>>>>>>>>>>>> > >>>>>>>>>>>>> // every method that takes SQL should have `Sql` in its name > >>>>>>>>>>>>> // supports multiline statement ??? > >>>>>>>>>>>>> TableEnvironment.executeSql(String statement): TableResult > >>>>>>>>>>>>> > >>>>>>>>>>>>> // new methods. supports explaining DQL and DML > >>>>>>>>>>>>> TableEnvironment.explainSql(String statement, > ExplainDetail... > >>>>>>>>>> details): > >>>>>>>>>>>>> String > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> *2. about proposed related classes:* > >>>>>>>>>>>>> > >>>>>>>>>>>>> the original proposed classes: > >>>>>>>>>>>>> > >>>>>>>>>>>>> interface DmlBatch { > >>>>>>>>>>>>> void addInsert(String insert); > >>>>>>>>>>>>> void addInsert(String targetPath, Table table); > >>>>>>>>>>>>> ResultTable execute() throws Exception ; > >>>>>>>>>>>>> String explain(boolean extended); > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> public interface ResultTable { > >>>>>>>>>>>>> TableSchema getResultSchema(); > >>>>>>>>>>>>> Iterable<Row> getResultRows(); > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> the new proposed classes: > >>>>>>>>>>>>> > >>>>>>>>>>>>> interface StatementSet { > >>>>>>>>>>>>> // every method that takes SQL should have `Sql` in > >>>>>>>>>>>>> its > >>>>>> name > >>>>>>>>>>>>> // return StatementSet instance for fluent > programming > >>>>>>>>>>>>> addInsertSql(String statement): StatementSet > >>>>>>>>>>>>> > >>>>>>>>>>>>> // return StatementSet instance for fluent > programming > >>>>>>>>>>>>> addInsert(String tablePath, Table table): > StatementSet > >>>>>>>>>>>>> > >>>>>>>>>>>>> // new method. support overwrite mode > >>>>>>>>>>>>> addInsert(String tablePath, Table table, boolean > >>>>>> overwrite): > >>>>>>>>>>>>> StatementSet > >>>>>>>>>>>>> > >>>>>>>>>>>>> explain(): String > >>>>>>>>>>>>> > >>>>>>>>>>>>> // new method. supports adding more details for the > >>>>> result > >>>>>>>>>>>>> explain(ExplainDetail... extraDetails): String > >>>>>>>>>>>>> > >>>>>>>>>>>>> // throw exception ??? > >>>>>>>>>>>>> execute(): TableResult > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> interface TableResult { > >>>>>>>>>>>>> getTableSchema(): TableSchema > >>>>>>>>>>>>> > >>>>>>>>>>>>> // avoid custom parsing of an "OK" row in > programming > >>>>>>>>>>>>> getResultKind(): ResultKind > >>>>>>>>>>>>> > >>>>>>>>>>>>> // instead of `get` make it explicit that this is > >>>>>>>>>>>>> might > >>>>> be > >>>>>>>>>>>> triggering > >>>>>>>>>>>>> an expensive operation > >>>>>>>>>>>>> collect(): Iterable<Row> > >>>>>>>>>>>>> > >>>>>>>>>>>>> // for fluent programming > >>>>>>>>>>>>> print(): Unit > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> enum ResultKind { > >>>>>>>>>>>>> SUCCESS, // for DDL, DCL and statements with a > simple > >>>>> "OK" > >>>>>>>>>>>>> SUCCESS_WITH_CONTENT, // rows with important > >>>>>>>>>>>>> content are > >>>>>>>>>> available > >>>>>>>>>>>>> (DML, DQL) > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> *3. new proposed methods in `Table`* > >>>>>>>>>>>>> > >>>>>>>>>>>>> `Table.insertInto()` will be deprecated, and the following > >>>>> methods > >>>>>>>> are > >>>>>>>>>>>>> introduced: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Table.executeInsert(String tablePath): TableResult > >>>>>>>>>>>>> Table.executeInsert(String tablePath, boolean overwrite): > >>>>>> TableResult > >>>>>>>>>>>>> Table.explain(ExplainDetail... details): String > >>>>>>>>>>>>> Table.execute(): TableResult > >>>>>>>>>>>>> > >>>>>>>>>>>>> There are two issues need further discussion, one is whether > >>>>>>>>>>>>> `TableEnvironment.executeSql(String statement): TableResult` > >>>>> needs > >>>>>> to > >>>>>>>>>>>>> support multiline statement (or whether `TableEnvironment` > >>>>>>>>>>>>> needs > >>>>> to > >>>>>>>>>>>> support > >>>>>>>>>>>>> multiline statement), and another one is whether > >>>>>>>>>> `StatementSet.execute()` > >>>>>>>>>>>>> needs to throw exception. > >>>>>>>>>>>>> > >>>>>>>>>>>>> please refer to the feedback document [2] for the details. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Any suggestions are warmly welcomed! > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >> > https://wiki.apache.org/confluence/pages/viewpage.action?pageId=134745878 > >>>>> > >>>>>>>>>>>>> [2] > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >> > https://docs.google.com/document/d/1ueLjQWRPdLTFB_TReAyhseAX-1N3j4WYWD0F02Uau0E/edit > >>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Best, > >>>>>>>>>>>>> Godfrey > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >> > > >
