luoyuxia commented on code in PR #23109: URL: https://github.com/apache/flink/pull/23109#discussion_r1299508111
########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. Review Comment: ```suggestion `Time travel` is a SQL syntax used for querying historical data. It allows users to specify a point in time and query the corresponding table data. ``` ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + Review Comment: Please add a desc for the time travel syntax just like other query pages. ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +## Expression Description + +<span class="label label-danger">Attention</span> `TIME TRAVEL` currently only supports some constant expressions and does not support the use of functions or udf. + +### Constant Expression + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +### Constant Expression Addition and Subtraction + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' - INTERVAL '1' DAY +``` + +### Time Function or UDF (Not Supported) + +When using UDF or functions, a valid timestamp cannot be generated due to limitations of the current framework, and an exception will be thrown when executing the following query. + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TO_TIMESTAMP_LTZ(0, 3) +``` + +## Time Zone Handling + +By default, the data type generated by the TIMESTAMP expression should be TIMESTAMP type, while the +interface provided by Catalog is `getTable(ObjectPath tablePath, long timestamp)`. +When executing the time travel statement, the framework will convert TIMESTAMP type time to LONG value based on the local time zone. Therefore, the results of the same time travel query statement may vary when queried in different time zones. Review Comment: ```suggestion When executing the time travel statement, the framework will convert the TIMESTAMP type to the LONG type based on the local time zone. Therefore, the results of the same time travel query statement may vary when queried in different time zones. ``` ########## docs/content/docs/dev/table/catalogs.md: ########## @@ -71,6 +71,29 @@ The provided factory identifier will be used for matching against the required ` User-defined catalogs should replace `Thread.currentThread().getContextClassLoader()` with the user class loader to load classes. Otherwise, `ClassNotFoundException` maybe thrown. The user class loader can be accessed via `CatalogFactory.Context#getClassLoader`. {{< /hint >}} +### Catalog supports time travel + +Starting from version 1.18, the Flink framework supports [time travel]({{< ref "docs/dev/table/sql/queries/time-travel" >}}). You can use time travel by implementing `getTable(ObjectPath tablePath, long timestamp)` as shown below. + +```java +public class MyCatalogSupportTimeTravel implements Catalog { + + @Override + public CatalogBaseTable getTable(ObjectPath tablePath, long timestamp) + throws TableNotExistException { + // Build a schema corresponding to the specific time point. + Schema schema = buildSchema(timestamp); + // Set parameters to read data at the corresponding time point. + Map<String, String> options = buildOptions(timestamp); + // Build CatalogTable + CatalogTable catalogTable = Review Comment: Wouldn't it be a good timing to show the use of method `getSnapshot` in here? ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +## Expression Description + +<span class="label label-danger">Attention</span> `TIME TRAVEL` currently only supports some constant expressions and does not support the use of functions or udf. Review Comment: Also, I don't think it does not support all the use of functions or udf. From the test of `ExpressionReductionRulesTest`, I believe it can reduce some simple expression even with udf. ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 Review Comment: ```suggestion <span class="label label-danger">Attention</span> Currently, `time travel` requires the corresponding catalog that the table belongs to implements the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method. ``` ########## docs/content/docs/dev/table/catalogs.md: ########## @@ -71,6 +71,29 @@ The provided factory identifier will be used for matching against the required ` User-defined catalogs should replace `Thread.currentThread().getContextClassLoader()` with the user class loader to load classes. Otherwise, `ClassNotFoundException` maybe thrown. The user class loader can be accessed via `CatalogFactory.Context#getClassLoader`. {{< /hint >}} +### Catalog supports time travel + +Starting from version 1.18, the Flink framework supports [time travel]({{< ref "docs/dev/table/sql/queries/time-travel" >}}). You can use time travel by implementing `getTable(ObjectPath tablePath, long timestamp)` as shown below. Review Comment: ```suggestion Starting from version 1.18, the Flink framework supports [time travel]({{< ref "docs/dev/table/sql/queries/time-travel" >}}). To use time travel, you should make sure the catalog implement `getTable(ObjectPath tablePath, long timestamp)` method. ``` ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +## Expression Description Review Comment: `Expression Description` confuses me. Just put it as `Limitation`? ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +## Expression Description + +<span class="label label-danger">Attention</span> `TIME TRAVEL` currently only supports some constant expressions and does not support the use of functions or udf. Review Comment: ```suggestion <span class="label label-danger">Attention</span> `Time travel` currently only supports some constant expressions and does not support the use of functions or udf. ``` ########## docs/content/docs/dev/table/sql/queries/time-travel.md: ########## @@ -0,0 +1,65 @@ +--- +title: Time Travel +weight: 18 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Time Travel + +{{< label Batch >}} {{< label Streaming >}} + +`TIME TRAVEL` is an SQL syntax used for querying historical data. It allows users to specify a point in time, query the corresponding table data. + +<span class="label label-danger">Attention</span> Currently, `TIME TRAVEL` requires corresponding catalog that the table belongs to implement the {{< gh_link file="flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java" name="getTable(ObjectPath tablePath, long timestamp)" >}} method。 + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +## Expression Description + +<span class="label label-danger">Attention</span> `TIME TRAVEL` currently only supports some constant expressions and does not support the use of functions or udf. + +### Constant Expression + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' +``` + +### Constant Expression Addition and Subtraction + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-07-31 00:00:00' - INTERVAL '1' DAY +``` + +### Time Function or UDF (Not Supported) + +When using UDF or functions, a valid timestamp cannot be generated due to limitations of the current framework, and an exception will be thrown when executing the following query. + +```sql +SELECT select_list FROM paimon_tb FOR SYSTEM_TIME AS OF TO_TIMESTAMP_LTZ(0, 3) +``` + +## Time Zone Handling + +By default, the data type generated by the TIMESTAMP expression should be TIMESTAMP type, while the Review Comment: ```suggestion The data type generated by the TIMESTAMP expression is TIMESTAMP type, but there's a special case in the time travel statement. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
