This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f26133374d7 [improvement](stream) List table streams as BASE TABLE,
not VIEW (#70560)
f26133374d7 is described below
commit f26133374d70d9abde85395adabf9d9690adf719
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Nov 19 09:59:15 2026 +0800
[improvement](stream) List table streams as BASE TABLE, not VIEW (#70560)
### Background
Since #61382, a table stream (`CREATE STREAM s ON TABLE t`) is listed
with the tables and views of its database in `SHOW FULL TABLES` and
`information_schema.tables`, with `ENGINE` `Stream`. Incremental refresh
(#62606) creates one for each base table of an incrementally refreshed
materialized view, named `__doris_ivm_stream_<mv id>_<hash>`.
`TableIf.TableType.toMysqlType` gives a stream the table type `VIEW`.
### What problem does this PR solve?
Issue Number: None
Related PR: #61382 (introduced table streams), #62606 (introduced
incremental refresh)
PR Type: Enhancement
Problem Summary: Metadata clients interpret streams listed as `VIEW` as
views and try to parse their `CREATE STREAM ... ON TABLE ...`
definitions; OpenMetadata exposes IVM streams as user views. List
streams as `BASE TABLE`, keeping `ENGINE = 'Stream'` so clients can
distinguish them. Adapt the global hash-bucket checker to select the
`Doris` engine, since streams have no physical buckets. Streams and
incremental refresh exist only on master, so released versions are
unaffected.
### Details
- **Before and after:**
| | Before | After |
|---|---|---|
| `Table_type` of a stream in `SHOW FULL TABLES` | `VIEW` | `BASE TABLE`
|
| `TABLE_TYPE`, `ENGINE` of a stream in `information_schema.tables` |
`VIEW`, `Stream` | `BASE TABLE`, `Stream` |
| the `__doris_ivm_stream_*` streams of an incrementally refreshed MV |
listed, `VIEW` | listed, `BASE TABLE` |
Views keep `VIEW` and system tables `SYSTEM VIEW`.
`information_schema.views`, `SHOW STREAMS`, `SHOW CREATE TABLE` and
`information_schema.table_streams` are unchanged: they select streams
and views by `TableType`, not by this string.
- **Configs and defaults:** None.
- **Impact on existing users:** None on released versions: table streams
and incremental refresh are on neither branch-4.1 nor branch-4.2. On
master, `SHOW FULL TABLES ... WHERE Table_type = 'VIEW'` and JDBC
`getTables` with the type `VIEW` no longer return streams, and the type
`TABLE` now does. Nothing is persisted: the type is computed whenever a
listing is served.
### Testing
- CloudP0 on `6f6109a339b8`: `test_stream_table_type` passed;
`check_hash_bucket_table` failed when `SHOW PARTITIONS` reached an
internal IVM stream. The engine selector added in this follow-up has not
been rerun against a cluster; this worktree has no deployed FE/BE.
- Regression test (new): `query_p0/schema_table/test_stream_table_type`
creates a table, a stream on it and a view, and checks the stream and
the view in `SHOW FULL TABLES`, all three in
`information_schema.tables`, and that `information_schema.views` lists
only the view. The `.out` is generated. On the code before this PR it
fails at its first check: `Expect cell is: BASE TABLE`, `But real is:
VIEW`.
- FE unit test (new): `TableTypeTest` checks that a stream maps to `BASE
TABLE` with the engine `Stream`, and that `VIEW`, `INLINE_VIEW`,
`SCHEMA`, `OLAP` and `MATERIALIZED_VIEW` keep their types.
`./run-fe-ut.sh --run org.apache.doris.catalog.TableTypeTest`: 2 tests,
0 failures.
- Manual: an incrementally refreshed materialized view over a
merge-on-write table with changelog; its `__doris_ivm_stream_*` stream
is listed as `BASE TABLE` / `Stream` in both listings, and as `VIEW` /
`Stream` before this PR.
- Original local checks used this branch's FE jar, or the same base
without the metadata change, with BE `6991e766329` from an earlier
master build.
### Notes for Reviewers
- `check_hash_bucket_table` selects the current database's `Doris`
engine tables from `information_schema.tables`. `BASE TABLE` includes
streams, for which `SHOW PARTITIONS` is invalid. Filtering by engine
covers user and IVM streams; async-MV exclusion stays in place.
- Why `BASE TABLE`: Connector/J maps it to `TABLE` and clients filter by
standard MySQL types; an unfamiliar value such as `STREAM` could drop
streams from listings or break type mapping.
- Why the streams incremental refresh creates stay listed: they are real
objects that `SHOW STREAMS` and `information_schema.table_streams` show
and that can be queried; hiding them by their name prefix in some
listings would make the listings disagree.
- The incremental-refresh case is checked by hand rather than in the
suite: it needs a merge-on-write base table with changelog, and a
materialized view's `ENGINE` in `information_schema.tables` is about to
change (#70523), which the `.out` would then depend on.
---
.../java/org/apache/doris/catalog/TableIf.java | 3 +-
.../org/apache/doris/catalog/TableTypeTest.java | 41 ++++++++++++++++
.../schema_table/test_stream_table_type.out | 15 ++++++
.../check_hash_bucket_table.groovy | 5 +-
.../schema_table/test_stream_table_type.groovy | 55 ++++++++++++++++++++++
5 files changed, 117 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java
index 0baece474e1..89ccbb259f9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableIf.java
@@ -370,8 +370,9 @@ public interface TableIf {
return "SYSTEM VIEW";
case INLINE_VIEW:
case VIEW:
- case STREAM:
return "VIEW";
+ // A stream is read like a table and has no view definition
+ case STREAM:
case OLAP:
case MYSQL:
case ODBC:
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/catalog/TableTypeTest.java
b/fe/fe-core/src/test/java/org/apache/doris/catalog/TableTypeTest.java
new file mode 100644
index 00000000000..9d0e1b3ba91
--- /dev/null
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/TableTypeTest.java
@@ -0,0 +1,41 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.catalog.TableIf.TableType;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+public class TableTypeTest {
+ @Test
+ public void testStreamIsListedAsBaseTable() {
+ Assertions.assertEquals("BASE TABLE", TableType.STREAM.toMysqlType());
+ Assertions.assertEquals("Stream", TableType.STREAM.toEngineName());
+ }
+
+ @Test
+ public void testViewsAndTablesKeepTheirTypes() {
+ Assertions.assertEquals("VIEW", TableType.VIEW.toMysqlType());
+ Assertions.assertEquals("VIEW", TableType.INLINE_VIEW.toMysqlType());
+ Assertions.assertEquals("SYSTEM VIEW", TableType.SCHEMA.toMysqlType());
+ Assertions.assertEquals("BASE TABLE", TableType.OLAP.toMysqlType());
+ Assertions.assertEquals("BASE TABLE",
TableType.MATERIALIZED_VIEW.toMysqlType());
+ }
+}
diff --git
a/regression-test/data/query_p0/schema_table/test_stream_table_type.out
b/regression-test/data/query_p0/schema_table/test_stream_table_type.out
new file mode 100644
index 00000000000..cb963b83913
--- /dev/null
+++ b/regression-test/data/query_p0/schema_table/test_stream_table_type.out
@@ -0,0 +1,15 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !show_full_tables_stream --
+user_stream BASE TABLE NONE NONE
+
+-- !show_full_tables_view --
+base_view VIEW NONE NONE
+
+-- !information_schema_tables --
+base_view VIEW View
+stream_base BASE TABLE Doris
+user_stream BASE TABLE Stream
+
+-- !information_schema_views --
+base_view
+
diff --git
a/regression-test/suites/check_hash_bucket_table/check_hash_bucket_table.groovy
b/regression-test/suites/check_hash_bucket_table/check_hash_bucket_table.groovy
index 3fe6713f66b..5617c97d69f 100644
---
a/regression-test/suites/check_hash_bucket_table/check_hash_bucket_table.groovy
+++
b/regression-test/suites/check_hash_bucket_table/check_hash_bucket_table.groovy
@@ -90,7 +90,10 @@ suite("check_hash_bucket_table") {
def checkDb = { String db ->
sql "use `${db}`;"
dbNum.incrementAndGet()
- def tables = sql("show full tables").stream().filter{ it[1] == "BASE
TABLE" }.collect{ it[0] }.toList()
+ def tables = sql("""
+ SELECT TABLE_NAME FROM information_schema.tables
+ WHERE TABLE_SCHEMA = DATABASE() AND ENGINE = 'Doris'
+ """).collect { it[0] }
def asyncMVs = sql_return_maparray("""select * from
mv_infos("database"="${db}");""").collect{ it.Name }.toSet()
tables.each {
def tblName = it
diff --git
a/regression-test/suites/query_p0/schema_table/test_stream_table_type.groovy
b/regression-test/suites/query_p0/schema_table/test_stream_table_type.groovy
new file mode 100644
index 00000000000..59c30425d76
--- /dev/null
+++ b/regression-test/suites/query_p0/schema_table/test_stream_table_type.groovy
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+// A table stream is listed as a BASE TABLE whose engine is Stream; a view
stays a VIEW.
+suite("test_stream_table_type") {
+ sql "DROP DATABASE IF EXISTS test_stream_table_type_db FORCE"
+ sql "CREATE DATABASE test_stream_table_type_db"
+ sql "USE test_stream_table_type_db"
+
+ sql """
+ CREATE TABLE `stream_base` (
+ `sid` int NULL,
+ `sname` varchar(32) NULL
+ ) ENGINE=OLAP
+ DUPLICATE KEY(`sid`)
+ DISTRIBUTED BY HASH(`sid`) BUCKETS 1
+ PROPERTIES (
+ "replication_allocation" = "tag.location.default: 1",
+ "changelog.enable" = "true"
+ );
+ """
+ sql """
+ CREATE STREAM `user_stream` ON TABLE stream_base
+ COMMENT 'user stream'
+ PROPERTIES('type' = 'append_only');
+ """
+ sql "CREATE VIEW `base_view` AS SELECT `sid`, `sname` FROM stream_base"
+
+ qt_show_full_tables_stream "SHOW FULL TABLES FROM
test_stream_table_type_db LIKE 'user_stream'"
+ qt_show_full_tables_view "SHOW FULL TABLES FROM test_stream_table_type_db
LIKE 'base_view'"
+ order_qt_information_schema_tables """
+ SELECT TABLE_NAME, TABLE_TYPE, ENGINE
+ FROM information_schema.tables
+ WHERE TABLE_SCHEMA = 'test_stream_table_type_db'
+ """
+ // Only the view is a view
+ order_qt_information_schema_views """
+ SELECT TABLE_NAME FROM information_schema.views WHERE TABLE_SCHEMA =
'test_stream_table_type_db'
+ """
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]