Gabriel39 commented on code in PR #68873:
URL: https://github.com/apache/doris/pull/68873#discussion_r4239762085
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/source/JdbcScanNode.java:
##########
@@ -213,13 +219,183 @@ private String getTvfQuery() {
if (!projectsTimestamps) {
return query;
}
- String source = query.trim();
- if (source.endsWith(";")) {
- source = source.substring(0, source.length() - 1);
+ boolean mysql = jdbcType == TOdbcTableType.MYSQL || jdbcType ==
TOdbcTableType.OCEANBASE;
+ if (mysql && mysqlSqlMode == null) {
+ // Resolve only for wrapped MySQL queries and reuse the result for
EXPLAIN and scan serialization.
+ org.apache.doris.datasource.jdbc.JdbcExternalCatalog catalog =
+ (org.apache.doris.datasource.jdbc.JdbcExternalCatalog)
org.apache.doris.catalog.Env
+
.getCurrentEnv().getCatalogMgr().getCatalog(catalogId);
+ mysqlSqlMode = catalog.getJdbcClient().getSessionSqlMode();
Review Comment:
Fixed in 02baebff2a. The TVF scan now reads SQL mode during setup from its
copied JDBC URL, credentials and driver settings using a private,
single-connection client, which is closed on both success and failure. EXPLAIN
and Thrift serialization use the captured mode without looking up the catalog
or borrowing its pool. The regression test verifies serialization after catalog
access becomes unavailable, connection configuration capture, and client
cleanup on a failed mode query. All 76 focused FE tests pass; the snapshot mode
lookup and generated queries also passed real MariaDB 10.4 execution.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/source/JdbcScanNode.java:
##########
@@ -213,13 +219,183 @@ private String getTvfQuery() {
if (!projectsTimestamps) {
return query;
}
- String source = query.trim();
- if (source.endsWith(";")) {
- source = source.substring(0, source.length() - 1);
+ boolean mysql = jdbcType == TOdbcTableType.MYSQL || jdbcType ==
TOdbcTableType.OCEANBASE;
+ if (mysql && mysqlSqlMode == null) {
+ // Resolve only for wrapped MySQL queries and reuse the result for
EXPLAIN and scan serialization.
+ org.apache.doris.datasource.jdbc.JdbcExternalCatalog catalog =
+ (org.apache.doris.datasource.jdbc.JdbcExternalCatalog)
org.apache.doris.catalog.Env
+
.getCurrentEnv().getCatalogMgr().getCatalog(catalogId);
+ mysqlSqlMode = catalog.getJdbcClient().getSessionSqlMode();
+ }
+ String source = stripTerminalDelimiter(query.trim(),
+ mysql && mysqlSqlMode.contains("NO_BACKSLASH_ESCAPES"), mysql
&& mysqlSqlMode.contains("ANSI_QUOTES"));
+ // WITH SESSION must stay outside the derived table; terminate
trailing line comments with a newline.
+ int start = jdbcType == TOdbcTableType.TRINO ?
trinoSessionQueryStart(source) : 0;
+ return source.substring(0, start) + "SELECT " + Joiner.on(",
").join(columns)
+ + " FROM (" + source.substring(start) + "\n)
doris_jdbc_source";
+ }
+
+ private String stripTerminalDelimiter(String sql, boolean
noBackslashEscapes, boolean ansiQuotes) {
+ boolean mysql = jdbcType == TOdbcTableType.MYSQL || jdbcType ==
TOdbcTableType.OCEANBASE;
+ List<Integer> delimiters = new java.util.ArrayList<>();
+ for (int i = 0; i < sql.length();) {
+ char c = sql.charAt(i);
+ if (Character.isWhitespace(c)) {
+ i++;
+ } else if (jdbcType == TOdbcTableType.CLICKHOUSE
+ && (sql.startsWith("//", i) || sql.startsWith("#!", i) ||
sql.startsWith("# ", i))) {
+ // ClickHouse comments may contain unmatched quotes; their
contents end only at LF.
+ int newline = sql.indexOf('\n', i + 2);
+ i = newline < 0 ? sql.length() : newline + 1;
+ // MySQL's second dash needs a following whitespace/control
character; --1 is arithmetic.
+ } else if ((sql.startsWith("--", i) && (!mysql || (i + 2 <
sql.length()
+ && (Character.isWhitespace(sql.charAt(i + 2)) ||
Character.isISOControl(sql.charAt(i + 2))))))
+ || (c == '#'
+ && (jdbcType == TOdbcTableType.MYSQL || jdbcType ==
TOdbcTableType.OCEANBASE))) {
+ while (i < sql.length() && sql.charAt(i) != '\n' &&
sql.charAt(i) != '\r') {
+ i++;
+ }
+ } else if (sql.startsWith("/*", i)) {
+ i = skipBlockComment(sql, i, jdbcType ==
TOdbcTableType.CLICKHOUSE);
+ } else if (jdbcType == TOdbcTableType.CLICKHOUSE && c == '$') {
+ i = skipClickHouseHeredoc(sql, i);
+ delimiters.clear();
+ } else if (c == '\'' || c == '"' || c == '`') {
Review Comment:
Fixed in 02baebff2a. With MSSQL mode enabled, the wrapper recognizes
bracket-quoted identifiers, doubled closing brackets, and literal backslashes,
preserving their contents while removing only terminal delimiters. Added
regression cases for an embedded apostrophe, escaped closing bracket with
comment/delimiter characters, and a trailing backslash. All three generated
queries passed timestamp readback on MariaDB 10.4. The FE tests and Checkstyle
pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
