This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new b27122d [Docs] add rpc function document (#7984)
b27122d is described below
commit b27122df8bedd25e3d3b8d5622b74e0d70093eeb
Author: Stalary <452024...@qq.com>
AuthorDate: Sun Feb 20 23:26:52 2022 +0800
[Docs] add rpc function document (#7984)
* add rpc function document
---
docs/.vuepress/sidebar/en.js | 3 +-
docs/.vuepress/sidebar/zh-CN.js | 3 +-
...function.md => native-user-defined-function.md} | 4 +-
.../udf/remote-user-defined-function.md | 106 ++++++++++
...function.md => native-user-defined-function.md} | 2 +-
.../udf/remote-user-defined-function.md | 108 ++++++++++
samples/doris-demo/pom.xml | 1 +
samples/doris-demo/udf-demo/pom.xml | 113 +++++++++++
.../java/org/apache/doris/udf/Application.java | 36 ++++
.../java/org/apache/doris/udf/FunctionGrpc.java | 102 ++++++++++
.../udf-demo/src/main/proto/function_service.proto | 63 ++++++
.../doris-demo/udf-demo/src/main/proto/types.proto | 217 +++++++++++++++++++++
12 files changed, 753 insertions(+), 5 deletions(-)
diff --git a/docs/.vuepress/sidebar/en.js b/docs/.vuepress/sidebar/en.js
index c8c1f48..5f314bc 100644
--- a/docs/.vuepress/sidebar/en.js
+++ b/docs/.vuepress/sidebar/en.js
@@ -255,7 +255,8 @@ module.exports = [
directoryPath: "udf/",
children: [
"contribute-udf",
- "user-defined-function",
+ "native-user-defined-function",
+ "remote-user-defined-function",
],
},
],
diff --git a/docs/.vuepress/sidebar/zh-CN.js b/docs/.vuepress/sidebar/zh-CN.js
index 0a57884..4c63d4c 100644
--- a/docs/.vuepress/sidebar/zh-CN.js
+++ b/docs/.vuepress/sidebar/zh-CN.js
@@ -256,7 +256,8 @@ module.exports = [
directoryPath: "udf/",
children: [
"contribute-udf",
- "user-defined-function",
+ "native-user-defined-function",
+ "remote-user-defined-function",
],
},
],
diff --git a/docs/en/extending-doris/udf/user-defined-function.md
b/docs/en/extending-doris/udf/native-user-defined-function.md
similarity index 99%
rename from docs/en/extending-doris/udf/user-defined-function.md
rename to docs/en/extending-doris/udf/native-user-defined-function.md
index f1ebcce..32311bf 100644
--- a/docs/en/extending-doris/udf/user-defined-function.md
+++ b/docs/en/extending-doris/udf/native-user-defined-function.md
@@ -1,6 +1,6 @@
---
{
- "title": "User Defined Function",
+ "title": "Native User Defined Function",
"language": "en"
}
---
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-# User Define Function
+# Native User Defined Function
UDF is mainly suitable for scenarios where the analytical capabilities that
users need do not possess. Users can implement custom functions according to
their own needs, and register with Doris through the UDF framework to expand
Doris' capabilities and solve user analysis needs.
There are two types of analysis requirements that UDF can meet: UDF and UDAF.
UDF in this article refers to both.
diff --git a/docs/en/extending-doris/udf/remote-user-defined-function.md
b/docs/en/extending-doris/udf/remote-user-defined-function.md
new file mode 100644
index 0000000..2a59f58
--- /dev/null
+++ b/docs/en/extending-doris/udf/remote-user-defined-function.md
@@ -0,0 +1,106 @@
+---
+{
+ "title": "Remote User Defined Function Service",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# User Defined Function Rpc
+
+Remote UDF Service The Remote UDF Service can be accessed through RPC to
implement the execution of user-defined functions. Compared with Native UDF
implementations, Remote UDF Service has the following advantages and
limitations:
+1. The advantage
+ * Cross-language: UDF services can be written in all languages supported by
Protobuf.
+ * Security: UDF execution failure or crash only affects the UDF Service and
does not cause the Doris process to crash.
+ * Flexibility: Any other Service or library class can be invoked within a
UDF Service to meet a wider variety of business requirements.
+
+2. Restrictions on use
+ * Performance: Compared to Native UDFs, UDF services incur extra network
overhead and thus have much lower performance than Native UDFs. At the same
time, the implementation of the UDF Service also affects the execution
efficiency of the function. Users need to deal with problems such as high
concurrency and thread safety by themselves.
+ * Single line mode and batch mode: Doris's original query execution
framework based on row memory would execute one UDF RPC call for each row of
data, so the execution efficiency was very poor. However, under the new
vectorization execution framework, one UDF RPC call would be executed for each
batch of data (2048 rows by default), so the performance was significantly
improved. In actual tests, the performance of Remote UDF based on vectorization
and batch processing is similar to tha [...]
+
+## Write UDF functions
+
+This section describes how to develop a Remote RPC Service. Samples for the
Java version are provided under `samples/doris-demo/udf-demo/` for your
reference.
+
+### Copy the proto file
+
+Copy gensrc/proto/function_service.proto and gensrc/proto/types.proto to Rpc
service
+
+- function_service.proto
+ - PFunctionCallRequest
+ - function_name：The function name, corresponding to the symbol specified
when the function was created
+ - args：The parameters passed by the method
+ - context：Querying context Information
+ - PFunctionCallResponse
+ - result：Return result
+ - status：Return Status, 0 indicates normal
+ - PCheckFunctionRequest
+ - function：Function related information
+ - match_type：Matching type
+ - PCheckFunctionResponse
+ - status：Return status, 0 indicates normal
+
+### Generated interface
+
+Use protoc generate code, and specific parameters are viewed using protoc -h
+
+### Implementing an interface
+
+The following three methods need to be implemented
+- fnCall：Used to write computational logic
+- checkFn：Used to verify function names, parameters, and return values when
creating UDFs
+- handShake：Used for interface probe
+
+## Create UDF
+
+Currently, UDAF and UDTF are not supported
+
+```sql
+CREATE FUNCTION
+name ([,...])
+[RETURNS] rettype
+PROPERTIES (["key"="value"][,...])
+```
+Instructions:
+
+1. PROPERTIES中`symbol`Represents the name of the method passed by the RPC
call, which must be set。
+2. PROPERTIES中`object_file`Represents the RPC service address. Currently, a
single address and a cluster address in BRPC-compatible format are supported.
Refer to the cluster connection mode[Format
specification](https://github.com/apache/incubator-brpc/blob/master/docs/cn/client.md#%E8%BF%9E%E6%8E%A5%E6%9C%8D%E5%8A%A1%E9%9B%86%E7%BE%A4)。
+3. PROPERTIES中`type`Indicates the UDF call type, which is Native by default.
Rpc is transmitted when Rpc UDF is used。
+4. name: A function belongs to a DB and name is of the
form`dbName`.`funcName`. When `dbName` is not explicitly specified, the db of
the current session is used`dbName`。
+
+Sample:
+```sql
+CREATE FUNCTION rpc_add(INT, INT) RETURNS INT PROPERTIES (
+ "SYMBOL"="add_int",
+ "OBJECT_FILE"="127.0.0.1:9090",
+ "TYPE"="RPC"
+);
+```
+
+## Use UDF
+
+Users must have the `SELECT` permission of the corresponding database to use
UDF/UDAF.
+
+The use of UDF is consistent with ordinary function methods. The only
difference is that the scope of built-in functions is global, and the scope of
UDF is internal to DB. When the link session is inside the data, directly using
the UDF name will find the corresponding UDF inside the current DB. Otherwise,
the user needs to display the specified UDF database name, such as
`dbName`.`funcName`.
+
+## Delete UDF
+
+When you no longer need UDF functions, you can delete a UDF function by the
following command, you can refer to `DROP FUNCTION`.
diff --git a/docs/zh-CN/extending-doris/udf/user-defined-function.md
b/docs/zh-CN/extending-doris/udf/native-user-defined-function.md
similarity index 99%
rename from docs/zh-CN/extending-doris/udf/user-defined-function.md
rename to docs/zh-CN/extending-doris/udf/native-user-defined-function.md
index d4c8c7b..9f32b00 100644
--- a/docs/zh-CN/extending-doris/udf/user-defined-function.md
+++ b/docs/zh-CN/extending-doris/udf/native-user-defined-function.md
@@ -1,6 +1,6 @@
---
{
- "title": "User Defined Function",
+ "title": "Native User Defined Function",
"language": "zh-CN"
}
---
diff --git a/docs/zh-CN/extending-doris/udf/remote-user-defined-function.md
b/docs/zh-CN/extending-doris/udf/remote-user-defined-function.md
new file mode 100644
index 0000000..8c471c8
--- /dev/null
+++ b/docs/zh-CN/extending-doris/udf/remote-user-defined-function.md
@@ -0,0 +1,108 @@
+---
+{
+ "title": "Remote User Defined Function Service",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Remote User Defined Function Service
+
+Remote UDF Service 支持通过 RPC 的方式访问用户提供的 UDF Service，以实现用户自定义函数的执行。相比于 Native 的
UDF 实现，Remote UDF Service 有如下优势和限制：
+1. 优势
+ * 跨语言：可以用 Protobuf 支持的各类语言编写 UDF Service。
+ * 安全：UDF 执行失败或崩溃，仅会影响 UDF Service 自身，而不会导致 Doris 进程崩溃。
+ * 灵活：UDF Service 中可以调用任意其他服务或程序库类，以满足更多样的业务需求。
+
+2. 使用限制
+ * 性能：相比于 Native UDF，UDF Service 会带来额外的网络开销，因此性能会远低于 Native UDF。同时，UDF
Service 自身的实现也会影响函数的执行效率，用户需要自行处理高并发、线程安全等问题。
+ * 单行模式和批处理模式：Doris 原先的的基于行存的查询执行框架会对每一行数据执行一次 UDF RPC
调用，因此执行效率非常差，而在新的向量化执行框架下，会对每一批数据（默认2048行）执行一次 UDF RPC
调用，因此性能有明显提升。实际测试中，基于向量化和批处理方式的 Remote UDF 性能和基于行存的 Native UDF 性能相当，可供参考。
+
+## 编写 UDF 函数
+
+
+本小节主要介绍如何开发一个 Remote RPC service。在 `samples/doris-demo/udf-demo/` 下提供了 Java
版本的示例，可供参考。
+
+### 拷贝 proto 文件
+
+拷贝 gensrc/proto/function_service.proto 和 gensrc/proto/types.proto 到 Rpc 服务中
+
+- function_service.proto
+ - PFunctionCallRequest
+ - function_name：函数名称，对应创建函数时指定的symbol
+ - args：方法传递的参数
+ - context：查询上下文信息
+ - PFunctionCallResponse
+ - result：结果
+ - status：状态，0代表正常
+ - PCheckFunctionRequest
+ - function：函数相关信息
+ - match_type：匹配类型
+ - PCheckFunctionResponse
+ - status：状态，0代表正常
+
+### 生成接口
+
+通过 protoc 生成代码，具体参数通过 protoc -h 查看
+
+### 实现接口
+
+共需要实现以下三个方法
+- fnCall：用于编写计算逻辑
+- checkFn：用于创建 UDF 时校验，校验函数名/参数/返回值等是否合法
+- handShake：用于接口探活
+
+## 创建 UDF
+
+目前暂不支持 UDAF 和 UDTF
+
+```sql
+CREATE FUNCTION
+name ([,...])
+[RETURNS] rettype
+PROPERTIES (["key"="value"][,...])
+```
+说明：
+
+1. PROPERTIES中`symbol`表示的是 rpc 调用传递的方法名，这个参数是必须设定的。
+2. PROPERTIES中`object_file`表示的 rpc 服务地址，目前支持单个地址和 brpc 兼容格式的集群地址，集群连接方式 参考
[格式说明](https://github.com/apache/incubator-brpc/blob/master/docs/cn/client.md#%E8%BF%9E%E6%8E%A5%E6%9C%8D%E5%8A%A1%E9%9B%86%E7%BE%A4)。
+3. PROPERTIES中`type`表示的 UDF 调用类型，默认为 Native，使用 Rpc UDF时传 RPC。
+4. name:
一个function是要归属于某个DB的，name的形式为`dbName`.`funcName`。当`dbName`没有明确指定的时候，就是使用当前session所在的db作为`dbName`。
+
+示例：
+```sql
+CREATE FUNCTION rpc_add(INT, INT) RETURNS INT PROPERTIES (
+ "SYMBOL"="add_int",
+ "OBJECT_FILE"="127.0.0.1:9090",
+ "TYPE"="RPC"
+);
+```
+
+## 使用 UDF
+
+用户使用 UDF 必须拥有对应数据库的 `SELECT` 权限。
+
+UDF 的使用与普通的函数方式一致，唯一的区别在于，内置函数的作用域是全局的，而 UDF 的作用域是 DB内部。当链接 session
位于数据内部时，直接使用 UDF 名字会在当前DB内部查找对应的 UDF。否则用户需要显示的指定 UDF 的数据库名字，例如
`dbName`.`funcName`。
+
+## 删除 UDF
+
+当你不再需要 UDF 函数时，你可以通过下述命令来删除一个 UDF 函数, 可以参考 `DROP FUNCTION`。
+
diff --git a/samples/doris-demo/pom.xml b/samples/doris-demo/pom.xml
index 3b19663..03dd013 100644
--- a/samples/doris-demo/pom.xml
+++ b/samples/doris-demo/pom.xml
@@ -29,6 +29,7 @@ under the License.
<module>spark-demo</module>
<module>stream-load-demo</module>
<module>spring-jdbc-demo</module>
+ <module>udf-demo</module>
</modules>
<name>doris-demo</name>
<url>http://www.example.com</url>
diff --git a/samples/doris-demo/udf-demo/pom.xml
b/samples/doris-demo/udf-demo/pom.xml
new file mode 100644
index 0000000..be78f81
--- /dev/null
+++ b/samples/doris-demo/udf-demo/pom.xml
@@ -0,0 +1,113 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns="http://maven.apache.org/POM/4.0.0";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>org.apache.doris</groupId>
+ <artifactId>udf-demo</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ <name>udf-demo</name>
+ <!-- FIXME change it to the project's website -->
+ <url>http://www.example.com</url>
+ <properties>
+ <protobuf.version>3.14.0</protobuf.version>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
+ <grpc.version>1.30.0</grpc.version>
+ <java.version>1.8</java.version>
+ <doris.thirdparty>${basedir}/../../../thirdparty</doris.thirdparty>
+ </properties>
+ <dependencies>
+ <!-- grpc -->
+ <dependency>
+ <groupId>net.devh</groupId>
+ <artifactId>grpc-server-spring-boot-starter</artifactId>
+ <version>2.9.0.RELEASE</version>
+ </dependency>
+ <dependency>
+ <groupId>org.projectlombok</groupId>
+ <artifactId>lombok</artifactId>
+ <version>1.18.22</version>
+ <scope>provided</scope>
+ </dependency>
+
+
+ </dependencies>
+
+ <build>
+ <finalName>udf-demo</finalName>
+ <plugins>
+ <!-- protobuf -->
+ <plugin>
+ <groupId>com.github.os72</groupId>
+ <artifactId>protoc-jar-maven-plugin</artifactId>
+ <version>3.11.1</version>
+ <executions>
+ <execution>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ <configuration>
+
<protocCommand>${doris.thirdparty}/installed/bin/protoc</protocCommand>
+ <!-->You can use following protocArtifact instead
of protocCommand, so that you don't need to install protobuf tools<-->
+
<!--protocArtifact>com.google.protobuf:protoc:${protobuf.version}</protocArtifact-->
+ <protocVersion>${protobuf.version}</protocVersion>
+ <inputDirectories>
+ <include>../udf-demo/src/main/proto</include>
+ </inputDirectories>
+ <outputTargets>
+ <outputTarget>
+ <type>java</type>
+ </outputTarget>
+ <outputTarget>
+ <type>grpc-java</type>
+
<pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}</pluginArtifact>
+ </outputTarget>
+ </outputTargets>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <!-- add gensrc java build src dir -->
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>3.2.0</version>
+ <executions>
+ <execution>
+ <id>add-source</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <!-- add arbitrary num of src dirs here -->
+
<source>${basedir}/target/generated-sources/build/</source>
+
<source>${basedir}/target/generated-sources/</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git
a/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/Application.java
b/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/Application.java
new file mode 100644
index 0000000..de4b81a
--- /dev/null
+++
b/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/Application.java
@@ -0,0 +1,36 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.udf;
+
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+
+/**
+ * Application
+ *
+ * @author lirongqian
+ * @since 2021/06/02
+ */
+@SpringBootApplication
+public class Application {
+
+ public static void main(String[] args) {
+ SpringApplication.run(Application.class, args);
+ }
+
+}
diff --git
a/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/FunctionGrpc.java
b/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/FunctionGrpc.java
new file mode 100644
index 0000000..60910e1
--- /dev/null
+++
b/samples/doris-demo/udf-demo/src/main/java/org/apache/doris/udf/FunctionGrpc.java
@@ -0,0 +1,102 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.udf;
+
+import org.apache.doris.proto.FunctionService;
+import org.apache.doris.proto.PFunctionServiceGrpc;
+import org.apache.doris.proto.Types;
+
+import java.util.List;
+
+import io.grpc.stub.StreamObserver;
+import lombok.extern.slf4j.Slf4j;
+import net.devh.boot.grpc.server.service.GrpcService;
+
+/**
+ * FunctionGrpc
+ *
+ * @author lirongqian
+ * @since 2022/02/08
+ */
+@GrpcService
+@Slf4j
+public class FunctionGrpc extends
PFunctionServiceGrpc.PFunctionServiceImplBase {
+
+
+ @Override
+ public void fnCall(FunctionService.PFunctionCallRequest request,
StreamObserver<FunctionService.PFunctionCallResponse> responseObserver) {
+ // symbol is functionName
+ String functionName = request.getFunctionName();
+ log.info("fnCall request={}", request);
+ FunctionService.PFunctionCallResponse res;
+ if ("add_int".equals(functionName)) {
+ // get args
+ List<Types.PValues> argsList = request.getArgsList();
+ // calculate logic
+ int sum = 0;
+ for (Types.PValues pValues : argsList) {
+ sum += pValues.getInt32Value(0);
+ }
+ res = FunctionService.PFunctionCallResponse.newBuilder()
+ .setStatus(Types.PStatus.newBuilder()
+ .setStatusCode(0)
+ .build())
+ .setResult(Types.PValues.newBuilder()
+ .setHasNull(false)
+ .addInt32Value(sum)
+ .setType(Types.PGenericType.newBuilder()
+
.setId(Types.PGenericType.TypeId.INT32).build()).build()).build();
+ } else {
+ res = FunctionService.PFunctionCallResponse.newBuilder()
+ .setStatus(Types.PStatus.newBuilder()
+ .setStatusCode(0)
+ .build()).build();
+ }
+ log.info("fnCall res={}", res);
+ ok(responseObserver, res);
+ }
+
+ @Override
+ public void checkFn(FunctionService.PCheckFunctionRequest request,
StreamObserver<FunctionService.PCheckFunctionResponse> responseObserver) {
+ // symbol is functionName
+ log.info("checkFn request={}", request);
+ int status = 0;
+ if ("add_int".equals(request.getFunction().getFunctionName())) {
+ // check inputs count
+ if (request.getFunction().getInputsCount() != 2) {
+ status = -1;
+ }
+ }
+ FunctionService.PCheckFunctionResponse res =
FunctionService.PCheckFunctionResponse.newBuilder()
+
.setStatus(Types.PStatus.newBuilder().setStatusCode(status).build()).build();
+ log.info("checkFn res={}", res);
+ ok(responseObserver, res);
+ }
+
+ @Override
+ public void handShake(Types.PHandShakeRequest request,
StreamObserver<Types.PHandShakeResponse> responseObserver) {
+ log.info("handShake request={}", request);
+ ok(responseObserver,
Types.PHandShakeResponse.newBuilder().setStatus(Types.PStatus.newBuilder().setStatusCode(0).build()).setHello(request.getHello()).build());
+ }
+
+ public static <T> void ok(StreamObserver<T> observer, T data) {
+ observer.onNext(data);
+ observer.onCompleted();
+ }
+
+}
\ No newline at end of file
diff --git a/samples/doris-demo/udf-demo/src/main/proto/function_service.proto
b/samples/doris-demo/udf-demo/src/main/proto/function_service.proto
new file mode 100644
index 0000000..561be9f
--- /dev/null
+++ b/samples/doris-demo/udf-demo/src/main/proto/function_service.proto
@@ -0,0 +1,63 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+syntax="proto2";
+
+package doris;
+option java_package = "org.apache.doris.proto";
+option cc_generic_services = true;
+
+import "types.proto";
+
+message PRequestContext {
+ optional string id = 1;
+ optional PFunctionContext function_context = 2;
+}
+
+message PFunctionCallRequest {
+ optional string function_name = 1;
+ repeated PValues args = 2;
+ optional PRequestContext context = 3;
+}
+
+message PFunctionCallResponse {
+ optional PValues result = 1;
+ optional PStatus status = 2;
+}
+
+message PCheckFunctionRequest {
+ enum MatchType {
+ IDENTICAL = 0;
+ INDISTINGUISHABLE = 1;
+ SUPERTYPE_OF = 2;
+ NONSTRICT_SUPERTYPE_OF = 3;
+ MATCHABLE = 4;
+ }
+ optional PFunction function = 1;
+ optional MatchType match_type = 2;
+}
+
+message PCheckFunctionResponse {
+ optional PStatus status = 1;
+}
+
+service PFunctionService {
+ rpc fn_call(PFunctionCallRequest) returns (PFunctionCallResponse);
+ rpc check_fn(PCheckFunctionRequest) returns (PCheckFunctionResponse);
+ rpc hand_shake(PHandShakeRequest) returns (PHandShakeResponse);
+}
+
diff --git a/samples/doris-demo/udf-demo/src/main/proto/types.proto
b/samples/doris-demo/udf-demo/src/main/proto/types.proto
new file mode 100644
index 0000000..17252d1
--- /dev/null
+++ b/samples/doris-demo/udf-demo/src/main/proto/types.proto
@@ -0,0 +1,217 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+syntax="proto2";
+
+package doris;
+option java_package = "org.apache.doris.proto";
+
+message PStatus {
+ required int32 status_code = 1;
+ repeated string error_msgs = 2;
+};
+message PScalarType {
+ // TPrimitiveType, use int32 to avoid redefine Enum
+ required int32 type = 1;
+ // Only set if type == CHAR or type == VARCHAR
+ optional int32 len = 2;
+ // Only set for DECIMAL
+ optional int32 precision = 3;
+ optional int32 scale = 4;
+};
+
+// Represents a field in a STRUCT type.
+// TODO: Model column stats for struct fields.
+message PStructField {
+ required string name = 1;
+ optional string comment = 2;
+};
+
+message PTypeNode {
+ // TTypeNodeType(SCALAR, ARRAY, MAP, STRUCT)
+ required int32 type = 1;
+ // only set for scalar types
+ optional PScalarType scalar_type = 2;
+ // only used for structs; has struct_fields.size() corresponding child
types
+ repeated PStructField struct_fields = 3;
+};
+
+// A flattened representation of a tree of column types obtained by depth-first
+// traversal. Complex types such as map, array and struct have child types
corresponding
+// to the map key/value, array item type, and struct fields, respectively.
+// For scalar types the list contains only a single node.
+// Note: We cannot rename this to TType because it conflicts with Thrift's
internal TType
+// and the generated Python thrift files will not work.
+// Note: TTypeDesc in impala is TColumnType, but we already use TColumnType,
so we name this
+// to TTypeDesc. In future, we merge these two to one
+message PTypeDesc {
+ repeated PTypeNode types = 1;
+};
+
+message PUniqueId {
+ required int64 hi = 1;
+ required int64 lo = 2;
+};
+
+message PGenericType {
+ enum TypeId {
+ UINT8 = 0;
+ UINT16 = 1;
+ UINT32 = 2;
+ UINT64 = 3;
+ UINT128 = 4;
+ UINT256 = 5;
+ INT8 = 6;
+ INT16 = 7;
+ INT32 = 8;
+ INT64 = 9;
+ INT128 = 10;
+ INT256 = 11;
+ FLOAT = 12; // map to Float32
+ DOUBLE = 13; // map to Float64
+ BOOLEAN = 14;
+ DATE = 15;
+ DATETIME = 16;
+ HLL = 17;
+ BITMAP = 18;
+ LIST = 19;
+ MAP = 20;
+ STRUCT =21;
+ STRING = 22;
+ DECIMAL32 = 23;
+ DECIMAL64 = 24;
+ DECIMAL128 = 25;
+ BYTES = 26;
+ NOTHING = 27;
+ UNKNOWN = 999;
+ }
+ required TypeId id = 2;
+ optional PList list_type = 11;
+ optional PMap map_type = 12;
+ optional PStruct struct_type = 13;
+ optional PDecimal decimal_type = 14;
+}
+
+message PList {
+ required PGenericType element_type = 1;
+}
+
+message PMap {
+ required PGenericType key_type = 1;
+ required PGenericType value_type = 2;
+}
+
+message PField {
+ required PGenericType type = 1;
+ optional string name = 2;
+ optional string comment = 3;
+}
+
+message PStruct {
+ repeated PField fields = 1;
+ required string name = 2;
+}
+
+message PDecimal {
+ required uint32 precision = 1;
+ required uint32 scale = 2;
+}
+
+message PDateTime {
+ optional int32 year = 1;
+ optional int32 month = 2;
+ optional int32 day = 3;
+ optional int32 hour = 4;
+ optional int32 minute = 5;
+ optional int32 second = 6;
+ optional int32 microsecond = 7;
+}
+
+message PValue {
+ required PGenericType type = 1;
+ optional bool is_null = 2 [default = false];
+ optional double double_value = 3;
+ optional float float_value = 4;
+ optional int32 int32_value = 5;
+ optional int64 int64_value = 6;
+ optional uint32 uint32_value = 7;
+ optional uint64 uint64_value = 8;
+ optional bool bool_value = 9;
+ optional string string_value = 10;
+ optional bytes bytes_value = 11;
+ optional PDateTime datetime_value = 12;
+}
+
+message PValues {
+ required PGenericType type = 1;
+ optional bool has_null = 2 [default = false];
+ repeated bool null_map = 3;
+ repeated double double_value = 4;
+ repeated float float_value = 5;
+ repeated int32 int32_value = 6;
+ repeated int64 int64_value = 7;
+ repeated uint32 uint32_value = 8;
+ repeated uint64 uint64_value = 9;
+ repeated bool bool_value = 10;
+ repeated string string_value = 11;
+ repeated bytes bytes_value = 12;
+ repeated PDateTime datetime_value = 13;
+}
+
+// this mesage may not used for now
+message PFunction {
+ enum FunctionType {
+ UDF = 0;
+ // not supported now
+ UDAF = 1;
+ UDTF = 2;
+ }
+ message Property {
+ required string key = 1;
+ required string val = 2;
+ };
+ required string function_name = 1;
+ repeated PGenericType inputs = 2;
+ optional PGenericType output = 3;
+ optional FunctionType type = 4 [default = UDF];
+ optional bool variadic = 5;
+ repeated Property properties = 6;
+}
+
+message PFunctionContext {
+ optional string version = 1 [default = "V2_0"];
+ repeated PValue staging_input_vals = 2;
+ repeated PValue constant_args = 3;
+ optional string error_msg = 4;
+ optional PUniqueId query_id = 5;
+ optional bytes thread_local_fn_state = 6;
+ optional bytes fragment_local_fn_state = 7;
+ optional string string_result = 8;
+ optional int64 num_updates = 9;
+ optional int64 num_removes = 10;
+ optional int64 num_warnings = 11;
+}
+
+message PHandShakeRequest {
+ optional string hello = 1;
+}
+
+message PHandShakeResponse {
+ optional PStatus status = 1;
+ optional string hello = 2;
+}
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
