hackergin commented on code in PR #23130: URL: https://github.com/apache/flink/pull/23130#discussion_r1290474952
########## docs/content/docs/dev/table/procedures.md: ########## @@ -0,0 +1,544 @@ +--- +title: "Procedures" +is_beta: true +weight: 50 +type: docs +aliases: + - /dev/table/procedures.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Procedures + +Flink Table API & SQL empowers users to perform data manipulation or administrative tasks with procedures. Procedures can run FLINK jobs with `StreamExecutionEnvironment` provided, which make procedures more powerful and flexible. + +## Implementation Guide + +To call a procedure, the procedure should be provided by a catalog. To provide procedures in a catalog, you must implement a procedure and then return the procedure +in method `Catalog.getProcedure(ObjectPath procedurePath)`. The following steps will guild how to implement a procedure and provide it in a catalog. + +### Procedure Class + +An implementation class must implement the interface `org.apache.flink.table.procedures.Procedure`. + +The class must be declared `public`, not `abstract`, and should be globally accessible. Thus, non-static inner or anonymous classes are not allowed. + +### Call Methods + +The interface doesn't provide any method，you have to define a method named `call` in which you can implement the logic of the procedure. +The methods must be declared `public` and take a well-defined set of arguments. + +Please note: + +* The first parameter of the method `call` should always be `ProcedureContext` which provides the method `getExecutionEnvironment` to get a `StreamExecutionEnvironment` to enable to run a Flink Job +* The return type should always be an array, like `int[]`, `String[]`, etc + +More detail can be found in the Java doc of the class `org.apache.flink.table.procedures.Procedure`. + +Regular JVM method calling semantics apply. Therefore, it is possible to: +- implement overloaded methods such as `call(ProcedureContext, Integer)` and `call(ProcedureContext, LocalDateTime)` +- use var-args such as `call(ProcedureContext, Integer...)` +- use object inheritance such as `call(ProcedureContext, Object)` that takes both `LocalDateTime` and `Integer` +- and combinations of the above such as `call(ProcedureContext, Object...)` that takes all kinds of arguments + +If you intend to implement procedures in Scala, please add the `scala.annotation.varargs` annotation in +case of variable arguments. Furthermore, it is recommended to use boxed primitives (e.g. `java.lang.Integer` +instead of `Int`) to support `NULL`. + +The following snippets shows an example of an overloaded procedure: + +{{< tabs "0819d780-3052-11ee-be56-0242ac120002" >}} +{{< tab "Java" >}} + +```java +import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; +import org.apache.flink.table.procedure.ProcedureContext; +import org.apache.flink.table.procedures.Procedure; + +// procedure with overloaded call methods +public class GenerateSequenceProcedure implements Procedure { + + public long[] call(ProcedureContext context, int n) { + return generate(context.getExecutionEnvironment(), n); + } + + public long[] call(ProcedureContext context, String n) { + return generate(context.getExecutionEnvironment(), Integer.parseInt(n)); + } + + private long[] generate(StreamExecutionEnvironment env, int n) throws Exception { + long[] sequenceN = new long[n]; + int i = 0; + try (CloseableIterator<Long> result = env.fromSequence(0, n - 1).executeAndCollect()) { + while (result.hasNext()) { + sequenceN[i++] = result.next(); + } + } + return sequenceN; + } +} + +``` +{{< /tab >}} +{{< tab "Scala" >}} +```scala +import org.apache.flink.table.procedure.ProcedureContext +import org.apache.flink.table.procedures.Procedure +import scala.annotation.varargs + +// procedures with overloaded call methods +class GenerateSequenceProcedure extends Procedure { + + def call(context: ProcedureContext, a: Integer, b: Integer): Array[Integer] = { + Array(a + b) + } + + def call(context: ProcedureContext, a: String, b: String): Array[Integer] = { + Array(Integer.valueOf(a) + Integer.valueOf(b)) + } + + @varargs // generate var-args like Java + def eval(context: ProcedureContext, d: Double*): Array[Integer] = { + Array(d.sum.toInt) + } +} + +``` +{{< /tab >}} +{{< /tabs >}} + +### Type Inference +The table ecosystem (similar to the SQL standard) is a strongly typed API. Therefore, both procedure parameters and return types must be mapped to a [data type]({{< ref "docs/dev/table/types" >}}). + +From a logical perspective, the planner needs information about expected types, precision, and scale. From a JVM perspective, the planner needs information about how internal data structures are represented as JVM objects when calling a procedure. + +The logic for validating input arguments and deriving data types for both the parameters and the result of a procedure is summarized under the term _type inference_. + +Flink's procedures implement an automatic type inference extraction that derives data types from the procedure's class and its `call` methods via reflection. If this implicit reflective extraction approach is not successful, the extraction process can be supported by annotating affected parameters, classes, or methods with `@DataTypeHint` and `@ProcedureHint`. More examples on how to annotate procedures are shown below. + +Note: although the return type in `call` method must be array type `T[]`, if use `@DataTypeHint` to annotate the return type , it's actually expected to annotate the component type of the array type, which is actually `T`. + +#### Automatic Type Inference + +The automatic type inference inspects the procedure's class and `call` methods to derive data types for the arguments and result of a procedure. `@DataTypeHint` and `@ProcedureHint` annotations support the automatic extraction. + +For a full list of classes that can be implicitly mapped to a data type, see the [data type extraction section]({{< ref "docs/dev/table/types" >}}#data-type-extraction). Review Comment: ```suggestion For a full list of classes that can be implicitly mapped to a data type, please refer to the [data type extraction section]({{< ref "docs/dev/table/types" >}}#data-type-extraction). ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
