[
https://issues.apache.org/jira/browse/FLINK-40472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu closed FLINK-40472.
---------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
Merged to master via 4eb8666832f7fb6fd0a08c01ac93ed135b1dc11d
> Add Arrow-native vectorized UDF support to DataFrame API
> --------------------------------------------------------
>
> Key: FLINK-40472
> URL: https://issues.apache.org/jira/browse/FLINK-40472
> Project: Flink
> Issue Type: Sub-task
> Reporter: Liu Liu
> Assignee: Liu Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> Support the following usage:
> {code:java}
> import pyarrow as pa
> import pyarrow.compute as pcfrom pyflink.dataframe import DataType, col, udf
> @udf(return_dtype=DataType.string(), batch_size=1024)
> def normalize_name(names: pa.Array) -> pa.Array:
> # Operates directly on an Arrow array without converting to pandas.
> return pc.utf8_upper(names)
> result = df.with_column(
> "normalized_name",
> normalize_name(col("name")),
> ) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
