[
https://issues.apache.org/jira/browse/FLINK-40433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115142#comment-18115142
]
Dian Fu commented on FLINK-40433:
---------------------------------
[~auroflow] Done, Thanks for working on it~
> Add map and map_batches support to DataFrame API
> ------------------------------------------------
>
> Key: FLINK-40433
> URL: https://issues.apache.org/jira/browse/FLINK-40433
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Liu Liu
> Assignee: Liu Liu
> Priority: Major
>
> Add Python-native row and batch transformations:
> {code:java}
> def map(
> self,
> func,
> *,
> return_dtype=None,
> concurrency=None,
> ) -> DataFrame {code}
> {{map}} receives each row as {{Dict[str, Any]}} and returns a dictionary
> representing the output row.
> {code:java}
> def map_batches(
> self,
> func,
> *,
> return_dtype=None,
> batch_format,
> concurrency=None,
> batch_size=None,
> ) -> DataFrame {code}
> {{map_batches}} should support:
> * {{dict[str, pandas.Series]}} input/output in pandas mode.
> * {{dict[str, pyarrow.Array]}} input/output in Arrow mode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
