xingyuan cheng created FLINK-36769:
--------------------------------------
Summary: Suport fury Serializer for pyflink
Key: FLINK-36769
URL: https://issues.apache.org/jira/browse/FLINK-36769
Project: Flink
Issue Type: New Feature
Components: API / Python
Affects Versions: 1.16.1
Reporter: xingyuan cheng
Hi, community. Currently, in the batch verification scenario of our algorithm
data, we use pyflink and encounter low transmission efficiency caused by low
performance of pickle4-based encoding. After research, we decided to adopt
Apache fury, a serialization framework based on pickle5 encoding. The
implementation of fury in python will define the transmission buffer size in
the protocol for transmission to improve the performance of large data
transmission. To this end, I Prepared a draft pull request. What do friends in
the community think about this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
