xingyuan cheng created FLINK-36769: -------------------------------------- Summary: Suport fury Serializer for pyflink Key: FLINK-36769 URL: https://issues.apache.org/jira/browse/FLINK-36769 Project: Flink Issue Type: New Feature Components: API / Python Affects Versions: 1.16.1 Reporter: xingyuan cheng
Hi, community. Currently, in the batch verification scenario of our algorithm data, we use pyflink and encounter low transmission efficiency caused by low performance of pickle4-based encoding. After research, we decided to adopt Apache fury, a serialization framework based on pickle5 encoding. The implementation of fury in python will define the transmission buffer size in the protocol for transmission to improve the performance of large data transmission. To this end, I Prepared a draft pull request. What do friends in the community think about this? -- This message was sent by Atlassian Jira (v8.20.10#820010)