[
https://issues.apache.org/jira/browse/FLINK-11775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee updated FLINK-11775:
---------------------------------
Description:
Blink new binary format is based on MemorySegment.
Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
bytes
{code:java}
/**
* Provides the interface for write(Segment).
*/
public interface MemorySegmentWritable {
/**
* Writes {@code len} bytes from memory segment {@code segment} starting at
offset {@code off}, in order,
* to the output.
*
* @param segment memory segment to copy the bytes from.
* @param off the start offset in the memory segment.
* @param len The number of bytes to copy.
* @throws IOException if an I/O error occurs.
*/
void write(MemorySegment segment, int off, int len) throws IOException;
}{code}
If we want to write a Memory Segment to DataOutputView, we need to copy bytes
to byte[] and then write it in, which is less effective.
If we let AbstractPagedOutputView have a write(MemorySegment) interface, we can
copy it directly.
We need to ensure this in network serialization, batch operator calculation
serialization, Streaming State serialization to avoid new byte[] and copy.
was:
Blink new binary format is based on MemorySegment.
Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
bytes
```
/**
* Provides the interface for write(Segment).
*/
public interface MemorySegmentWritable {
/**
* Writes \{@code len} bytes from memory segment \{@code segment} starting at
offset \{@code off}, in order,
* to the output.
*
* @param segment memory segment to copy the bytes from.
* @param off the start offset in the memory segment.
* @param len The number of bytes to copy.
* @throws IOException if an I/O error occurs.
*/
void write(MemorySegment segment, int off, int len) throws IOException;
}
```
> Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
> bytes
> -----------------------------------------------------------------------------------
>
> Key: FLINK-11775
> URL: https://issues.apache.org/jira/browse/FLINK-11775
> Project: Flink
> Issue Type: New Feature
> Components: Core
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
>
> Blink new binary format is based on MemorySegment.
> Introduce MemorySegmentWritable to let DataOutputView direct copy to internal
> bytes
> {code:java}
> /**
> * Provides the interface for write(Segment).
> */
> public interface MemorySegmentWritable {
> /**
> * Writes {@code len} bytes from memory segment {@code segment} starting at
> offset {@code off}, in order,
> * to the output.
> *
> * @param segment memory segment to copy the bytes from.
> * @param off the start offset in the memory segment.
> * @param len The number of bytes to copy.
> * @throws IOException if an I/O error occurs.
> */
> void write(MemorySegment segment, int off, int len) throws IOException;
> }{code}
>
> If we want to write a Memory Segment to DataOutputView, we need to copy bytes
> to byte[] and then write it in, which is less effective.
> If we let AbstractPagedOutputView have a write(MemorySegment) interface, we
> can copy it directly.
> We need to ensure this in network serialization, batch operator calculation
> serialization, Streaming State serialization to avoid new byte[] and copy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
