Greg Hogan created FLINK-4705:
---------------------------------
Summary: Instrument FixedLengthRecordSorter
Key: FLINK-4705
URL: https://issues.apache.org/jira/browse/FLINK-4705
Project: Flink
Issue Type: Improvement
Components: Local Runtime
Affects Versions: 1.2.0
Reporter: Greg Hogan
Assignee: Greg Hogan
The {{NormalizedKeySorter}} sorts on the concatenation of (potentially partial)
keys plus an 8-byte pointer to the record. After sorting each pointer must be
dereferenced, which is not cache friendly.
The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
followed by the remainder of the record. The records can then be deserialized
in sequence.
Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
comparator methods {{writereadWithKeyNormalization}} and
{{readWithKeyNormalization}}.
Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
501.8 s (8.8% faster).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
