Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/2177#discussion_r68908810
--- Diff: docs/setup/config.md ---
@@ -85,6 +85,8 @@ The default fraction for managed memory can be adjusted
using the `taskmanager.m
- `taskmanager.memory.preallocate`: Can be either of `true` or `false`.
Specifies whether task managers should allocate all managed memory when
starting up. (DEFAULT: false)
+- `taskmanager.runtime.large-record-handler`: Whether to use the
LargeRecordHandler when spilling. This feature is experimental. (DEFAULT: false)
--- End diff --
Not really, I'm afraid. I just recently disabled it by default because of
the known issues.
What I can gather from the code is that it is a separate sort buffer that
is intended for very large records. It is not obvious from the code what "very
large records" are, however.
The known problem is that key serialization does not work correctly if the
user specified a custom type or if Scala types are used because the
`TypeAnalyzer` is used in the `LargeRecordHandler` to get a `TypeInformation`
on the fly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
