[
https://issues.apache.org/jira/browse/FLINK-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594853#comment-14594853
]
ASF GitHub Bot commented on FLINK-1085:
---------------------------------------
GitHub user dabaitu opened a pull request:
https://github.com/apache/flink/pull/854
FLINK-1085: Unnecessary failing of GroupReduceCombineDriver
I have a unit test failure and it seems it may have been there before my
changes:
Tests in error:
UtilsTest.testUberjarLocator:39 NullPointer
It's looking for some uber jar and not finding it. Did I need to compile
something else first?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dabaitu/flink master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/854.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #854
----
commit e693ba2398c170b6d65a2365a4cc0fe091f3319b
Author: dabaitu <tom....@gmail.com>
Date: 2015-06-20T22:35:48Z
FLINK-1085: change exception to warning log and track oversized Record count
----
> Unnecessary failing of GroupReduceCombineDriver
> -----------------------------------------------
>
> Key: FLINK-1085
> URL: https://issues.apache.org/jira/browse/FLINK-1085
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 0.6.1-incubating, 0.7.0-incubating
> Reporter: Fabian Hueske
> Labels: starter
>
> With a recent update (commit cbbcf7820885a8a9734ffeba637b0182a6637939) the
> GroupReduceCombineDriver was changed to not use an asynchronous partial
> sorter. Instead, the driver fills a sort buffer with records, sorts it,
> combines them, clears the buffer, and continues to fill it again.
> The GroupReduceCombineDriver fails if a record cannot be serialized into an
> empty sort buffer, i.e., if the record is too large for the buffer.
> Alternatively, we should emit a WARN message for the first record that is too
> large and just forward all records which do not fit into the empty sort
> buffer (maybe continue to count how many records were simply forwarded and
> give a second WARN message with this statistic).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
