[
https://issues.apache.org/jira/browse/FLINK-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020522#comment-15020522
]
ASF GitHub Bot commented on FLINK-2318:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/1390
[FLINK-2318] Union can be used as BroadcastVariable
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 2318_bc_union
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1390.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1390
----
commit 91320b5c44721a2a36be6a2153f00c53a0609210
Author: zentol <ches...@apache.org>
Date: 2015-11-19T12:20:18Z
[FLINK-2318] Union can be used as BroadcastVariable
----
> BroadcastVariable of unioned data set fails
> -------------------------------------------
>
> Key: FLINK-2318
> URL: https://issues.apache.org/jira/browse/FLINK-2318
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime, Optimizer
> Affects Versions: 0.9
> Reporter: Fabian Hueske
> Assignee: Chesnay Schepler
>
> Using a unioned data set as broadcast variable such as this:
> {code}
> DataSet d1 = [...]
> DataSet d2 = [...]
> DataSet d3 = [...]
> d1
> .map(new MyMapper())
> .withBroadcastSet(d2.union(d3), "myBroadcast");
> {code}
> throws an exception at runtime:
> {code}
> java.lang.Exception: Call to registerInputOutput() of invokable failed
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:504)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Initializing the input streams failed
> in Task MapPartition (MapPartition at
> translatHashJoinAsMap(FlinkFlowStep.java:755)): Illegal input group size in
> task configuration: -1
> at
> org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:246)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:501)
> ... 1 more
> Caused by: java.lang.Exception: Illegal input group size in task
> configuration: -1
> at
> org.apache.flink.runtime.operators.RegularPactTask.initBroadcastInputReaders(RegularPactTask.java:783)
> at
> org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:243)
> ... 2 more
> {code}
> A simple workaround is to apply an identity mapper on the unioned data set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
