I've created FLINK-10100 [1] to track the problem and suggest a solution
and workaround.
Best, Fabian
[1] https://issues.apache.org/jira/browse/FLINK-10100
2018-08-08 10:39 GMT+02:00 Fabian Hueske :
> Hi Dylan,
>
> Yes, that's a bug.
> As you can see from the plan, the partitioning step is push
Hi Dylan,
Yes, that's a bug.
As you can see from the plan, the partitioning step is pushed past the
Filter.
This is possible, because the optimizer knows that a Filter function cannot
modify the data (it only removes records).
A workaround should be to implement the filter as a FlatMapFunction. A
I agree, please open a JIRA.
On 08.08.2018 05:11, vino yang wrote:
Hi Dylan,
I roughly looked at your job program and the DAG of the job. It seems
that the optimizer chose the wrong optimization execution plan.
cc Till.
Thanks, vino.
Dylan Adams mailto:dylan.ad...@gmail.com>>
于2018年8月8日周三
Hi Dylan,
I roughly looked at your job program and the DAG of the job. It seems that
the optimizer chose the wrong optimization execution plan.
cc Till.
Thanks, vino.
Dylan Adams 于2018年8月8日周三 上午2:26写道：
> I'm trying to use the Flink DataSet API to validate some records and have
> run into an i
I'm trying to use the Flink DataSet API to validate some records and have
run into an issue. My program uses joins to validate inputs against
reference data. One of the attributes I'm validating is optional, and only
needs to be validated when non-NULL. So I added a filter to prevent the
null-keyed
