Re: Semantic Properties and Functions with Iterables

2015-03-09 Thread Stephan Ewen
That is right, Timo, as far as I would understand it. On Mon, Mar 9, 2015 at 12:04 PM, Timo Walther wrote: > Thanks for the clarification. If I have understood it correctly, forwarded > fields are only interesting for key fields, right? So I will implement that > key information is passed to the

Re: Semantic Properties and Functions with Iterables

2015-03-09 Thread Timo Walther
Thanks for the clarification. If I have understood it correctly, forwarded fields are only interesting for key fields, right? So I will implement that key information is passed to the analyzer for consideration. So if GroupReduce1 is grouped by f1, the result will be @ForwardedFields("1") in t

Re: Semantic Properties and Functions with Iterables

2015-03-08 Thread Fabian Hueske
I added you comment and an answer to FLINK-1656: "Right, that's a good point. +1 limiting to key fields. That's much easier to reason about for users. However, I am not sure how it is implemented right now. I guess secondary sort info is already removed by the property filtering, but I need to v

Re: Semantic Properties and Functions with Iterables

2015-03-08 Thread Stephan Ewen
Any other thoughts in this? On Fri, Mar 6, 2015 at 12:12 PM, Stephan Ewen wrote: > I think the order of emitting elements is not part of the forward field > properties, but would rather be a separate one that we do not have right > now. > > At the moment, we would assume that all group operation

Re: Semantic Properties and Functions with Iterables

2015-03-06 Thread Stephan Ewen
I think the order of emitting elements is not part of the forward field properties, but would rather be a separate one that we do not have right now. At the moment, we would assume that all group operations destroy secondary orders. In that sense, forward fields in group operations only make sens

Re: Semantic Properties and Functions with Iterables

2015-03-06 Thread Fabian Hueske
Hi Timo, there are several restrictions for forwarded fields of operators with iterator input. 1) forwarded fields must be emitted in the order in which they are received through the iterator 2) all forwarded fields of a record must stick together, i.e., if your function builds record from field 0

Semantic Properties and Functions with Iterables

2015-03-06 Thread Timo Walther
Hey all, I'm currently working a lot on the UDF static code analyzer. But I have a general question about Semantic Properties which might be also interesting for other users. How is the ForwardedFields annotation interpreted for UDF functions with Iterables? An example can be found in: or