I added some comments to your documents. I think we should work on these
limitations step by step. A first step could be to support Map<String,
?> by considering only the raw types. Another step would be to allow
eval(Object) as a wild card for operands.
Regards,
Timo
Am 14.05.18 um 18:23 schrieb Rong Rong:
Thanks for the reply Timo / Fabian,
Yes that's what I had in mind. ParameterType can be vague but return type
has to be exact.
I can image that: depending on the input parameter type, the output type
can be different. But I cannot think of a concrete use cases as of now.
I actually created a doc [1] regarding the use cases we currently have, and
some very preliminary solution possibilities.
Please kindly take a look when you have time, any comments and suggestions
are highly appreciated.
--
Rong
[1]
https://docs.google.com/document/d/1zKSY1z0lvtQdfOgwcLnCMSRHew3weeJ6QfQjSD0zWas/edit?usp=sharing
On Mon, May 14, 2018 at 4:36 AM, Timo Walther <twal...@apache.org> wrote:
Hi Rong,
yes I think we can improve the type infererence at this point. Input
parameter type inference can be more tolerant but return types should be as
exact as possible.
The change should only touch ScalarSqlFunction and
UserDefinedFunctionUtils#createEvalOperandTypeInference, right?
Regards,
Timo
Am 14.05.18 um 11:52 schrieb Fabian Hueske:
Hi Rong,
I didn't look into the details of the example that you provided, but I
think if we can improve the internal type resolution of scalar UDFs we
should definitely go for it.
There is quite a bit of information available such as the signatures of
the eval() methods but also the argument types provided by Calcite's
analyzer.
Not sure if we leverage all that information to the full extend.
The ScalarFunction interface also provides methods to override some of
the type extraction behavior.
@Timo, what do you think?
Best,
Fabian
2018-05-04 20:09 GMT+02:00 Rong Rong <walter...@gmail.com <mailto:
walter...@gmail.com>>:
Hi,
We have been looking into more intelligent UDF supports such as
creating a
better type inference module to infer automatically composite data
types[1].
One most comment pain point we have are some use cases where users
would
like to re-use a rather generic UDF, for example:
public List<String> eval(Map<String, ?> myMap) {
return new ArrayList<>(myMap.keySet());
>
}
>
In this case, since we are only interested in the key sets of the map,
value type cannot be easily resolved or overrided using concrete
types.
Eventually we end up overriding the exact same function with
multiple case
classes, so that each one uses a different ValueTypeInfo.
This is rather inefficient in terms of user development cycle. I was
wondering if there's a better way in FunctionCatalog lookup to
match a UDF
in context.
Best,
Rong
[1] https://issues.apache.org/jira/browse/FLINK-9294
<https://issues.apache.org/jira/browse/FLINK-9294>
