Hi Rong,
yes I think we can improve the type infererence at this point. Input
parameter type inference can be more tolerant but return types should be
as exact as possible.
The change should only touch ScalarSqlFunction and
UserDefinedFunctionUtils#createEvalOperandTypeInference, right?
Regards,
Timo
Am 14.05.18 um 11:52 schrieb Fabian Hueske:
Hi Rong,
I didn't look into the details of the example that you provided, but I
think if we can improve the internal type resolution of scalar UDFs we
should definitely go for it.
There is quite a bit of information available such as the signatures
of the eval() methods but also the argument types provided by
Calcite's analyzer.
Not sure if we leverage all that information to the full extend.
The ScalarFunction interface also provides methods to override some of
the type extraction behavior.
@Timo, what do you think?
Best,
Fabian
2018-05-04 20:09 GMT+02:00 Rong Rong <walter...@gmail.com
<mailto:walter...@gmail.com>>:
Hi,
We have been looking into more intelligent UDF supports such as
creating a
better type inference module to infer automatically composite data
types[1].
One most comment pain point we have are some use cases where users
would
like to re-use a rather generic UDF, for example:
public List<String> eval(Map<String, ?> myMap) {
return new ArrayList<>(myMap.keySet());
>
}
>
In this case, since we are only interested in the key sets of the map,
value type cannot be easily resolved or overrided using concrete
types.
Eventually we end up overriding the exact same function with
multiple case
classes, so that each one uses a different ValueTypeInfo.
This is rather inefficient in terms of user development cycle. I was
wondering if there's a better way in FunctionCatalog lookup to
match a UDF
in context.
Best,
Rong
[1] https://issues.apache.org/jira/browse/FLINK-9294
<https://issues.apache.org/jira/browse/FLINK-9294>
