This may be a matter of personal opinion but I don't see connectors as more
self contained than functions.
The problem I see in looking at this from a Function point of view is that
a Function could have both its Sink and Source not connected to Pulsar
which feels awkward. I understand that functio
To my understanding, the Pulsar IO Connectors (i.e. Sources/Sinks) are
quite self-contained. They move data around.
If we want to enable functionality described inside the PIP (process ->
write to otherplace), can we think in another way -- allow flexible
configuring of a Pulsar Function?
Origina
Thanks for the feedback Jerry.
We don't modify the way sources, sinks and functions are detected when it's
based on their fields. The proposal is just to modify the classname of the
function applied in the instance so the same detection rules apply. The
only difference is when detecting if the sink
When the users registers a Sink, the validation code will check that the
return type of the Function is a Record.
The Function can then be used either individually or together with a Sink.
IMO, adding a new API would only add complexity.
Le lun. 25 juil. 2022 à 14:55, Baodi Shi a
écrit :
> > Can
My feedback is to make this change as self contained as possible. Can we
just have a special implementation of a sink that will run the logic of the
"preprocess" function? There are many places in the code where we figure
out if it is a source, sink or a function based on the fields in the
Functi
> Can you explain more what you mean ?
This PIP doesn't change the API of a Function and it's already possible to
write a Function>.
And when declaring a Sink with a Function we'll check that it's the case.
I mean: we should constrain the function interface, otherwise, the user may
return a struc
Thanks for the feedback Asaf
> >- preprocess-function: the preprocess function applied before the
> >Sink. Starts by builtin:// for built-in functions, function:// for
> >package function, http:// or file://
> >
> > 1. While this function is applied only before sink? I thought it repl
Thanks for your comments Baodi
> This proposal looks good to me.
>
> > Only Functions returning a Record will be authorized to ensure that the
> Function sets the Schema explicitly.
>
> Does this mean that the function return type is fixed Record?
Yes
> Can the interface declaration of the fu
Hi,
Few questions:
>- preprocess-function: the preprocess function applied before the
>Sink. Starts by builtin:// for built-in functions, function:// for
>package function, http:// or file://
>
> 1. While this function is applied only before sink? I thought it replaces
the identity f
This proposal looks good to me.
> Only Functions returning a Record will be authorized to ensure that the
> Function sets the Schema explicitly.
Does this mean that the function return type is fixed Record? Can the interface
declaration of the function be displayed in the API changes?
Thanks
10 matches
Mail list logo
