I had some back and forth last year about this, I'll forward the discussion
email chain to you privately (it was in this mailing list).
Basically, the idea was to make *DispatcherRestEndpoint* and/or
*WebMonitorExtension* more accessible so we can extend them. It didn't look
too much work on Flink's side, but of course I can't tell confidently.
--
[image: MotaWord]
Oytun Tez
M O T A W O R D | CTO & Co-Founder
oy...@motaword.com
<https://www.motaword.com/blog>
On Tue, Apr 21, 2020 at 11:12 AM Jeff Zhang <zjf...@gmail.com> wrote:
> I know some users do the same thing in spark. Usually the service run
> spark driver side. But flink is different from spark. Spark driver is equal
> to flink client + flink job manager. I don't think currently we allow to
> run any user code in job manager. So allow running user defined service in
> job manager might a big change for flink.
>
>
>
> Flavio Pompermaier <pomperma...@okkam.it> 于2020年4月21日周二 下午11:06写道：
>
>> In my mind the user API could run everywhere but the simplest thing is to
>> make them available in the Job Manager (where the other REST API lives).
>> They could become a very simple but powerful way to add valuable services
>> to Flink without adding useless complexity to the overall architecture for
>> just a few methods.
>> I don't know whether Spark or Beam allow you to do something like that
>> but IMHO it's super useful (especially from a maintenance point of view wrt
>> the overall architecture complexity).
>>
>> @Oytun indeed we'd like to avoid recompiling everything when a single
>> user class (i.e. not related to Flink classes) is modified or added. Glad
>> to see that there are other people having the same problem here
>>
>> On Tue, Apr 21, 2020 at 4:39 PM Jeff Zhang <zjf...@gmail.com> wrote:
>>
>>> Hi Flavio,
>>>
>>> I am curious know where service run, Do you create this service in UDF
>>> and run it in TM ?
>>>
>>> Flavio Pompermaier <pomperma...@okkam.it> 于2020年4月21日周二 下午8:30写道：
>>>
>>>> Hi to all,
>>>> many times it happens that we use Flink as a broker towards the data
>>>> layer but we need to be able to get some specific info from the
>>>> data sources we use (i.e. get triggers and relationships from jdbc).
>>>> The quick and dirty way of achieving this is to run a Flink job that
>>>> calls another service to store the required info. Another solution could be
>>>> to add a custom REST service that contains a lot of dependencies already
>>>> provided by Flink, with the risk of having misaligned versions between the
>>>> 2..
>>>> It would be much simpler to enable users to add custom REST services to
>>>> Flink in a configurable file. something like:
>>>> /myservice1/* -> com.example.MyRestService1
>>>> /myservice2/* -> com.example.MyRestService2
>>>>
>>>> The listed classes should be contained in a jar within the Flink lib
>>>> dir and should implement a common interface.
>>>> In order to avoid path collisions with already existing FLINK services,
>>>> the configured path can be further prefixed with some other token (e.g.
>>>> /userapi/*).
>>>>
>>>> What do you think about this? Does it sound reasonable to you?
>>>> Am I the only one that thinks this could be useful for many use cases?
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
