To a naive Flink newcomer (me) it's a little surprising that there is no
pure "data" mechanism for specifying a Flink pipeline, only "code"
interfaces. With the DataStream interface I can use Java, Scala or Python
to set up a pipeline and then execute it - but that doesn't really
seem to *need
*a programming model, it seems like configuration, which could be done with
data? OK, one does need occasionally to specify some custom code, e.g. a
ProcessFunction, but for any given use-case, a relatively static library of
such functions would seem fine.

My use case is that I have lots of customers, and I'm doing a similar job
for each of them, so I'd prefer to have a library of common code (e.g.
ProcessFunctions), and then specify each customer's specific requirements
in a single config file.  To do that in Java, I'd have to do
metaprogramming (to build various pieces of Java out of that config file).

Flink SQL seems to be the closest solution, but doesn't appear to support
fundamental Flink concepts such as timers (?). Is there a plan to evolve
Flink SQL to support timers? Timeouts is my specific need.

Thanks,

-Pilgrim
--
Learn more at https://devicepilot.com @devicepilot
<https://t.sidekickopen70.com/s2t/c/5/f18dQhb0S7kv8cpgQZVc6VPt59hl3kW7_k2842PjkFxW2R1KhZ7v4vclW2Rxbb82bzNKzf7GYHvr01?te=W3R5hFj4cm2zwW4fQ47l4fGCmnW3Fbt5S3H4THtF3F6jFSWsSg1&si=5987503666495488&pi=527a1892-cb03-476e-bce3-95b7b9783178>

Reply via email to