Hi Ken,
you can let a class implement both the SourceFunction and the SinkFunction.
However when running a job, the source and the sink will be distinct
instances. Thus, there is no way that they share instance variables.
What you could do is to write the updated and newly discovered URLs to a
me
Hi Fabian,
Thanks for responding. Comments and questions inline below.
Regards,
— Ken
> On Sep 29, 2016, at 6:10am, Fabian Hueske wrote:
>
> Hi Ken,
>
> you can certainly have partitioned sources and sinks. You can control the
> parallelism by calling .setParallelism() method.
So I assume
Hi Ken,
you can certainly have partitioned sources and sinks. You can control the
parallelism by calling .setParallelism() method.
If you need a partitioned sink, you can call .keyBy() to hash partition.
I did not completely understand the requirements of your program. Can you
maybe provide pseud
Hi all,
I’ve got a very specialized DB (runs in the JVM) that I need to use to both
keep track of state and generate new records to be processed by my Flink
streaming workflow. Some of the workflow results are updates to be applied to
the DB.
And the DB needs to be partitioned.
My initial app
