I am looking to get readers from kafka / keyBy and Sink working with all 60
threads.
For the most part it is working correctly
DataStream stream =env.addSource(new
FlinkKafkaConsumer08<>("kafkatopic", schema,
properties)).setParallelism(60).flatMap(new
SchemaRecordSplit()).setParallelism(60).n
Sure you can.
Examples come with Java and Scala versions. e.g.
https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/datastream_api.html
On Tue, Mar 21, 2017 at 9:46 AM, Madabhattula Rajesh Kumar <
mrajaf...@gmail.com> wrote:
> Hi,
>
> How stable scala API's when compare to Java API's
Hi,
How stable scala API's when compare to Java API's with respect to features
and capabilities.
I am basically scala developer. I want to explore flink. Can I start with
scala?
Regards,
Rajesh
Thanks, please do keep me posted!
> On 20 Mar 2017, at 21:50, Florian König wrote:
>
> @Aljoscha Thank you for the pointer to ProcessFunction. That looks like a
> better approach with less code and other overhead.
>
> After restoring, the job is both reading new elements and emitting some, but
Not sure if that is the issue here, but the code does a "fs.exists()" for
the path. We should not do this anywhere in all checkpointing related code,
because this fails frequently with s3 due to its consistency model.
This should eventually succeed though, so I think it is not the problem
here.
Hi,
There currently isn’t any shaded version Kafka 0.8 connector version available,
so yes, you would need to do build that yourself.
I’m not completely sure if there will be any class name clashes, because the
Kafka 0.8 API is typically packaged under `kafka.javaapi.*`, while in 0.9 /
0.10 th
I think it very depends on your use case, maybe you can use combiner
first to reduce the amount of records per key. Maybe you can explain
your application a bit more (which window, type of aggregations).
It often helps e.g. to introduce an artifical key und merge the result
of multiple windows
Hi Gwenhael,
I will loop in Gordon becaue he is more familar with the Kafka
connectors. Have you experiences with two versions in the same project?
Am 20/03/17 um 15:57 schrieb Gwenhael Pasquiers:
Hi,
Before doing it myself I thought it would be better to ask.
We need to consume from kafk
