Hi,
I have a source table using a Kinesis connector reading events from AWS
EventBridge using PyFlink 1.15.0. An example of the sorts of data that are in
this stream is here:
https://docs.aws.amazon.com/codebuild/latest/userguide/sample-build-notifications.html#sample-build-notifications-ref.
Yes of-course. I already feel a bit less intelligent for having asked the
question ;-)
The status now is that I managed to have it all puzzled together. Copying
the files from s3 to an ephemeral volume takes all of 2 seconds so it's
really not an issue. The cluster starts and our fat jar and Ap
Hi Matt,
Yes. There are several official Flink images with various JVMs including
Java 11.
https://hub.docker.com/_/flink
Cheers,
Matyas
On Fri, Jun 24, 2022 at 2:06 PM Matt Casters
wrote:
> Hi Mátyás & all,
>
> Thanks again for the advice so far. On a related note I noticed Java 8
> being us
I've had success using the Java in pyflink via pyflink.java_gateway.
Something like:
from pyflink.java_gateway import get_gateway
jvm = get_gateway()
# then perhaps something like:
FlinkKinesisConsumer = jvm.
org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer
There also seems to
Hi all,
There are a number of connectors which do not appear to be in the Python API
v1.15.0, e.g. Kinesis. I can see that it's possible to use these connectors by
using the Table API:
CREATE TABLE my_table (...)
WITH ('connector' = 'kinesis' ...)
I guess if you wanted the stream as a DataStre
Hi Mátyás & all,
Thanks again for the advice so far. On a related note I noticed Java 8
being used, indicated in the log.
org.apache.flink.runtime.entrypoint.ClusterEntrypoint[] -
JAVA_HOME: /usr/local/openjdk-8
Is there a way to use Java 11 to start Flink with?
Kind regards,
Matt
On
