Attaching client code here again as it got blocked previously by the firewall.
Thanks,
GIrish
From: Girish Narayanan
Date: Thursday, 20 July 2023 at 12:06 PM
To: user@flink.apache.org
Subject: Flink plain avrò deserilization issue
Hi Team/Users,
We are trying to send plain AVRO messages to a
One or more of the following files ( client code.zip ) violates IBM policy and
all attachment(s) have been removed from the message.
**
Hi Team/Users,
We are trying to send plain AVRO messages to a Kafka topic but they are not
I have fixed the issue by increasing the CPU and memory for my JM and TM
pods.
Make sure your instance type can accommodate the required resources.
On Wed, 19 Jul 2023 at 13:35, Orkhan Dadashov
wrote:
> Hi Flink users,
>
> I'm following up on this guide to try the Flink K8S operator (1.5.0
> ve
Hey!
Please help us understand why you need to delete and recreate the
FlinkDeployment objects in your ecosystem. Maybe we can help suggest some
alternative to make your life easier :)
Of course every prod ecosystem is unique in its own way and larger
platforms generally have a layer on top of th
Hi Rajat,
We are using apache zeppelin as our entry point for submitting flink ad-hoc
queries (and spark jobs actually).
It supports interactive queries, data visualization, multiple data query
engines, multiple auth models. You can check out other features on its
official website.
But because o
Hi Rajat,
Currently sql-gateway supports REST[1] and Hive[2] endpoints.
For Hive endpoints, you can submit sql jobs with existing Hive clients,
such as hive jdbc, apache superset and other systems.
For REST endpoints, you can use flink sql-client to submit your sql jobs.
We support jdbc-driver[3
Hi Neha,
The HOP window will increase the size of the checkpoint and I'm sorry
that I'm not very familiar with the HOP window.
If the configurations are all right, and you want to confirm if it's a HOP
window issue, I think you can submit a flink job without HOP window but
with regular agg operat
Hi Gyula,
Got it. Our use case might be unique to our own ecosystem here at
Robinhood, so I will have to look into creating a service that can search
for the latest savepoint / checkpoint in S3 and provide that to the
FlinkDeployment resource.
Will the Flink Community be okay with us adding this
Hi Flink users,
I'm following up on this guide to try the Flink K8S operator (1.5.0 version
):
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5/docs/try-flink-kubernetes-operator/quick-start/
When I try to deploy a basic example, JM and TM start, but TM fails to
regi
Hi!
I don’t understand why you need to delete the deployment to restart. You
can suspend, use the restartNonce or simply upgrade .
These should cover most upgrade/restart scenarios. Like with other
resources in Kubernetes once you delete them the status is gone, so the
FlinkDeployment won’t keep
Hi Gyula,
Thank you for responding so quickly. I went through the page you sent me a
bit more, and I see the following (
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.4/docs/custom-resource/job-management/#running-suspending-and-deleting-applications
):
Deleting a de
Hey Tony,
Please see:
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/job-management/#stateful-and-stateless-application-upgrades
The operator is made especially to handle stateful application upgrades
robustly. In general any spec change that you make
Hi Flink Community,
My name is Tony Chen, and I am a software engineer at Robinhood. I have
some questions on restarting a Flink Application from a savepoint or
checkpoint.
We currently store our checkpoints and savepoints in S3, and we would like
to use the Apache Flink Kubernetes Operator to ma
Hi Flink users,
I'm following up on this guide to try the Flink K8S operator (1.5.0 version
):
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5/docs/try-flink-kubernetes-operator/quick-start/
When I try to deploy a basic example, JM and TM start, but TM fails to
regi
Hi team,
I have set up a session cluster on k8s via sql gateway. I am looking for
an open source Flink sql editor that can submit sql queries on top of the
k8s session cluster. Any suggestions for sql editor to submit queries ?
Thanks
Hi,
As documented [1] this option "enables uploading and starting jobs through
the Flink UI (true by default). Please note that even when this is
disabled, session clusters still accept jobs through REST requests (HTTP
calls). This flag only guards the feature to upload jobs in the UI."
It won't
Hi Team,
I am using the upsert-kafka table API connector of Apache Flink to consume
events from a kafka topic, I want to log the kafka payloads that are
consumed. Is there a way to log it?
My code looks as below:
EnvironmentSettings settings =
EnvironmentSettings.newInstance().inStreamingMode().
Thanks for sharing the information.
I also observed the same, S3 (Primary Checkpoint Storage) + EBS (Task Local
Recovery) performs better than EBS as Primary Checkpoint storage.
On Tue, Jul 18, 2023 at 12:21 PM Konstantin Knauf wrote:
> Hi Prabhu,
>
> this should be possible, but is quite exp
18 matches
Mail list logo
