Re: Cannot run pyflink example using Flink CLI

Hi Biao, Thanks for your help. That solved my issue. It turned out that in setup1 (in EMR), I got apache-flink installed, but the package (and its dependencies) are not in the directory `/usr/lib/python3.7/site-packages` (corresponding to the python binary in `/usr/bin/python3`). For some reason,

Re: Flink Native K8S RBAC

Hi! As a reference you can look at how the Flink Kubernetes Operator manages RBAC settings: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/rbac/ https://github.com/apache/flink-kubernetes-operator/blob/main/helm/flink-kubernetes-operator/templates/rbac.yaml

Flink Native K8S RBAC

Hi, I am using custom service account for flink native k8s. These are the rules for the clusterrole I’m using: rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - apiGroups: [""] resources: ["configmaps"] verbs

RE: [EXTERNAL]multiple s3 sinks

Hi- Never mind. Error must have been from shutting down the prior version of the app (timestamp was after restart), issue is now resolved. Thanks, -Dave David L Katz (he/him/his) why?

multiple s3 sinks

Hi- I'm getting the following error when trying to write to 3 s3 sinks (in an aws kinesis java app): Caused by: org.apache.flink.util.FlinkRuntimeException: Cannot have more than one execute() or executeAsync() call in a single environment. Code snippet (s3Sum,s3Agg and s3Det are the table cre

Re: Cannot run pyflink example using Flink CLI

Hi Levan, For your setup1 & 2, it looks like the python environment is not ready. Have you tried python -m pip install apache-flink for the first 2 setups? For your setup3, as you are trying to use `flink run ...` command, it will try to connect to a launched flink cluster but I guess you did not

Re: Does kubernetes operator support manually triggering savepoint with canceling the job?

I think you are confusing manual savepoints with savepoint upgrades. Manual savepoints will trigger a savepoint but not shut down the job. If you want to stop the job with savepoint you set the upgradeMode to savepoint and set the state to SUSPENDED. https://nightlies.apache.org/flink/flink-kubern

Re: Flink CEP Incremental Checkpoint Issue

Hi, Given that Flink 1.12 is no longer supported by the community, can you validate this with the latest Flink version? (Currently 1.15). Next to that, the contents of your checkpoints is not only the results of your CEP, but given that you're using Exactly Once also there's internal information n

Re: Table to DataStream conversion - Error while generating structured type converter.

Hi again, To be extra sure it isn't something related to scala I've created the same example in Java: import java.math.BigDecimal; > import java.time.Instant; > import java.time.temporal.ChronoUnit; > import org.apache.flink.api.common.typeinfo.TypeInformation; > import org.apache.flink.streaming

Table to DataStream conversion - Error while generating structured type converter.

Hi all, I would like to interchangeably convert between Tables and DataStreams with the ability to specify what are the types used in the conversion from a POJO to a Table and vice-versa. To convert from a POJO to a table I'm creating a Schema using a ROW-based DataType, and to convert from a Tab

Cannot run pyflink example using Flink CLI

Hi, I'm new to PyFlink, and I couldn't run a basic example that shipped with Flink. This is the command I tried: ./bin/flink run -py examples/python/datastream/word_count.py Here below are the results I got with different setups: 1. On AWS EMR 6.8.0 (Flink 1.15.1): *Error: No module named 'goog

Re: Limiting backpressure during checkpoints

Hey Robin, Thanks for sharing the detailed information. May I ask, when you are saying "CPU usage is around 80% when checkpoints aren't running, and capped at 100% when they are", do you see zigzag patterns of CPU usage, or is it kept capped at 100% of CPU? I think one possibility is that the sy

Re: SQL Lookup join on nested field

Hi Krzysztof, I'm not aware of any plans on this topic. Best regards, Martijn On Wed, Oct 19, 2022 at 3:56 AM yuxia wrote: > AFAK，there's no any plan/ticket for it. If you think it's needed to be > supported, you can create a tiket in jira[1] for it. > [1] https://issues.apache.org/jira/proje

Re: Activate Flink HA without checkpoints on k8S

Add some more information to Gyula's comment. For application mode without checkpoint, you do not need to activate the HA since it will not take any effect and the Flink job will be submitted again after the JobManager restarted. Because the job submission happens on the JobManager side. For sess

回复: Does kubernetes operator support manually triggering savepoint with canceling the job?

hi, Geng: I successfully triggered savePoint manually, but the job was still running after finish taking savepoint. I expect this job to be deleted, because the savepoint has been taken. jobStatus: jobId: 9de925e9d4a67e04ef6279925450907c jobName: sql-te-lab-s334c9 savepointInfo: