Re: Filescheme GS not found sometimes - inconsistent exceptions for reading from GCS

to a bug in the > de/serializer or the network stack. > > Do you actually have a minimal working example that you can share > (either privately with me, or here) and shows this error? > > > Nico > > On 29/08/18 14:19, Encho Mishinev wrote: > > Hello, > > > >

Acknowledging Pubsub messages in Flink Runner

Hello, I have a simple question - when does the Flink Runner for Apache Beam acknowledge Pubsub messages when using PubsubIO? Thanks

History Server in Kubernetes

Hello, I am struggling to find how to run a history server in Kubernetes. The docker image takes an argument that starts a jobmanager or a taskmanager, but no history server. What's the best way to set up one in K8S? Thanks, Encho

Filescheme GS not found sometimes - inconsistent exceptions for reading from GCS

Hello, I am using Flink 1.5.3 and executing jobs through Apache Beam 2.6.0. One of my jobs involves reading from Google Cloud Storage which uses the file scheme "gs://". Everything was fine but once in a while I would get an exception that the scheme is not recognised. Now I've started seeing them

Re: JobGraphs not cleaned up in HA mode

e standby JobManager thinks that it is also the leader. > Could you maybe share the logs of the two JobManagers/ClusterEntrypoints > with us? > > Running only a single JobManager/ClusterEntrypoint in HA mode via a > Kubernetes Deployment should do the trick and there is nothing wrong

Re: JobGraphs not cleaned up in HA mode

in an YARN cluster but does not specify such option for Kubernetes. Is there anything that can go wrong with this approach? Thanks On Wed, Aug 29, 2018 at 11:10 AM Encho Mishinev wrote: > Hi, > > Unfortunately the thing I described does indeed happen every time. As > mentioned in the

Re: JobGraphs not cleaned up in HA mode

raphs in both Zookeeper and HDFS remained.* > > Is it necessarily happening every time? > > In the Standalone cluster, the problems we encountered were sporadic. > > Thanks, vino. > > Encho Mishinev 于2018年8月28日周二 下午8:07写道： > >> Hello Till, >> >> I spe

Re: JobGraphs not cleaned up in HA mode

s "/jobgraph". >> Another solution, modify the source code, rudely modify the cleanup mode >> to the synchronous form, but the flink operation Zookeeper's path needs to >> obtain the corresponding lock, so it is dangerous to do so, and it is not >> recommended.

Re: JobGraphs not cleaned up in HA mode

; > vino yang 于2018年8月28日周二 上午9:49写道： > >> Hi Encho, >> >> This is a problem already known to the Flink community, you can track its >> progress through FLINK-10011[1], and currently Till is fixing this issue. >> >> [1]: https://issues.apache.org/jira/brows

JobGraphs not cleaned up in HA mode

I am running Flink 1.5.3 with two job managers and two task managers in Kubernetes along with HDFS and Zookeeper in high-availability mode. My problem occurs after the following actions: - Upload a .jar file to jobmanager-1 - Run a streaming job from the jar on jobmanager-1 - Wait for 1 or 2 chec