Re: Stopping a job

Hi Arvid:    Thanks for the links.   A few questions: 1. Is there any particular interface in 1.9+ that identifies the source as stoppable ?2. Is there any distinction b/w stop and cancel  in 1.9+ ?3. Is there any list of sources which are documented as stoppable besides the one listed in your

Flink not restoring from checkpoint when job manager fails even with HA

Hi, We are running Flink in K8S. We used https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/jobmanager_high_availability.html to set high availability. We set max number of retries for a task to 2. After task fails twice and then the job manager fails. This is expected. But

Failed to deserialize Avro record

We are using AvroRowDeserializationSchema with Kafka Table source to deserialize the messages. Application failed with "Failed to deserialize Avro record." for different messages it seems. Caused by: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -26 Caused by: java.

Re: Stopping a job

Yes, it seems as if FlinkKinesisConsumer does not implement it. Here are the links to the respective javadoc [1] and code [2]. Note that in later releases (1.9+) this interface has been removed. Stop is now implemented through a cancel() on source level. In general, I don't think that in a Kinesi

Re: Stopping a job

Hi Arvid: I check the link and it indicates that only Storm SpoutSource, TwitterSource and NifiSource support stop.    Does this mean that FlinkKinesisConsumer is not stoppable ? Also, can you please point me to the Stoppable interface mentioned in the link ?  I found the following but am not su

Re: Flink s3 streaming performance

Thanks Arvid! Will try to increase the property you recommended and will post the update. On Sat, Jun 6, 2020, 7:33 AM Arvid Heise wrote: > Hi Venkata, > > you can find them on the Hadoop AWS page (we are just using it as a > library) [1]. > > [1] > https://hadoop.apache.org/docs/current/hadoop

Re: Run command after Batch is finished

It would run in the client side where ExecutionEnvironment is created. Mark Davis 于2020年6月6日周六 下午8:14写道： > Hi Jeff, > > Thank you very much! That is exactly what I need. > > Where the listener code will run in the cluster deployment(YARN, k8s)? > Will it be sent over the network? > > Thank you!

Re: Run command after Batch is finished

Hi Jeff, Thank you very much! That is exactly what I need. Where the listener code will run in the cluster deployment(YARN, k8s)? Will it be sent over the network? Thank you! Mark ‐‐‐ Original Message ‐‐‐ On Friday, June 5, 2020 6:13 PM, Jeff Zhang wrote: > You can try JobListener

Re: Flink s3 streaming performance

Hi Venkata, you can find them on the Hadoop AWS page (we are just using it as a library) [1]. [1] https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#General_S3A_Client_configuration On Sat, Jun 6, 2020 at 1:26 AM venkata sateesh` kolluru < vkollur...@gmail.com> wrote: