Questions about user doc.

Hey guys, In this document : https://ci.apache.org/projects/flink/flink-docs-stable/internals/job_scheduling.html , there is a line in the beginning of the scheduling section which says that : "A pipeline consists of multiple successive tasks, such as the *n-th* parallel instance of a MapFunction t

Re: How to run multiple InputFormat on multiple nodes

Hi Han, from your description, looks that the 3 source nodes you created are running in the same shared slot, thus in the same node. So far DataSet API does not support customizing slot sharing group. So we may not enforce the sources to be in different shared groups to avoid that. But I think the

Re:Job leak in attached mode (batch scenario)

Hi, Qi As far as I know, there is no such mechanism now. To achieve this, I think it may be necessary to add a REST-based heartbeat mechanism between Dispatcher and Client. At present, perhaps you can add a monitoring service to deal with these residual Flink clusters. Best, Haibo At 2019-0

How to run multiple InputFormat on multiple nodes

Dears, I have installed the Flink standalone cluster with 3 nodes, and I want to select some data from database. For high performance, I split the one sql to many, and I want these many sqls can run on the 3 nodes distributed.But actually these sqls are runing on the same node. Here is my codes

Checkpoints timing out for no apparent reason

We have an issue with a job when it occasionally times out while creating snapshots for no apparent reason: Details: - Flink 1.7.2 - Checkpoints are saved to S3 with presto - Incremental checkpoints are used What might be the cause of this issue? It feels like some internal s3 client timeout i

Cannot access the data from Hive-Tables in Blink

Hello, I'm trying to use BigBench queries on Blink in Cloudera. I have defined a catalog in YAML-file and can see my Hive-tables in SQL-client. But I can't see the data of the tables, or run any other SQL-Query. I get this error: [ERROR] Could not execute SQL statement. Reason: org.apache.flink

Trying to Convert Tuple[Boolean,Row] to [Row]

Hi All, I am trying to convert a Tuple[Boolean,Row] to Row using map function, I am getting this error asking me for InferedR , what is InferedR in FLink? val mymapFunction: MapFunction[tuple.Tuple2[Boolean, Row],AnyVal] = new MapFunction[tuple.Tuple2[Boolean, Row],AnyVal]() { overrid

Re: table toRetractStream missing last record and adding extra column (True)

windows for question 1 or question 2 or both ? Thanks Sri On Tue, Jul 16, 2019 at 12:25 PM taher koitawala wrote: > Looks like you need a window > > On Tue, Jul 16, 2019, 9:24 PM sri hari kali charan Tummala < > kali.tumm...@gmail.com> wrote: > >> Hi All, >> >> I am trying to write toRetractSre

Re: table toRetractStream missing last record and adding extra column (True)

Looks like you need a window On Tue, Jul 16, 2019, 9:24 PM sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Hi All, > > I am trying to write toRetractSream to CSV which is kind of working ok but > I get extra values like True and then my output data values. > > Question1 :- > I don

Re: org.apache.flink.table.api.TableException: Only tables that originate from Scala DataStreams can be converted to Scala DataStreams.

is this a Bug in Flink Scala? Full code and Maven POM:- package com.aws.examples.kinesis.consumer.TransactionExample import java.lang import java.util.Properties import com.amazonaws.auth.DefaultAWSCredentialsProviderChain import org.apache.flink.api.common.functions.MapFunction import org.apac

Fwd: org.apache.flink.table.api.TableException: Only tables that originate from Scala DataStreams can be converted to Scala DataStreams.

> > > Hi All, > > I am trying to convert sql query results value to distinct and writing to > CSV which is failing with below error. > > *Exception in thread "main" org.apache.flink.table.api.TableException: > Only tables that originate from Scala DataStreams can be converted to Scala > DataStreams

Re: Converting Metrics from a Reporter to a Custom Events mapping

You can configure multiple reporters, so just configured both the reporter that the app users want and your own which does the transformation and sending to kinesis. On 16/07/2019 09:37, Haibo Sun wrote: Hi, Vijay Or can you implement a Reporter that transforms the metrics and sends them dir

Re:Converting Metrics from a Reporter to a Custom Events mapping

Hi, Vijay Or can you implement a Reporter that transforms the metrics and sends them directly to a Kinesis Stream? Best, Haibo At 2019-07-16 00:01:36, "Vijay Balakrishnan" wrote: Hi, I need to capture the Metrics sent from a Flink app to a Reporter and transform them to an Events API for

Re: Automatic deployment of new version of streaming stateful job

Hi Maxim You could write a script yourself which triggers the cancel with savepoint and then starts a new version using the savepoint that was created during the cancel. However, I’ve built a tool that allows you to perform these steps more easily:  https://github.com/ing-bank/flink-deployer. T