Re: datadog metrics

2020-03-08 Thread Fanbin Bu
quote from the following link: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/SQL-Query-named-operator-exceeds-80-characters-td24807.html#a24818 "This is a safeguard in the metric system to prevent extremely long names (as these could cause the reporting to fail); so long as t

datadog metrics

2020-03-08 Thread Fanbin Bu
Hi, Has anybody seen this error before and what is the suggested way to solve it? 2020-03-07 02:54:34,100 WARN org.apache.flink.metrics.datadog.DatadogHttpClient- Failed to send request to Datadog (response was Response{protocol=http/1.1, code=413, message=Request Entity Too Large, u

Flink Kafka consumer auto-commit timeout

2020-03-08 Thread Rong Rong
Hi All, I would like to bring back this discussion which I saw multiple times in previous ML threads [1], but there seem to have no solution if checkpointing is disabled. All of these ML reported exceptions have one common pattern: > *INFO* org.apache.kafka.clients.consumer.internals.AbstractCoo

Re:Re: Error handler strategy in Flink Kafka connector with json format

2020-03-08 Thread sunfulin
yep. Glad to see the progress. Best At 2020-03-09 12:44:05, "Jingsong Li" wrote: Hi Sunfulin, I think this is very important too. There is an issue to fix this[1]. Is that meet your requirement? [1] https://issues.apache.org/jira/browse/FLINK-15396 Best, Jingsong Lee On Mon, Mar

Re: Error handler strategy in Flink Kafka connector with json format

2020-03-08 Thread Jingsong Li
Hi Sunfulin, I think this is very important too. There is an issue to fix this[1]. Is that meet your requirement? [1] https://issues.apache.org/jira/browse/FLINK-15396 Best, Jingsong Lee On Mon, Mar 9, 2020 at 12:33 PM sunfulin wrote: > hi , community, > I am wondering if there is some config

Error handler strategy in Flink Kafka connector with json format

2020-03-08 Thread sunfulin
hi , community, I am wondering if there is some config params with error handler strategy as [1] refers when defining a Kafka stream table using Flink SQL DDL. For example, the following `json.parser.failure.strategy' can be set to `silencly skip` that can skip the malformed dirty data proces

Re: EXTERNAL: Re: Should I use a Sink or Connector? Or Both?

2020-03-08 Thread Jark Wu
Hi Fernando, Thanks for reporting back. >From my point of view, this is a short-comming of current elasticsearch connector, i.e. out-of-box doesn't work. I created FLINK-16495 [1] to improve this to have a default flush interval. Best, Jark [1]: https://issues.apache.org/jira/browse/FLINK-16495

Re: Understanding n LIST calls as part of checkpointing

2020-03-08 Thread Yun Tang
Hi Piyush Which version of Flink do you use? After Flink-1.5, Flink would not call any "List" operation on checkpoint side with FLINK-8540 [1]. The only left "List" operation would be used when reading files in file input format. In a nut shell, these "List" calls should not come from Flink if

Re: (DISSCUSS) flink cli need load '--classpath' files

2020-03-08 Thread Yang Wang
I tend to not change the current behavior. For other framework(e.g. hadoop, yarn), the arguments after user jar are also parsed by user `main()`, not the framework client. Best, Yang tison 于2020年3月6日周五 下午10:55写道: > Hi Jingsong, > > I think your propose is "--classpath can occur behind the jar

Re: Flink Conf "yarn.flink-dist-jar" Question

2020-03-08 Thread Yang Wang
Hi Hailu, tison, I created a very similar ticket before to accelerate Flink submission on Yarn[1]. However, we do not get a consensus in the PR. Maybe it's time to revive the discussion and try to find a common solution for both the two tickets[1][2]. [1]. https://issues.apache.org/jira/browse/F

Re: Deploy flink-dist_2.11 fail in flink-10.0.0

2020-03-08 Thread Yang Wang
It seems that execute goal "create-build-target-link" failed. So could you login your jenkins worker and manually test the link operation. ln -sfn ${project.basedir}/target/flink-${project.version}-bin/flink-${project.version} ${project.basedir}/../build-target Best, Yang 刘建刚 于2020年3月7日周六 下午

[ANNOUNCE] Weekly Community Update 2020/10

2020-03-08 Thread Konstantin Knauf
Dear community, happy to share this week's community update featuring a set of exciting proposals from very different areas of Apache Flink, e.g. * Initial GPU Support * Flink Development Blog * "LIKE" in "CREATE TABLE" and a bit more. Enjoy. Flink Development == * [deployment] In

Re: [DISCUSS] FLIP-111: Docker image unification

2020-03-08 Thread Konstantin Knauf
Hi Andrey, thanks a lot for this proposal. The variety of Docker files in the project has been causing quite some confusion. For the entrypoint, have you considered to also allow setting configuration via environment variables as in "docker run -e FLINK_REST_BIN_PORT=8081 ..."? This is quite comm