Thanks everyone for jumping in. BTW, we are using flink-1.4.1. deployment
is stand-alone mode.
here is the JIRA: https://issues.apache.org/jira/browse/FLINK-9693
On Fri, Jun 29, 2018 at 12:09 PM, Stephan Ewen wrote:
> Just saw Stefan's response, it is basically the same.
>
> We either null out
I am following this Flink Kafka example
https://stackoverflow.com/questions/31446374/can-anyone-share-a-flink-kafka-example-in-scala
This is my edited program. I am using Flink 1.5 in flink-scala shell
import org.apache.flink.streaming.api.scala._
import org.apache.flink.streaming.connectors.kaf
As it turned out in the application log, it could mot find yarn
configuraration! not anything to do with port
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.conf.YarnConfiguration.
I had installed flink *w**ithout bundled Hadoop* and my version of Hadoop
is 3.1
I went back a
Just saw Stefan's response, it is basically the same.
We either null out the field on deploy or archival. On deploy would be even
more memory friendly.
@Steven - can you open a JIRA ticket for this?
On Fri, Jun 29, 2018 at 9:08 PM, Stephan Ewen wrote:
> The problem seems to be that the Executi
The problem seems to be that the Executions that are kept for history
(mainly metrics / web UI) still hold a reference to their TaskStateSnapshot.
Upon archival, that field needs to be cleared for GC.
This is quite clearly a bug...
On Fri, Jun 29, 2018 at 11:29 AM, Stefan Richter <
s.rich...@dat
Thanks! I'm working on a way to deliver the data in order (or closer to in
order) and deliver watermarks more often. I'll let you know my results.
On Thu, Jun 28, 2018 at 5:36 AM, Fabian Hueske wrote:
> In a nutshell the Over operator works as follows:
> - When a row arrives it is put into a Map
(fixed formatting)
Hello,
We have experienced some problems where a task just hangs without showing
any kind of log error while other tasks running in the same task manager
continue without problems. When these tasks are restarted the task manager
gets killed and shows several errors similar to
Thanks!
On Thu, Jun 28, 2018 at 7:32 PM Tzu-Li (Gordon) Tai
wrote:
> Hi,
>
> It should be possible to deploy a single Flink cluster across
> geo-distributed nodes, but Flink currently offers no optimization for such
> a specific use case.
> AFAIK, the general pattern for dealing with geographica
Hello,We have experienced some problems where a task just hangs without
showing any kind of log error while other tasks running in the same task
manager continue without problems. When these tasks are restarted the task
manager gets killed and shows several errors similar to these
ones:[Canceler/In
port should be consistent.
1> nc -l 2219
2>./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 2219
On Fri, Jun 29, 2018 at 9:21 PM, Mich Talebzadeh
wrote:
> thanks Hequn.
>
> This the port I started with
>
> hduser@rhes75: /data6/hduser/flink-1.5.0> nc -l 2219
> hello
>
> and a
Hi Rohil,
this sounds a little bit strange. If the GoogleHadoopFileSystem jar is on
the classpath and the implementation is specified in core-site.xml, then
the Hadoop Filesystem should be able to load the GCS filesystem. I just
tried it out locally (without K8s though) and it seemed to work.
Cou
thanks Hequn.
This the port I started with
hduser@rhes75: /data6/hduser/flink-1.5.0> nc -l 2219
hello
and as I expected I should collect from port 2219? However, I did what you
suggested
./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 2199
Starting execution of program
-
Hi Mich,
You port is not matching. Start netcat with "nc -l 2219 &", but run flink
job with "--port 2199".
On Fri, Jun 29, 2018 at 8:40 PM, Mich Talebzadeh
wrote:
> Hi,
>
> I have installed flink 1.5 in standalone mode and trying a basic run as
> per this example
>
> https://ci.apache.org/pr
Hi,
I have installed flink 1.5 in standalone mode and trying a basic run as per
this example
https://ci.apache.org/projects/flink/flink-docs-release-1.2/quickstart/setup_quickstart.html
started netcat on port 2199
nc -l 2219 &
Run the example
./bin/flink run examples/streaming/SocketWindowWo
Thank you Hequn,
I got it working. Here is the tumbling window query, in both SQL and Table
API.
I'm getting same results with these:
SQL API
tableEnvironment.registerDataStream("pageViews", eventStream,
"pageViewTime.rowtime, username, url");
String continuousQuery =
"SELECT TUMBLE_END(page
Hi Steven,
from your analysis, I would conclude the following problem. ExecutionVertexes
hold executions, which are bootstrapped with the state (in form of the map of
state handles) when the job is initialized from a checkpoint/savepoint. It
holds a reference on this state, even when the task i
By the way, I do not think below is a correct way. As @ Fabian said. The
BucketingSink closes files once they reached a certain size (BatchSize) or have
not been written to for a certain amount of time (InactiveBucketThreshold).
> . If we can close
> file during checkpoint, then the result is a
Hi Xilang
I think you are doing a together work with the offline team. Also what you said
ETL, ETL team want to use the data in HDFS. I would like to confirm one
question from you. What is their scheduling time for every job ? 5mins or 10
mins ?
> My user case is we read data from message que
18 matches
Mail list logo
