the program is composed by:
6 Kafka /source/ connector with custom timestamp and watermark /extractor/
and /map/ function each.
then I use 6 instance of an external library called flink-htm (quite heavy)
moreover I have 6 /process/ method and 2 /union/ method to merge result
streams.
Finally I hav
Btw, what load are you putting on the cluster, i.e. what is your computation?
If you don't have load, the cluster and job just keep on running, right?
Best,
Aljoscha
> On 19. Sep 2017, at 12:00, AndreaKinn wrote:
>
> Thank you, unfortunately it had no effects.
>
> As I add more load on the co
Thank you, unfortunately it had no effects.
As I add more load on the computation appears the error taskmanager killed
on the node on use, without calling other nodes to sustain the computation.
I also increased
akka.watch.heartbeat.interval
akka.watch.heartbeat.pause
akka.transport.heartbeat.i
Hi,
You can set it to G1GC with the following setting. In my example it is
only for the taskmanager, but env.java.opts should work in the same way.
env.java.opts.taskmanager: -XX:+UseG1GC
Update:
Following other discussions I even tried to reduce memory.fraction to 10%
without success.
How can I set G1 as garbage collector?
the key is env.java.opts but the value?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
I tried also to set the only job manager on the first node and reconfiguring
the cluster admitting just two task manager. In this way I obtain
immediately a NoResourceAvailable error
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
I investigated the semantics of cpu percentage on top. I have to correct my
sentence:
When I start the program it has a peak at 160% (max is 200%), but after a
second it falls down until the 4%.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
I think it might be that the computation is to CPU heavy, which makes the
TaskManager unresponsive to any JobManager messages and so the JobManager
thinks that the TaskManager is lost.
@Till, do you have another idea about what could be going on?
> On 15. Sep 2017, at 13:52, AndreaKinn wrote:
the job manager log probably is more interesting:
2017-09-15 12:47:45,420 WARN org.apache.hadoop.util.NativeCodeLoader
- Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2017-09-15 12:47:45,650 INFO org.apache.flink.runt
This is the log:
2017-09-15 12:47:49,143 WARN org.apache.hadoop.util.NativeCodeLoader
- Unable to load native-hadoop library for your platform... using
builtin-java classe$
2017-09-15 12:47:49,257 INFO
org.apache.flink.runtime.taskmanager.TaskManager -
---
Hi,
Can you check in the TaskManager logs whether there is any message that
indicates why the TaskManager was lost? Also, there might be information in
your machine logs, i.e. "dmesg" or /var/log/messages or some such.
Best,
Aljoscha
> On 14. Sep 2017, at 22:28, AndreaKinn wrote:
>
> P.S.: I
P.S.: I tried on my laptop with the same configuration of the job-task
manager (ram, slots, parallelism etc...) and it works perfectly.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Update.
the previous error probably was caused because I didn't restart the cluster
before a re-execution. (maybe)
Then, I tried to execute the program on a cluster of one node on my laptop
and, after solved some little issues, everything works fine.
Now I'm trying to deploy the same jar on the r
The error message says that the total number of slots is 0,
It is thus very likely that no task manager is connected to the jobmanager.
How exactly are you starting the cluster?
On 14.09.2017 18:03, AndreaKinn wrote:
Hi,
I'm executing a program on a flink cluster.
I tried the same on a local no
14 matches
Mail list logo
