I take time to read some source codes about the keyed stream
windowing, and I make below understanding:
a) the keyed stream would be split and dispatched to downstream tasks
in hash manner, and the hash base is the parallelism of the downstream
operator:
See
org.apache.flink.runtime.state.KeyGro
Hi,
When testing Flink 1.4 locally the error below keeps getting thrown. I've
followed the setup by moving the flink-s3-fs-presto.jar from the opt/
folder to the lib/ folder. Is there something additional I need to do?
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.flink.fs
Flink 1.4 added regex pattern matching for FlinkKafkaConsumer's which is a
neat feature. I would like to use this feature, but I'm wondering how that
impacts the FsStateBackend checkpointing mechanism. Before I would
subscribe to one topic and set a checkpoint path specific to that topic for
exampl
Thanks Eron
I have tried to read an EOF symbol and invoke FlinkKafkaConsumer's cancel
method, it doesn't work. But I invoke the method in a FlatMap operator
which is next to source operator, I guess that is the problem. I will try
your answer, thanks for your suggestion.
--
Sent from: http://a
> but soon later, no results produced, and flink seems busy doing
something forever.
Jinhua, don't know if you have checked these things. if not, maybe worth a
look.
have you tried to do a thread dump?
How is the GC pause?
do you see flink restart? check the exception tab in Flink web UI for you
Thanks Gordon for your explanation.
Mans
On Wednesday, December 20, 2017 2:16 PM, Tzu-Li (Gordon) Tai
wrote:
#yiv6533607487 body{font-family:Helvetica, Arial;font-size:13px;}Hi Mans,
What's the difference between an operator and a function ?
An operator in Flink needs to handle pr
Hi:
Apache Flink documentation
(https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/index.html)
indicates that a reduce function on a KeyedStream as follows:
A "rolling" reduce on a keyed data stream. Combines the current element with
the last reduced value and emit
Hi:
Referring to documentation
(https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/index.html)
for ConnectedStreams:
"Connects" two data streams retaining their types. Connect allowing for shared
state between the two streams.DataStream someStream = //...
DataStream
I checked the logs, but no information indicates what happens.
In fact, in the same app, there is another stream, but its kafka
source is low traffic, and I aggregate some field of that source too,
and flink gives correct results continuously.
So I doubt if keyby() could not handle high traffic we
