Hi,
I am using Flink on yarn cluster. My job was running for 2-3 days. After
that it failed with two errors
org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
Error at remote task manager 'ip-xx.xx.xx.xxx'.
at
org.apache.flink.runtime.io.network.netty.PartitionR
There are 2 flaws in your code:
Let's start with the fundamental one:
At no point do you associate your mapConf with the flatMap or even the job.
THeoretically you should add it to the flatMap using
flatMap(...).withConfiguration(mapConf) method.
But here's is the second a more subtle flaw:
Hi,
I am trying to send some static integer values down to each map function,
using the following code
public static void main(String[] args) throws Exception {
ParameterTool params = ParameterTool.fromArgs(args);
String fi
Hello again! :)
I have been checking the solution that you proprosed and but I don't really
get how the KeyValueState helps on it. Could you please explain it a little
bit more?
I have drawn a diagram to make what I want clear, notice that the middle
table doesn't need to be a table, it is just w
Thank you for the advice!
Now I have a new question. I read the source[1] streaming env exploits
FileSourceFunction, which inherits RichParallelSourceFunction, to create
split input[2]. I know I can set parallelism in streaming env, but any way
I can verify that at runtime the split files or the f
a very good book on big data interview preparation
https://notionpress.com/read/big-data-interview-faqs
chaturvedi
ExecutionEnvironment.readTextFile will read the file in parallel.
On 28.05.2016 09:59, David Olsen wrote:
After searching on the internet I still do not find the answer (with
key word like 'apache flink parallel read text') I am looking for. So
asking here before jumping to write code ...
My
After searching on the internet I still do not find the answer (with key
word like 'apache flink parallel read text') I am looking for. So asking
here before jumping to write code ...
My problem is I want to a read text file or split text files (from local
file system). Therefore I want to paralle
