Looks like using the following overload of
StreamExecutionEnvironment.addSource which takes a TypeInformation as well,
does the trick ..
env.addSource(
FlinkSource.collectionSourceFunction(data),
TypeInformation.of(Data.class)
)
regards.
On Mon, Aug 26, 2019 at 11:24 AM Debasish Ghosh
wrote
oh .. and I am using Flink 1.8 ..
On Mon, Aug 26, 2019 at 12:09 AM Debasish Ghosh
wrote:
> Thanks for the feedback .. here are the details ..
>
> Just to give u some background the original API is a Scala API as follows
> ..
>
> final def readStream[In: TypeInformation: DeserializationSchema](in
[image: image.png]
When there are new events the old events just get stuck for many hours
(more than a day). So if there is a buffering going on it seems it is not
time based but size based (?). Looks like unless the buffered events exceed
a certain threshold they don't get flushed out (?). Is tha
Thanks for the feedback .. here are the details ..
Just to give u some background the original API is a Scala API as follows ..
final def readStream[In: TypeInformation: DeserializationSchema](inlet:
CodecInlet[In]): DataStream[In] =
context.readStream(inlet)
and the *Scala version of the co
I am not sure how the function `readStream` is implemented (also which
version of Flink are you using?).
Can you share more information on your code blocks and exception logs?
Also to answer your question, DataStream return type is determined by its
underlying transformation, so you cannot set it
Hi,
Thanks for the response. Can you point me to some examples of such strategy?
On Sat, Aug 24, 2019, 06:01 Eduardo Winpenny Tejedor <
eduardo.winpe...@gmail.com> wrote:
> Hi Prakhar,
>
> Everything is probably working as expected, if a partition does not
> receive any messages then the waterma
I have an issue with tumbling windows running in parallel.
I run a Job on a set of CSV files.
When the parallelism is set to 1. I get the proper results.
While it runs in parallel. I get no output.
Is it due to the fact the parallel streams take the MAX(watermark) from all
the parallel sou
You can also link at runtime by providing the path to the dylib by adding
-Djava.library.path= in jvm options in the task manager
On Sat, Aug 24, 2019 at 9:11 PM Zhu Zhu wrote:
> Hi Abhishek,
>
> You need to export the environment variables on all the worker
> machines(not the machine to submit
Hello,
I'm trying to use hivecatalog in flink1.9. I modified the yaml file like this:
catalogs:
- name: mynewhive
type: hive
hive-conf-dir: /home/user/Downloads/apache-hive-1.2.2-bin/conf
default-database: myhive
But when I try to run ./sql-client.sh embedded I get this error:
Any idea on how I can use log back instead ?
On Fri, Aug 23, 2019 at 1:22 PM Vishwas Siravara
wrote:
> Hi ,
> From the flink doc , in order to use logback instead of log4j " Users
> willing to use logback instead of log4j can just exclude log4j (or delete
> it from the lib/ folder)."
> https://c
Got it.Thank you
On Thu, Aug 22, 2019 at 8:54 PM Congxian Qiu wrote:
> Hi, Vishwas
>
> As Zhu Zhu said, you can set "state.checkpoints.num-retained"[1] to
> specify the maximum number of completed checkpoints to retain.
> maybe you can also ref the external checkpoint cleanup type[2] config for
Nope, I submitted it throw the flink job master itself by running flink run -c
sandbox.jar
Best regards
Eyal Peer / Data Platform Developer
[cid:image001.png@01D55B62.D5E98030]
From: miki haiat
Sent: Sunday, August 25, 2019 4:21 PM
To: Eyal Pe'er
Cc: user
Subject: Re: timeout error while co
I'm trying to understand.
Did you submitted your jar throw the flink web UI ,
And then you got the time out error ?
On Sun, Aug 25, 2019, 16:10 Eyal Pe'er wrote:
> What do you mean by “remote cluster”?
>
> I tried to run dockerized Flink version (
> https://ci.apache.org/projects/flink/flink-doc
What do you mean by “remote cluster”?
I tried to run dockerized Flink version
(https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html)
on a remote machine and to submit a job that supposed to communicate with
Kafka, but still I cannot access the topic.
Best regards
E
Did you try to submit it to remote cluster ?
On Sun, Aug 25, 2019 at 2:55 PM Eyal Pe'er wrote:
> BTW, the exception that I see in the log is: ERROR
> org.apache.flink.runtime.rest.handler.job.JobDetailsHandler - Exception
> occurred in REST handler…
>
> Best regards
>
> Eyal Peer */ *Data Pl
Replication factor is 1. In most of my topics this is the case.
Is it a problem to consume events from non-replicated topics ?
Best regards
Eyal Peer / Data Platform Developer
[cid:image001.png@01D55B59.352FCE00]
From: Yitzchak Lieberman
Sent: Sunday, August 25, 2019 3:13 PM
To: Eyal Pe'er
Cc:
What is the topic replication factor? how many kafka brokers do you have?
I were facing the same exception when one of my brokers was down and the
topic had no replica (replication_factor=1)
On Sun, Aug 25, 2019 at 2:55 PM Eyal Pe'er wrote:
> BTW, the exception that I see in the log is: ERROR
>
BTW, the exception that I see in the log is: ERROR
org.apache.flink.runtime.rest.handler.job.JobDetailsHandler - Exception
occurred in REST handler…
Best regards
Eyal Peer / Data Platform Developer
[cid:image001.png@01D55B55.21C20990]
From: Eyal Pe'er
Sent: Sunday, August 25, 2019 2:20 PM
To:
Hi,
I removed that dependency, but it still fails.
The reason why I used Kafka 1.5.0 is because I followed a training which used
it (https://www.baeldung.com/kafka-flink-data-pipeline).
If needed, I can change it.
I’m not sure, but maybe in order to consume events from Kafka 0.9 I need to
connec
19 matches
Mail list logo
