buckets / e.g. how do you avoid reading the same file
> on multiple parallel sinks?
> Is your sink running in a keyed context? Maybe even provide the general
> pipeline.
>
> On Thu, Mar 5, 2020 at 2:29 PM David Morin
> wrote:
>
>> Hello Arvid,
>>
>> After some in
>
> Did you restart from checkpoint or savepoint?
>
> On Thu, Mar 5, 2020 at 1:14 AM David Morin
> wrote:
>
>> Hello,
>>
>> I have this Exception in my datastream app and I can't find the root
>> cause.
>> I consume data from Kafka and it fails
Hello,
I have this Exception in my datastream app and I can't find the root cause.
I consume data from Kafka and it fails when I try to get a value from my
MapState in RocksDB.
It was working in previous release of my app but I can't find the cause of this
error.
java.lang.ArrayIndexOutOfBounds
ry mentioned
> specifying "-s 1" at starting the session would be enough, and it should
> work with all existing versions above (including) 1.8.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Tue, Feb 25, 2020 at 7:41 PM David Morin
> wrote:
>
>> Hi Gary,
>>
ink/flink-docs-stable/concepts/runtime.html#task-slots-and-resources
> [3]
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/operators/#task-chaining-and-resource-groups
>
> On Tue, Feb 25, 2020 at 10:28 AM David Morin
> wrote:
>
>> Hi Xintong,
>>
>
same JVM. Not that I'm aware
> of.
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Mon, Feb 24, 2020 at 8:43 PM David Morin
> wrote:
>
>> Hi,
>>
>> Thanks Xintong.
>> I've noticed than when I use yarn-session.sh with --slots (-s) para
eline
and of course to prevent from thread-safe issue
Do you think I cannot be confident on that behaviour ?
Regards,
David
On 2020/02/22 17:11:25, David Morin wrote:
> Hi,
> My app is based on a lib that is not thread safe (yet...).
> In waiting of the patch has been pushed, how can I b
Hi,
My app is based on a lib that is not thread safe (yet...).
In waiting of the patch has been pushed, how can I be sure that my Sink that
uses this lib is in one JVM ?
Context: I use one Yarn session and send my Flink jobs to this session
Regards,
David
e5a135c5ccfd6630634dd .
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface
>
> On Mon, Jan 6, 2020 at 1:16 PM David Morin
> wrote:
>
>> My naive solution can't work because a dump can be quite long.
>> So, yes I have to find a
tions/59201503/flink-kafka-gracefully-close-flink-consuming-messages-from-kafka-source-after-a
Any suggestions are welcomed
thx.
David
On 2020/01/06 09:35:37, David Morin wrote:
> Hi,
>
> Thanks for your replies.
> Yes Terry. You are right. I can try to create a custom sourc
at you need adjust the Kafka
> > consumer to have communication each other. When your begin the dump
> > process, you need to notify another CDC-topic consumer to wait idle.
> >
> >
> > Best,
> > Terry Wang
> >
> >
> >
> >> 2020年1月2日 1
Hi,
Is there a way to stop temporarily to consume one kafka source in streaming
mode ?
Use case: I have to consume 2 topics but in fact one of them is more
prioritized.
One of this topic is dedicated to ingest data from db (change data capture) and
one of them is dedicated to make a synchroniza
is enabled or it picks up the configured JobManager address from the
> configuration).
>
> Of course, you could also provide the JobManager address as a parameter.
>
> Cheers,
> Till
>
> On Mon, Sep 23, 2019 at 9:08 AM David Morin
> wrote:
>
> > Hi,
> >
&g
> > yourself.
> >
> > Regards,
> > Dian
> >
> > [1] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html
> > [2]
> > https://ci.apache.org/projects/flink/flink-docs-master/monitoring/rest_api.html#jars-jarid-run
> >
> > 在 201
Hi,
What is the best way to prevent from launching 2 jobs with the same name
concurrently ?
Instead of doing a check in the script that starts the Flink job, I would
prefer to stop a job if another one with the same name is in progress
(Exception or something like that).
David
cc_c', partition=7},
> KafkaTopicPartition{topic='gbl_auth_raw_occ_c', partition=14},
> KafkaTopicPartition{topic='gbl_auth_raw_occ_c', partition=13},
> KafkaTopicPartition{topic='gbl_auth_raw_occ_c', partition=12},
> KafkaTopicPartition{topic='
Hello Vishwas,
You can use a keytab if you prefer. You generate a keytab for your user and
then you can reference it in the Flink configuration.
Then this keytab will be handled by Flink in a secure way and TGT will be
created based on this keytab.
However, that seems to be working.
Did you check
d, official
> 1.9 should be out soon
> - Hive features are in beta in 1.9
>
> Feel free to open tickets if you have feature requests.
>
>
> On Fri, Aug 9, 2019 at 8:00 AM David Morin
> wrote:
>
>> Hi,
>>
>> I want to connect my Flink streaming job to Hive.
Hi,
I want to connect my Flink streaming job to Hive.
At the moment, what is the best way to connect to Hive.
Some features seems to be in development.
Some really cool features have been described here:
https://fr.slideshare.net/BowenLi9/integrating-flink-with-hive-xuefu-zhang-and-bowen-li-seatt
if you implement a RichFunction:
>
> getRuntimeContext().getMetricGroup().gauge("MyGauge", new Gauge()
> {
> @Override
> public Integer getValue() {
> return valueToExpose;
> }
> });
>
> Cheers,
> Till
>
> On Fri, Jun 21, 2019 at 12:36 A
Hi,
I want to create one metric related to the number of errors but in fact I
would like to add some contextual labels ?
What is the best way to do that ? gauge ?
How to create this kind of metric dynamically during the run of the task
(because open method is not possible because too early) ?
Than
21 matches
Mail list logo
