Data stream to write to multiple rds instances

Hi All, Is there any preferred way to manage multiple jdbc connections from flink..? I am new to flink and looking for some guidance around the right pattern and apis to do this. The usecase needs to route a stream to a particular jdbc connection depending on a field value.So the records are wri

Re: Flink, Yarn and MapR Kerberos issue

Hi Aniket, Thanks a lot for reporting this. I’m afraid this seems to be a bug with Flink on YARN’s Kerberos authentication. It is incorrectly checking for Kerberos credentials even for non-Kerberos authentication methods. I’ve filed a JIRA for this: https://issues.apache.org/jira/browse/FLINK-5

Re: Connecting workflows in batch

It looks like JobExecutionResult can be used here by using the accumulators? On Wed, Mar 1, 2017 at 8:37 AM, Aljoscha Krettek wrote: > I think right now the best option is the JobManager REST interface: > https://ci.apache.org/projects/flink/flink-docs- > release-1.3/monitoring/rest_api.html > >

Flink, Yarn and MapR Kerberos issue

I am trying to setup Flink 1.2 using yarn on MapR (v5.2.0). The MapR cluster, on which, I am trying to setup this is a secure cluster. But, this cluster does not use Kerberos. Mapr, by default, uses some variant of ssl

Re: Event-time tumbling window doesn't fire- Flink 1.2.0, Kafka-0.8_2.10

Hi Sam, do you have an idea what the timestamps in your data are, i.e. the tuple.f1 field you're extracting. What you could try is instead of windowing simply print your data and observe the timestamps. Maybe we can learn something from this about why the window doesn't trigger. Best, Aljoscha On

Re: Connecting workflows in batch

I think right now the best option is the JobManager REST interface: https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html You would have to know the ID of your job and then you can poll the status of your running jobs. On Mon, 27 Feb 2017 at 18:15 Mohit Anchlia wro

Re: unclear exception when writing to elasticsearch

Hi Martin, I followed your setup: 1. Maven java quick start archetype (Flink version 1.1.3) 2. Added `flink-connector-elasticsearch2_2.10` version 1.1.3 dependency 3. Ran the example in the Flink Elasticsearch docs against a Elasticsearch 2.4.1 installation and everything worked fine. Just to

Re: unclear exception when writing to elasticsearch

Hi Martin, Just letting you know I’m trying your setup right now, and will get back to you once I confirm the results. - Gordon On March 1, 2017 at 9:15:16 PM, Martin Neumann (mneum...@sics.se) wrote: I created the project using the maven archetype so I'm using the packaged version pulled by

Re: unclear exception when writing to elasticsearch

I created the project using the maven archetype so I'm using the packaged version pulled by maven. At this point, I just try to run it directly from inside the IDE (IntelliJ), mostly since I don't wan

Re: unclear exception when writing to elasticsearch

Did you build Flink from sources or are you using the packeged version? Because I had an annoying problem when compiling Flink with maven > 3.3. >From https://ci.apache.org/projects/flink/flink-docs- release-1.2/setup/building.html#dependency-shading: Maven 3.0.x, 3.1.x, and 3.2.x It is sufficient

Re: unclear exception when writing to elasticsearch

I tried to change the elastic search version to 2.4.1 which results in a new exception: Caused by: java.lang.NoSuchMethodError: > com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; > at org.elasticsearch.threadpool.ThreadPool.(ThreadPool.java:192) > at

Re: Checkpointing with RocksDB as statebackend

@vinay Can you try to not set the buffer timeout at all? I am actually not sure what would be the effect of setting it to a negative value, that can be a cause of problems... On Mon, Feb 27, 2017 at 7:44 PM, Seth Wiesman wrote: > Vinay, > > > > The bucketing sink performs rename operations dur

test 2

ignore this one 2

test

ignore

Re: [test][ignore] Sending an email to user@flink without being subscribed ...

As part of https://issues.apache.org/jira/browse/INFRA-13594, I'm sending another message from my other email address, to check out the rejection message :) On Thu, Feb 23, 2017 at 4:58 PM, Robert Metzger wrote: > Please ignore these messages. > > I'll talk to the ASF infra how we can resolve th

Re: Flink requesting external web service with rate limited requests

Hi, I assume the problem with the slow savepoints is because the checkpoint barriers which ensure the consistency of the savepoint get stuck between the records which are buffered due to backpressure. At some point the savepoint might get cancelled because it does not seem to make progress. You ca

Re: Re: Combine two independant streams

In a periodic watermark assigner, the getCurrentWatermark method is called in regular intervals (configurable via the ExecutionEnvironment). You could implement the method in a way that it increases the watermark if it was not increased by record timestamps after a certain number of invocations. On