Session Window set max timeout

Is it possible to limit session windowing to a max of n seconds/hours etc? i.e. I would like a session window, but if a session runs for an unacceptable amount of time, I would like to close it. Thanks,

Savepoints - jobmanager.rpc.address

Hi, When taking a savepoint on AWS EMR I get the following error [hadoop@ip-10-12-169-172 ~]$ flink savepoint e14a6402b6f1e547c4adf40f43861c27 Retrieving JobManager. The program finished with the following exception: org.apache.flink

Flink Savepoint Config parameter

Hey, When running in EMR and taking a savepoint with flink cancel -s SAVEPOINT_DIR JOB_ID results in the following error Caused by: org.apache.flink.util.ConfigurationException: Config parameter 'Key: 'jobmanager.rpc.address' , default: null (deprecated keys: [])' is missing (hostname/addre

Re: Flink on AWS EMR Protobuf

; proceeding with flushing remote transports. End of LogType:jobmanager.log > On 6 Sep 2017, at 13:36, Aljoscha Krettek wrote: > > Hi, > > Could you please give a bit more context around that exception? Maybe a log > or a full stack trace. > > Best, > Aljoscha > &g

Flink on AWS EMR Protobuf

Hello, Has anybody experienced the following error on AWS EMR 5.8.0 with Flink 1.3.1 java.lang.ClassCastException: org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto cannot be cast to com.google.protobuf.Message Thanks,

Re: Error submitting flink job

Is this of any help https://stackoverflow.com/questions/33890759/how-to-specify-overwrite-to-writeastext-in-apache-flink-streaming-0-10-0 fs.overwrite-files: true in your flin

Re: Flink Elastic Sink AWS ES

ou try this. > > curl -XGET 'http:///_cat/nodes?v&h=ip,port' > > This should give you ip and port > > On Mon, Aug 28, 2017 at 3:42 AM, ant burton <mailto:apburto...@gmail.com>> wrote: > Hi Arpit, > > The response fromm _nodes doesn’t con

Re: Flink Elastic Sink AWS ES

Hi Arpit, The response fromm _nodes doesn’t contain an ip address in my case. Is this something that you experienced? > curl -XGET 'http:///_nodes' Thanks, > On 27 Aug 2017, at 14:32, ant burton wrote: > > Thanks! I'll check later this evening. > > On

Re: Flink Elastic Sink AWS ES

from your flink cluster nodes > > $ *telnet 9300* > > If this works then you can use above solution.​ > > > On Sun, Aug 27, 2017 at 4:09 AM, ant burton wrote: > >> Hi Ted, >> >> Changing the port from 9300 to 9200 in the example you provides causes >> t

Re: Flink Elastic Sink AWS ES

ml > > <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html> > > Please use Flink 1.3.1+ > > On Sat, Aug 26, 2017 at 3:00 PM, ant burton <mailto:apburto...@gmail.com>> wrote: > Hello, > > Has anybody been able to use th

Flink Elastic Sink AWS ES

Hello, Has anybody been able to use the Flink Elasticsearch connector to sink data to AWS ES. I don’t believe this is possible as AWS ES only allows access to port 9200 (via port 80) on the master node of the ES cluster, and not port 9300 used by the the Flink Elasticsearch connector. The err

Re: Access to datastream from BucketSink- RESOLVED

That one requires the implementation of 1 method with signature: > > Path getBucketPath(Clock clock, Path basePath, T element); > > which from what I understand from you requirements gives you access > to the element that you need. > > Cheers, > Kostas > >> On

Re: Access to datastream from BucketSink

T element); > > which from what I understand from you requirements gives you access > to the element that you need. > > Cheers, > Kostas > >> On Aug 16, 2017, at 3:31 PM, ant burton wrote: >> >> >> Thanks Kostas, >> >> I

Re: Access to datastream from BucketSink

et your > own Bucketer using the setBucketer method. You do not have to > implement your own sink from scratch. > > Kostas > >> On Aug 16, 2017, at 1:39 PM, ant burton > <mailto:apburto...@gmail.com>> wrote: >> >> or rather >> https://ci.ap

Re: Access to datastream from BucketSink

tml> > On 16 Aug 2017, at 12:24, Kostas Kloudas wrote: > > Hi Ant, > > I think you can do it by implementing your own Bucketer. > > Cheers, > Kostas > > . >> On Aug 16, 2017, at 1:09 PM, ant burton wrote: >> >> Hello, >> >> Gi

Re: Access to datastream from BucketSink

Kloudas wrote: > > Hi Ant, > > I think you can do it by implementing your own Bucketer. > > Cheers, > Kostas > > . >> On Aug 16, 2017, at 1:09 PM, ant burton wrote: >> >> Hello, >> >> Given >> >> //

Access to datastream from BucketSink

Hello, Given // Set StreamExecutionEnvironment final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); // Set checkpoints in ms env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); // Add source (input stre

Re: Flink Data Streaming to S3

g 2017, at 07:32, vinay patil wrote: > > Hi, > > Yes, I am able to write to S3 using DataStream API. > > I have answered you the approach on SO > > Regards, > Vinay Patil > > On Mon, Aug 14, 2017 at 4:21 AM, ant burton [via Apache Flink User Mailing > L

Flink Data Streaming to S3

Hello, Has anybody been able to write to S3 when using the data streaming API ? I’m having this problem https://stackoverflow.com/questions/45655850/flink-s3-write-fails-unable-to-load-aws-credentials-from-any-provider-in-the-cha

No file system found with scheme s3

Hello, After following the instructions to set the S3 filesystem in the documentation (https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/aws.html#set-s3-filesystem ) I encountered the