Best practice to write data from a stream to non-relational, distributed database (hbase)

Hi, what is the prefered way to wirte streaming data to hbase? Rolling File Sink or Streaming File Sink? How can I configure this (open the connection with conf, and the write handling(key,data)? What do I have to consider about the partitions? I prefer a write pro partition. Thanks! Marke

Re: Where is the "Latest Savepoint" information saved?

Hey Hao and Paul, 1) Fetch checkpoint info manually from ZK (problematic, not recommended) - As Paul pointed out, this is problematic as the node is a serialized pointer (StateHandle) to a CompletedCheckpoint in the HA storage directory and not a path [1]. - I would not recommend this approach at

Re: Where is the "Latest Savepoint" information saved?

This is great, I will try option 3 and let you know. Can I log some message so I know job is recovered from the latest savepoint? On Sun, Nov 11, 2018 at 10:42 AM Ufuk Celebi wrote: > Hey Hao and Paul, > > 1) Fetch checkpoint info manually from ZK (problematic, not recommended) > - As Paul point

Re: java.io.IOException: NSS is already initialized

Hey Hao, 1) Regarding Hadoop S3: are you using the repackaged Hadoop S3 dependency from the /opt folder of the Flink distribution? Or the actual Hadoop implementation? If latter, would you mind also running it with the one that comes packaged with Flink? For this you can remove all Hadoop-related

Re: Where is the "Latest Savepoint" information saved?

I think there should be a log message, but I don't know what the exact format is (you would need to look through it and search for something related to CompletedCheckpointStore). An alternative is the web UI checkpointing tab. It shows the latest checkpoint used for restore of the job. You should

Re: Where is the "Latest Savepoint" information saved?

Thanks, I'll check it out. On Sun, Nov 11, 2018 at 10:52 AM Ufuk Celebi wrote: > I think there should be a log message, but I don't know what the exact > format is (you would need to look through it and search for something > related to CompletedCheckpointStore). > > An alternative is the web UI

Any examples on invoke the Flink REST API post method ?

HI Experts, I am trying to trigger a savepoint from Flink REST API on version 1.6 , in the document it shows that I need to pass a json as a request body { "type" : "object”, "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:savep

Re: Any examples on invoke the Flink REST API post method ?

Hi Henry, Maybe Gary can help you, ping him for you. Thanks, vino. 徐涛 于2018年11月12日周一 下午12:45写道： > HI Experts, > I am trying to trigger a savepoint from Flink REST API on version 1.6 , in > the document it shows that I need to pass a json as a request body > { > "type" : "object”, > "id" : >

RE: RichInputFormat working differently in eclipse and in flink cluster

Hi Till, We are using 1.4.0. We have not tried this any other releases. We will try this on 1.6.2 and see what happens. Thank you. Regards, Teena From: Till Rohrmann Sent: 07 November 2018 20:23 To: Teena Kappen // BPRISE Cc: user Subject: Re: RichInputFormat working differently in eclipse

Ingesting data from an API

Hi, I have an API that emits output that I want to use as a data source for Flink. I have written a custom source function that is as follows - public class DynamicRuleSource extends AlertingRuleSource { private ArrayList rules = new ArrayList(); public void run(SourceContext ctx) thro