Hi all,
We are currently using BucketingSink to save data into HDFS in parquet format.
But when the flink job was canceled, we always got Exception in BucketingSink’s
close method. The detailed exception info is as below:
[ERROR] [2017-09-26 20:51:58,893]
[org.apache.flink.streaming.runtime.tas
Hai Zhou UTC+8 created FLINK-7697:
-
Summary: Add metrics for Elasticsearch Sink
Key: FLINK-7697
URL: https://issues.apache.org/jira/browse/FLINK-7697
Project: Flink
Issue Type: Wish
Hi folks, I'm happy to share with you a draft of a FLIP for service
authorization. As I mentioned at the top of this thread, the goal is to
protect a deployed Flink cluster/session from unauthorized use. In the
doc, I propose the use of SSL client authentication for internal
communication, plus
Thank you for the response!
It turns out actually that elastic 5.5.1 was throwing that exception but
when I used the elastic 5.1.1 it works fine.
org.elasticsearch
elasticsearch
5.1.1
Like Fabian Hueske mentioned previously it is not working with 5.5.1 and
probably elastic versions
Fabian Hueske created FLINK-7696:
Summary: Add projection push-down support for TableSources with
time attributes
Key: FLINK-7696
URL: https://issues.apache.org/jira/browse/FLINK-7696
Project: Flink
Till Rohrmann created FLINK-7695:
Summary: Port JobConfigHandler to new REST endpoint
Key: FLINK-7695
URL: https://issues.apache.org/jira/browse/FLINK-7695
Project: Flink
Issue Type: Sub-task
Hi,
I did some experimenting and found something that is interesting and looks
off.
So the only problem is when the ZK leader is restarted, not related to any
retry/reconnect logic (not affected by the timeout setting).
I think the following is happening (based on the logs
https://gist.github.com
Bowen Li created FLINK-7694:
---
Summary: Port JobMetricsHandler to new REST handler
Key: FLINK-7694
URL: https://issues.apache.org/jira/browse/FLINK-7694
Project: Flink
Issue Type: Sub-task
Hi Chesnay,
That sounds great to me. I think I will be interested in it.
Best Regards,
Tony Wei
2017-09-26 21:57 GMT+08:00 Chesnay Schepler :
> Hello,
>
> i see the value in supporting this, and it's also quite easy to do so
> actually.
>
> I've filed FLINK-7692, containing instructions on how
Till Rohrmann created FLINK-7693:
Summary: Adapt web frontend to use proper DELETE REST call for job
termination
Key: FLINK-7693
URL: https://issues.apache.org/jira/browse/FLINK-7693
Project: Flink
Hello,
i see the value in supporting this, and it's also quite easy to do so
actually.
I've filed FLINK-7692, containing instructions on how to implement this.
@Tony Are you interested in implementing this?
On 26.09.2017 14:10, Tony Wei wrote:
Hi Hai Zhou,
It's a good idea to implement my
Chesnay Schepler created FLINK-7692:
---
Summary: Support user-defined variables
Key: FLINK-7692
URL: https://issues.apache.org/jira/browse/FLINK-7692
Project: Flink
Issue Type: Improvement
BTW open() enables me only to initialize data to my objects. It still
impossible to access to previous records’ data unless I use state’ tools.
That’s why I said that Flink’ state functionality is futile. Because when you
use primitives or any serializable object then you don’t need to use open(
Timo Walther created FLINK-7691:
---
Summary: Remove ClassTag in Scala DataSet API
Key: FLINK-7691
URL: https://issues.apache.org/jira/browse/FLINK-7691
Project: Flink
Issue Type: Improvement
Hi Hai Zhou,
It's a good idea to implement my own reporter, but I think it is not the
best solution.
After all, reporter needs to be set well when starting the cluster. It is
not efficient to update cluster whenever you have a new metric for a new
streaming job.
Anyway, it is still a workaround f
Hi Ziv,
if I mark it as transient, then its value erased and I get null pointer
exception.
on the other hand, leave it without 'transient', it's a compilation error -
implementation of function is not serializable.
For unserializable class fields that you need to use for processing, you need
Hi Tony,
you can consider implementing a reporter, use a trick to convert the flink's
metrics to the structure that suits your needs.
This is just my personal practice, hoping to help you.
Cheers,
Hai Zhou
> 在 2017年9月26日，17:49，Tony Wei 写道：
>
> Hi,
>
> Recently, I am using PrometheusReporte
hi Tzu-Li,
sorry I have to reopen it but I'm still having troubles with using data from
previous records.
you said:
“If that’s the case, and you don’t care about losing state on failures and
don’t care about at-least-once / exactly-once, then yes you don’t have to use
the managed state APIs in
hi Tzu-Li,
sorry I have to reopen it but I'm still having troubles with using data from
previous records.
you said:
“If that’s the case, and you don’t care about losing state on failures and
don’t care about at-least-once / exactly-once, then yes you don’t have to
use the managed state APIs in
Hi,
Recently, I am using PrometheusReporter to monitor every metrics from Flink.
I found that the metric name in Prometheus will map to the identifier from
User Scope and System Scope [1], and the labels will map to Variables [2].
To monitor the same metrics from Prometheus, I would like to use
Hi Jagadisha,
the benefit of using Flink's connector is that you don't have to write the
sink function on your own. Moreover, it integrates with Flink's
checkpointing mechanism which gives you up to exactly once processing
guarantees if the update operation is idempotent.
I've included Chesnay wh
Hi,
I have been recently into the application development with flink. We are
trying to use the flink-apache connectors to get the data in and out from
Cassandra.
We attempted both Datastax drivers and Flink-cassandra connectors. In this
process felt that flink-cassandra connector is more of a wr
Hi,
usually such an error indicates that you have a different version of
Elasticsearch in your classpath than the one used at compile time.
Please make sure that you don't have accidentally an older version of
Elasticsearch in your classpath. You can see the complete classpath at the
beginning of
Shuyi Chen created FLINK-7690:
-
Summary: Do not call actorSystem.awaitTermination from the main
akka message handling thread
Key: FLINK-7690
URL: https://issues.apache.org/jira/browse/FLINK-7690
Project:
24 matches
Mail list logo
