Re: logback variable substitution in kubernetes

>From the logback documentation[1], it could support OS environment substitution. Could you please check that the environment variables have been properly set? Maybe you could tunnel into the Kubernetes pod via "kubectl exec" and do such verification. Best, Yang houssem 于2021年8月31日周二 下午7:28写道：

Re: Deploying Flink on Kubernetes with fractional CPU and different limits and requests

Great. If no one wants to work on this ticket FLINK-15648, I will try to get this done in the next major release cycle(1.15). Best, Yang Denis Cosmin NUTIU 于2021年8月31日周二 下午4:59写道： > Hi everyone, > > Thanks for getting back to me! > > > I think it would be nice if the task manager pods get thei

Re: when does app to print

Hi! With streaming API you'll need to call env.execute() at the end to start the job. igyu 于2021年9月1日周三 上午11:29写道： > this is my data > > {"timestamp":"2021-08-01 15:31:56,895","msg":" INFO > org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: > get_table : db=hivetest tbl=ch

Re: Queries regarding Flink upgrade strategies

Hi Matthias, Thanks for the confirmation. @Yang Wang : Any comments from your side ? Regards, Amit On Fri, Aug 27, 2021 at 7:27 PM Matthias Pohl wrote: > Thanks for clarifying that, Amit. Rolling updates with JobManagers and > TaskManagers coming from different Flink versions in the same Fli

when does app to print

this is my data {"timestamp":"2021-08-01 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"} {"timestamp":"2021-08-02 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thr

Re: Clarifying Documentation on Custom Triggers

Hi! I don't quite understand this problem. But if you look into WindowedStream#trigger you'll find that the trigger of WindowOperatorBuilder will change when you call that method, and thus the default trigger will be overwritten by calling WindowedStream#trigger. Aeden Jameson 于2021年9月1日周三 上午12:

Re: Session windows - how to get the last value from a window using FlinkSQL?

Hi, The exception in the above StackOverflow link because built-in last_value agg function class ( `LastValueAggFunction`) does not support merge method. While you use session window aggregate, `merge` is required when merge two windows into one session window. [image: image.png] *You could define

Re: Session windows - how to get the last value from a window using FlinkSQL?

Hi! Yes it is, by using the last_value() aggregate function. For example SELECT last_value(v) FROM T GROUP BY k, session(ts, interval '1' minute) Michał Rudko 于2021年8月31日周二 下午11:40写道： > Hi, > > I had trouble to find in the documentation of Flink/Ververica which > aggregations or analytical fun

Re: Flink performance with multiple operators reshuffling data

Hi Jason, > In our case, our input/output ratio of these Flin operators are all 1 to 1, so I guess it doesn't matter that much.. Yes > But I think the keys we are using in general are pretty uniform. Cool. You could run for a period of time to see if there is data skew. If there is indeed a data sk

Re: Delete Keyed State outside of StateTTL

Hi, After you call `clear()`, you could be sure that they would not be returned when you query the state under the key. The state would be removed. The implication on memory occupation depends on which state backend. Different state backend may have different implication here. For example, if you u

Re: checkpoints/.../shared cleanup

I'm running Flink in Application Mode and set jobId explicitly From: Khachatryan Roman Sent: Monday, August 30, 2021 7:16 AM To: Alexey Trenikhun Cc: Matthias Pohl ; Flink User Mail List ; sjwies...@gmail.com Subject: Re: checkpoints/.../shared cleanup Hi, I

Re: [ANNOUNCE] Apache Flink Stateful Functions 3.1.0 released

Congrats on the release! And thank you for driving this release, Igal. Cheers Gordon On Tue, Aug 31, 2021, 23:13 Igal Shilman wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink Stateful Functions (StateFun) 3.1.0. > > StateFun is a cross-platform stack

Clarifying Documentation on Custom Triggers

Flink Version: 1.13.2 In the section on Default Triggers of Window Assigners the documentation States By specifying a trigger using trigger() you are overwriting the default trigger of a WindowAssigner. For example, if you specify a CountTrigger for TumblingEventTimeWindows you will no longer get

Re: Table API demo problem

After the images were rebuilt for linux/amd64, it started working. Thank you very much Matthias and Chesnay! Wow, you guys are really fast. Great demo as well. From: Matthias Pohl Date: Tuesday, August 31, 2021 at 7:46 AM To: "Tatla, Manraj" Cc: "user@flink.apache.org" Subject: RE: [EXTERNAL

Re: Flink issues with Avro GenericRecord serialization

This is resolved by the first approach I mentioned. Thanks Team On Mon, 30 Aug 2021 at 12:35, tarun joshi <1985.ta...@gmail.com> wrote: > An update on this , I see that `IndexedRecord` is part of Avro Library. > Please correct me If I am wrong in assuming that the "Pojo's generated by > Avro POJ

Session windows - how to get the last value from a window using FlinkSQL?

Hi, I had trouble to find in the documentation of Flink/Ververica which aggregations or analytical functions are supported within such SESSION WINDOW. In particular - I would like to capture the last value of a certain field. Is it possible? I have included all the details in StackOverflow here

[ANNOUNCE] Apache Flink Stateful Functions 3.1.0 released

The Apache Flink community is very happy to announce the release of Apache Flink Stateful Functions (StateFun) 3.1.0. StateFun is a cross-platform stack for building Stateful Serverless applications, making it radically simpler to develop scalable, consistent, and elastic distributed applications.

Re: 退订

Hi, Please send email to dev-unsubscr...@flink.apache.org if you want to unsubscribe the mail from d...@flink.apache.org . Please send email to user-unsubscr...@flink.apache.org

Re: Kafka connector depending on Table API

Hi Maciek, thanks for testing the RC! You are absolutely right. This is a bug. I will create an issue for it. Thanks again, Timo On 31.08.21 16:33, Maciek Próchniak wrote: Hello, we are testing 1.14 RC0 and we discovered that we need to include table-api as dependency when using kafka con

Kafka connector depending on Table API

Hello, we are testing 1.14 RC0 and we discovered that we need to include table-api as dependency when using kafka connector - e.g. due to this change: https://github.com/apache/flink/blame/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/

Re: Savepoint failure along with JobManager crash

Hi Prasanna, thanks for reaching out to the community. What you're experiencing is that the savepoint was created but the job itself ended up in an inconsistent state with Executions being cancelled instead of being finished. This should have triggered a global failover resulting in a job restart.

Re: Table API demo problem

I missed the point that it's the purpose of the walkthrough to have the functionality being implemented by the user. So, FLINK-24076 is actually not valid. I initially thought of it as some kind of demo implementation. Sorry for the confusion. On Tue, Aug 31, 2021 at 11:15 AM Matthias Pohl wrote:

logback variable substitution in kubernetes

Hello, I am running a flink application cluster in standalone kubernetes mode and i a using logback as a logging framework , th problem is i am not able tu use environment variables configured in my pod inside my logback-console.xml file . I copied this file from my file system while buil

Help us stabilise the first release candidate for Apache Flink 1.14

Dear Apache Flink users, We just released the first release candidate for Apache Flink 1.14. The whole community is now stabilising this release to make it as flawless as possible. We’d like to ask every Apache Flink user to give it a try and provide some feedback. Please read how you can help

Re: Table API demo problem

Hi Manraj, the error messages about libjemalloc.so are caused by Flink 1.13.1 that has been published with the wrong architecture accidentally. I created FLINK-24075 [1] to cover this issue. As a workaround, you could upgrade the base image to Flink 1.13.2 until the Flink 1.13.1 images are republis

Re: Deploying Flink on Kubernetes with fractional CPU and different limits and requests

Hi everyone, Thanks for getting back to me! > I think it would be nice if the task manager pods get their values from the > configuration file only if the pod templates don’t specify any resources. > That was the goal of supporting pod templates, right? Allowing more custom > scenarios withou

Re: Flink performance with multiple operators reshuffling data

Thanks for the help guys! Yea we can potentially append random strings to the keys and duplicate data across them to avoid skewness, if necessary. But I think the keys we are using in general are pretty uniform. The lowest selectivity at the up fornt method is really interesting though. In our cas