Re: is there any detrimental side-effect if i set the max parallelism as 32768

Hi Hangxiang, David, Thank you for your replies. Your responses are very helpful. Best regards, Tony Wei David Anderson 於 2023年3月14日 週二 下午12:12寫道： > I believe there is some noticeable overhead if you are using the > heap-based state backend, but with RocksDB I think the difference is > negligi

Re: is there any detrimental side-effect if i set the max parallelism as 32768

I believe there is some noticeable overhead if you are using the heap-based state backend, but with RocksDB I think the difference is negligible. David On Tue, Mar 7, 2023 at 11:10 PM Hangxiang Yu wrote: > > Hi, Tony. > "be detrimental to performance" means that some extra space overhead of the

Re: Watermarks lagging behind events that generate them

Watermarks always follow the corresponding event(s). I'm not sure why they were designed that way, but that is how they are implemented. Windows maintain this contract by emitting all of their results before forwarding the watermark that triggered the results. David On Mon, Mar 13, 2023 at 5:28 P

Re: Are the Table API Connectors production ready?

The plan shows the filters has been pushed down. But remeber, although pused down, the filesystem table won't accept the filter. So, it'll be still like scan all files. Best regards, Yuxia 发件人: "Maryam Moafimadani" 收件人: "Hang Ruan" 抄送: "yuxia" , "ravi suryavanshi" , "Yaroslav Tkachenk

Re: Are the Table API Connectors production ready?

Thanks Hang for taking it. Assigned to you~ Best regards, Yuxia 发件人: "Hang Ruan" 收件人: "yuxia" 抄送: "ravi suryavanshi" , "Yaroslav Tkachenko" , "Shammon FY" , "User" 发送时间: 星期一, 2023年 3 月 13日 下午 8:54:49 主题: Re: Are the Table API Connectors production ready? Hi, yuxia, I would like to

Re: Watermarks lagging behind events that generate them

Hi Alexis Do you use both event-time watermark generator and TimerService for processing time in your job? Maybe you can try using event-time watermark first. Best, Shammon.FY On Sat, Mar 11, 2023 at 7:47 AM Alexis Sarda-Espinosa < sarda.espin...@gmail.com> wrote: > Hello, > > I recently ran in

Re: [EXT] Re: Kubernetes operator set container resources and limits

Thanks, this looks like what I need kubernetes.jobmanager.memory.limit-factor: '1.5' resources: limits: cpu: "1" memory: 4608Mi requests: cpu: "1" memory: 3Gi От: Jasmin Redzepovic Отправлено: 13 марта 2023 г. 1

Re: Are the Table API Connectors production ready?

Hi All, It's exciting to see file filtering in the plan for development. I am curious whether the following query on a filesystem connector would actually push down the filter on metadata `file.path`? *Select score, `file.path` from MyUserTable WHERE `file.path` LIKE '%prefix_%' * == Optimized Ex

Re: Kubernetes operator set container resources and limits

I want to set different values for resources and limits. От: Andrew Otto Отправлено: 13 марта 2023 г. 17:50 Кому: Evgeniy Lyutikov Копия: user@flink.apache.org Тема: Re: Kubernetes operator set container resources and limits Hi, > return to the same values from

Re: Are the Table API Connectors production ready?

Hi, yuxia, I would like to help to complete this task. Best, Hang yuxia 于2023年3月13日周一 09:32写道： > Yeah, you're right. We don't provide filtering files with patterns. And > actually we had already a jira[1] for it. > I was intended to do this in the past, but don't have much time. Anyone > who a

Re: Kubernetes operator set container resources and limits

Hi, > return to the same values from jobManager.resource FlinkDeployment manifest parameter I believe this is the correct way; using jobManager.resources and taskManager.resource

Kubernetes operator set container resources and limits

Hi all Is there any way to specify different values for resources and limits for a jobmanager container? The problem is that sometimes kubernetes kills the jobmanager container because it exceeds the memory consumption. Last State: Terminated Reason: OOMKilled Exit Code:137