Re: Task Manger is getting failed

2023-02-05 Thread P Singh
Hi weijie, I have 8gb memory still TM going oom though I have assigned all. Moreover I am just print 15 word in beam pipeline program. Any idea why taking too much memory? Get Outlook for iOS From: weijie guo Sent: Monday, February 6, 202

Re: Task Manger is getting failed

2023-02-05 Thread weijie guo
Hi, It seems that TM is unexpected un-reachable. You can confirm whether TM is OutOfMemory. Best regards, Weijie P Singh 于2023年2月5日周日 15:06写道: > Hi Team, > > I am submitting a beam python job to flink clusters on a local machine by > minikube. when i submit the job it appears on the flink UI

Re: Design decisions around flink table store

2023-02-05 Thread yuxia
Hi, Bright. Thanks for reaching out. That's a really good question. Briefly speaking, the reason is both Hudi and iceberg are not efficient for updating. Also, the FLIP for flink-table-store has said why not hudi [1]: " Why doesn't FileStore use Hudi directly? 1: Hudi aims to support the

Design decisions around flink table store

2023-02-05 Thread graceking lau
Hi there, Recently I had a chance to get to know the flink-table-store project. I was attracted by the idea behind it at first glance. After reading the docs, I've got a question in my head for a while. It's about the design of the file storage. It looks like we can implement it based on the oth

Re: What is the state of Scala wrappers?

2023-02-05 Thread Alexey Novakov via user
Hi Erwan, I think those 2 projects you mentioned are pretty much the options we have at the moment if you want to use Scala 2.13 or 3. I believe your contribution to upgrade one of them to Flink 1.16 will be very welcomed. Best regards, Alex On Thu, Feb 2, 2023 at 9:32 AM Erwan Loisant wrote: