Re: Projection pushdown for Avro files seems to be buggy

Hi, Xingcan After deep dive into the source code, I also think it is a bug. Best, Ron Xingcan Cui 于2023年8月5日周六 23:27写道： > Hi all, > > We tried to read some Avro files with the Flink SQL (1.16.1) and noticed > that the projection pushdown seems to be buggy. > > The Avro schema we used has 4 fie

[jira] [Created] (FLINK-32763) Uncaught exceptions cause retry not to take effect in SocketTextStreamFunction

jianfeng miao created FLINK-32763: - Summary: Uncaught exceptions cause retry not to take effect in SocketTextStreamFunction Key: FLINK-32763 URL: https://issues.apache.org/jira/browse/FLINK-32763 Proj

[jira] [Created] (FLINK-32762) Support concurrency control when submitting OLAP jobs to Dispatcher

xiangyu feng created FLINK-32762: Summary: Support concurrency control when submitting OLAP jobs to Dispatcher Key: FLINK-32762 URL: https://issues.apache.org/jira/browse/FLINK-32762 Project: Flink

Projection pushdown for Avro files seems to be buggy

Hi all, We tried to read some Avro files with the Flink SQL (1.16.1) and noticed that the projection pushdown seems to be buggy. The Avro schema we used has 4 fields, namely f1, f2, f3 and f4. When using "SELECT *" or SELECT the first n fields (e.g., SELECT f1 or SELECT f1, f2) to read the table,

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

Congrats Matthias!! On Sat, Aug 5, 2023 at 09:35 Runkang He wrote: > Congratulations! > > Best, > Runkang He > > Yanfei Lei 于2023年8月5日周六 20:13写道： > > > Congratulations! > > > > Best regards, > > Yanfei > > > > Hong Liang 于2023年8月4日周五 20:02写道： > > > > > > Congratulations Matthias! > > > > > > V

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

Congratulations! Best, Runkang He Yanfei Lei 于2023年8月5日周六 20:13写道： > Congratulations! > > Best regards, > Yanfei > > Hong Liang 于2023年8月4日周五 20:02写道： > > > > Congratulations Matthias! > > > > Very well deserved! > > > > Regards, > > Hong > > > > On Fri, Aug 4, 2023 at 10:38 AM Markos Sfikas >

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

Congratulations, Weihua! Best, Runkang He Kelu Tao 于2023年8月4日周五 18:21写道： > Congratulations! > > On 2023/08/04 08:35:49 Danny Cranmer wrote: > > Congrats and welcome to the team, Weihua! > > > > Thanks, > > Danny > > > > On Fri, Aug 4, 2023 at 9:30 AM Feng Jin wrote: > > > > > Congratulations W

Re: Heartbeat of TaskManager with id container

Hi Nagireddy, I'm not particularly familiar with StreamingFileSink but I checked with the implementation of HadoopFsCommitter. AFAIK, when committing files to HDFS the committer will check if the temp file exist in the first place. [image: image.png] In your case, could u check why the committing

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

Congratulations! Best regards, Yanfei Hong Liang 于2023年8月4日周五 20:02写道： > > Congratulations Matthias! > > Very well deserved! > > Regards, > Hong > > On Fri, Aug 4, 2023 at 10:38 AM Markos Sfikas > wrote: > > > Congrats Matthias! > > Best > > Markos > > > > On Fri, Aug 4, 2023 at 11:25 AM Alexan

Re: Heartbeat of TaskManager with id container

Hi Xiangyu/Dev, Did any one has solution handle below important note in StreamingFileSink: Caused by: java.io.IOException: Cannot clean commit: Staging file does not exist. at org.apache.flink.runtime.fs.hdfs. HadoopRecoverableFsDataOutputStream$HadoopFsCommitter.commit( HadoopRecoverableFsDa

[jira] [Created] (FLINK-32761) Use md5 sum of PhysicalStateHandleID as SharedStateRegistryKey ChangelogStateHandleStreamImpl

Feifan Wang created FLINK-32761: --- Summary: Use md5 sum of PhysicalStateHandleID as SharedStateRegistryKey ChangelogStateHandleStreamImpl Key: FLINK-32761 URL: https://issues.apache.org/jira/browse/FLINK-32761

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

Hi Rui! Thanks for the proposal. I agree with Max on that the state store abstractions could be improved and be more specific as we know what goes into the state. It could simply be public interface AutoScalerStateStore { Map getState(KEY jobKey) void updateState(KEY jobKey, Map state);