[Release-2.48.0] Uploading images to dockerhub

2023-05-23 Thread Ritesh Ghorse via dev
Hey everyone, I'm at the stage of pushing docker containers to the apache repository of [Dockerhub](https://hub.docker.com/search?q=apache%2Fbeam&type=image). Since I'm not a part of the `beammaintainers` group, I'm getting permission denied. For the people in the `beammaintainers` group, please

Re: [PMC Request] Add gpg key to release keys file

2023-05-23 Thread Robert Bradshaw via dev
Done. On Tue, May 23, 2023 at 7:36 AM Danny McCormick via dev wrote: > Hey everyone, as part of automating our release process (see thread here - > https://lists.apache.org/thread/mw9dbbdjtkqlvs0mmrh452z3jsf68sct), could > a PMC member please add the infra supplied gpg public key to our release

[PMC Request] Add gpg key to release keys file

2023-05-23 Thread Danny McCormick via dev
Hey everyone, as part of automating our release process (see thread here - https://lists.apache.org/thread/mw9dbbdjtkqlvs0mmrh452z3jsf68sct), could a PMC member please add the infra supplied gpg public key to our release KEYS file? I added it to our dev KEYS file

Local Combiner for GroupByKey on Flink Streaming jobs

2023-05-23 Thread Talat Uyarer via dev
Sorry for cross posting -- Forwarded message - From: Talat Uyarer Date: Fri, May 19, 2023, 2:25 AM Subject: Local Combiner for GroupByKey on Flink Streaming jobs To: Hi, I have a stream aggregation job which is running on Flink 1.13 I generate DAG by using Beam SQL. My SQL que

Beam High Priority Issue Report (34)

2023-05-23 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/26785 [Failing Test]: Tes

Re: Watermark Alignment on Flink Runner's UnboundedSourceWrapper

2023-05-23 Thread Jan Lukavský
Yes, FlinkRunner supports Beam's event-time semantics without any additional configuration options.  Jan On 5/23/23 09:52, Talat Uyarer via dev wrote: Hi Jan, Yes My plan is implementing this feature on FlinkRunner. I have one more question. Does Flink Runner support EventTime or Beam  Cust

Re: Watermark Alignment on Flink Runner's UnboundedSourceWrapper

2023-05-23 Thread Talat Uyarer via dev
Hi Jan, Yes My plan is implementing this feature on FlinkRunner. I have one more question. Does Flink Runner support EventTime or Beam Custom Watermark ? Do I need to set AutoWatermarkInterval for stateful Beam Flink Jobs. Or Beam timers can handle it without setting that param ? Thanks On Tue,

Re: Watermark Alignment on Flink Runner's UnboundedSourceWrapper

2023-05-23 Thread Jan Lukavský
Hi Talat, your analysis is correct, aligning watermarks for jobs with high watermark skew in input partitions really results in faster checkpoints and reduces the size of state. There are generally two places you can implement this - in user code (the source) or inside runner. The user code c