[GH] (flink): Workflow run "Nightly (beta)" failed!
The GitHub Actions job "Nightly (beta)" on flink.git has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]). Head commit for run: a79a649eaa6ad851078dd1f5630fc3009d0c1cc3 / Arvid Heise [FLINK-36287] Disallow UC for inner sink channels Between sink writer, the committer, and any pre/post commit topology (including global committer), we don't send actual payload but just committables. These committables must be committed on notifyCheckpointCompleted. However, if a barrier overtakes these committables, they may only be read after the RPC call has been made leading to violations. In particular, we could even have these issues during a final checkpoint. This commit generalizes the way, we disable UC for broadcast and pointwise connections, such that the SinkTransformationTranslator can also disable it for other distribution pattern. This commit also Simplifies committer with UC disabled: Without unaligned checkpoints, we receive all committables of a given upstream task before the respective barrier. Thus, when the barrier reaches the committer, all committables of a specific checkpoint must have been received. Committing happens even later on notifyCheckpointComplete. Added an assertion that verifies that all committables are indeed collected on commit. Note that this change also works when we recover a sink with channel state as it will only be called on the next (partially aligned) checkpoint. (cherry picked from commit 9d21878befa72f3c2b8f005c137544e2467d5a5e) Report URL: https://github.com/apache/flink/actions/runs/12002078231 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Nightly (beta)" failed!
The GitHub Actions job "Nightly (beta)" on flink.git has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]). Head commit for run: 2546d604a687a7212be20ddf2edecd886155182b / Arvid Heise [FLINK-36368] Do not prematurely merge CommittableManager (#25405) When a sink contains a shuffle between writer and committer, a committer may receive committables coming from multiple subtasks. So far, we immediately merged them on receiving. However, that makes it later impossible to trace whether we received all messages from an upstream task. It also made rescaling of the committer awkward: during normal processing all committables of a committer have the same subtaskId as the committer. On downscale, these subtaskIds suddenly don't match and need to be replaced, which we solved by merging the SubtaskCommittableManagers. This commit decouples the collection of committables from changing the subtaskId for emission. Committables retain the upstream subtask id in the CommittableCollection, which survives serialization and deserialization. Only upon emission, we substitute the subtask id with the one of the emitting committer. This is, in particular, useful for a global committer, where all subtasks are collected. As a side fix, the new serialization also contains the numberOfSubtasks such that different checkpoints may have different degree of parallelism. The old approach probably has edge cases where scaling a UC would result in stalled pipelines because certain metadata doesn't match. This would not affect pipelines which chain Writer/Committer (no channel state), Writer and Committer have same DOP (results in a Forward channel, which doesn't use UC for exactly these reasons), and a non-keyed shuffles (because they don't provide any guarantees). Since a keyed shuffle must use the subtask id of the committables, the new approach should be safe. However, since we disabled UC entirely for sinks to adhere to the contract of notifyCheckpointComplete, this shouldn't matter going forward. It's still important to consider these cases though for restoring from Flink 1 checkpoints. (cherry picked from commit 8dc212c32036c3a4afd3af2e50e95cb87c8cee23) Update version to 3.4.0 Report URL: https://github.com/apache/flink/actions/runs/12002078340 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Nightly (beta)" failed!
The GitHub Actions job "Nightly (beta)" on flink.git has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]). Head commit for run: ced02ce25a208a1154aa114439cda508637a47d7 / Peng Lu [FLINK-35977][docs] Add missing imports in examples in docs Report URL: https://github.com/apache/flink/actions/runs/12002078075 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Flink CI (beta)" is working again!
The GitHub Actions job "Flink CI (beta)" on flink.git has succeeded. Run started by GitHub user snuyanzin (triggered by snuyanzin). Head commit for run: ced02ce25a208a1154aa114439cda508637a47d7 / Peng Lu [FLINK-35977][docs] Add missing imports in examples in docs Report URL: https://github.com/apache/flink/actions/runs/11999226533 With regards, GitHub Actions via GitBox
[GH] (flink-cdc): Workflow run "Flink CDC CI" failed!
The GitHub Actions job "Flink CDC CI" on flink-cdc.git has failed. Run started by GitHub user liuxiao2shf (triggered by ruanhang1993). Head commit for run: 518de7364dee543fecc2bd7fa642c5e91a86bfc8 / molin.lxd [FLINK-36315][cdc-base]Fix code style issues Report URL: https://github.com/apache/flink-cdc/actions/runs/11970176268 With regards, GitHub Actions via GitBox
[GH] (flink-cdc): Workflow run "Flink CDC CI" is working again!
The GitHub Actions job "Flink CDC CI" on flink-cdc.git has succeeded. Run started by GitHub user liuxiao2shf (triggered by ruanhang1993). Head commit for run: 518de7364dee543fecc2bd7fa642c5e91a86bfc8 / molin.lxd [FLINK-36315][cdc-base]Fix code style issues Report URL: https://github.com/apache/flink-cdc/actions/runs/11970176268 With regards, GitHub Actions via GitBox
[GH] (flink-cdc): Workflow run "Flink CDC CI" failed!
The GitHub Actions job "Flink CDC CI" on flink-cdc.git has failed. Run started by GitHub user lvyanquan (triggered by ruanhang1993). Head commit for run: 57b3a78fa309d78de0dda5fc13e833311513a824 / lvyanquan [FLINK-36525] Support for AI Model Integration for Data Processing. Report URL: https://github.com/apache/flink-cdc/actions/runs/11964945778 With regards, GitHub Actions via GitBox
[GH] (flink-kubernetes-operator): Workflow run "Flink Kubernetes Operator CI" is working again!
The GitHub Actions job "Flink Kubernetes Operator CI" on flink-kubernetes-operator.git has succeeded. Run started by GitHub user 1996fanrui (triggered by 1996fanrui). Head commit for run: 5a825f0248228cb792410ed949646109b9091f5b / Rui Fan [FLINK-36535][autoscaler] Optimize the scale down logic based on historical parallelism to reduce the rescale frequency 1. Using the maximum parallelism within the window instead of the latest parallelism when scaling down 2. Never scale down when (currentTime - triggerTime) < scale-down.interval Report URL: https://github.com/apache/flink-kubernetes-operator/actions/runs/12002263968 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Flink CI (beta)" failed!
The GitHub Actions job "Flink CI (beta)" on flink.git has failed. Run started by GitHub user snuyanzin (triggered by snuyanzin). Head commit for run: 5739b16dcb26c5f4a38b5cc914082f69e4a001bf / Sergey Nuyanzin [FLINK-34600][table] Migrate PushLimitIntoLegacyTableSourceScanRule to java Report URL: https://github.com/apache/flink/actions/runs/11997007260 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Build documentation" failed!
The GitHub Actions job "Build documentation" on flink.git has failed. Run started by GitHub user zentol (triggered by zentol). Head commit for run: ced02ce25a208a1154aa114439cda508637a47d7 / Peng Lu [FLINK-35977][docs] Add missing imports in examples in docs Report URL: https://github.com/apache/flink/actions/runs/12000705560 With regards, GitHub Actions via GitBox
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:release-1.19 - apache-flink - a79a649e
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:release-1.19 - apache-flink - a79a649e Azure DevOps Build #20241124.3 failed flink-ci.flink-master-mirror Ran for 8 hours View results Summary Build pipeline flink-ci.flink-master-mirror Finished Sun, Nov 24 2024 08:43:15 GMT+00:00 Reason Schedule Details compile_cron_jdk21 0 error(s), 0 warning(s) compile_cron_jdk17 0 error(s), 0 warning(s) compile_ci 0 error(s), 0 warning(s) compile_cron_adaptive_scheduler 0 error(s), 0 warning(s) compile_cron_hadoop313 0 error(s), 0 warning(s) docs_404_check 0 error(s), 0 warning(s) docs_404_check 0 error(s), 0 warning(s) compile_cron_jdk11 0 error(s), 0 warning(s) e2e_2_cron_jdk17 1 error(s), 0 warning(s) Bash exited with code '1'. test_cron_jdk21 0 error(s), 2 warning(s) cron_snapshot_deployment_maven 0 error(s), 3 warning(s) com
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:release-1.20 - apache-flink - 2546d604
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:release-1.20 - apache-flink - 2546d604 Azure DevOps Build #20241124.4 failed flink-ci.flink-master-mirror Ran for 9 hours View results Summary Build pipeline flink-ci.flink-master-mirror Finished Sun, Nov 24 2024 08:59:44 GMT+00:00 Reason Schedule Details compile_cron_jdk21 0 error(s), 0 warning(s) compile_ci 0 error(s), 0 warning(s) compile_cron_adaptive_scheduler 0 error(s), 0 warning(s) compile_cron_jdk11 0 error(s), 0 warning(s) test_cron_jdk21 0 error(s), 1 warning(s) compile_cron_jdk17 0 error(s), 0 warning(s) docs_404_check 0 error(s), 0 warning(s) test_cron_jdk11 0 error(s), 2 warning(s) compile_cron_hadoop313 0 error(s), 0 warning(s) compile_cron_azure 0 error(s), 3 warning(s) docs_404_check 0 error(s), 0 warning(s) cron_snapshot_deployment_binary 0 error(s), 4 warning(s) test_cron_jdk17
[GH] (flink): Workflow run "Nightly (beta)" failed!
The GitHub Actions job "Nightly (beta)" on flink.git has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]). Head commit for run: b2a260ac957dac3b6af5dc73684624dd36dc92ea / Martijn Visser <2989614+martijnvis...@users.noreply.github.com> [FLINK-36591] Remove `flink-contrib` This module was introduced with FLINK-1452 to add new contributions in, but only contains a Wikiedits connector that connected to IRC. However, that is now consistently failing and contributions are either merged in directly into OSS Flink (via FLIP/Dev discussion), https://github.com/flink-extended and/or https://flink-packages.org/ (cherry picked from commit 28ae3ae65d13b98dc28b919ef75c14780c512618) Report URL: https://github.com/apache/flink/actions/runs/11992385071 With regards, GitHub Actions via GitBox
[GH] (flink-connector-aws): Workflow run "flink-connector-aws: nightly build" is working again!
The GitHub Actions job "flink-connector-aws: nightly build" on flink-connector-aws.git has succeeded. Run started by GitHub user dannycranmer (triggered by dannycranmer). Head commit for run: b55dec16855785b5b0af0fb0ff57816e71ad3e31 / Hong Teoh [FLINK-36668][build] Update AWS build for v5.0 Report URL: https://github.com/apache/flink-connector-aws/actions/runs/12000779918 With regards, GitHub Actions via GitBox
[GH] (flink-cdc): Workflow run "Flink CDC CI" is working again!
The GitHub Actions job "Flink CDC CI" on flink-cdc.git has succeeded. Run started by GitHub user yuxiqian (triggered by leonardBang). Head commit for run: d3de5f5c2d3e10d8611b4471f5d8b286f0b1dbcf / yuxiqian <34335406+yuxiq...@users.noreply.github.com> Add third-party OSS notice Signed-off-by: yuxiqian <34335406+yuxiq...@users.noreply.github.com> Report URL: https://github.com/apache/flink-cdc/actions/runs/12003066969 With regards, GitHub Actions via GitBox
[GH] (flink): Workflow run "Nightly (beta)" failed!
The GitHub Actions job "Nightly (beta)" on flink.git has failed. Run started by GitHub user github-actions[bot] (triggered by github-actions[bot]). Head commit for run: b2a260ac957dac3b6af5dc73684624dd36dc92ea / Martijn Visser <2989614+martijnvis...@users.noreply.github.com> [FLINK-36591] Remove `flink-contrib` This module was introduced with FLINK-1452 to add new contributions in, but only contains a Wikiedits connector that connected to IRC. However, that is now consistently failing and contributions are either merged in directly into OSS Flink (via FLIP/Dev discussion), https://github.com/flink-extended and/or https://flink-packages.org/ (cherry picked from commit 28ae3ae65d13b98dc28b919ef75c14780c512618) Report URL: https://github.com/apache/flink/actions/runs/12002078044 With regards, GitHub Actions via GitBox
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:master - apache-flink - ced02ce2
[Build failed] flink-ci.flink-master-mirror - flink-ci/flink-mirror:master - apache-flink - ced02ce2 Azure DevOps Build #20241125.1 failed flink-ci.flink-master-mirror Ran for 7 hours View results Summary Build pipeline flink-ci.flink-master-mirror Finished Mon, Nov 25 2024 07:21:00 GMT+00:00 Reason Schedule Details docs_404_check 0 error(s), 0 warning(s) docs_404_check 0 error(s), 0 warning(s) compile_ci 0 error(s), 0 warning(s) cron_snapshot_deployment_maven 0 error(s), 3 warning(s) compile_cron_azure 0 error(s), 3 warning(s) cron_snapshot_deployment_binary 0 error(s), 4 warning(s) compile_cron_jdk17 0 error(s), 0 warning(s) compile_cron_adaptive_scheduler 0 error(s), 0 warning(s) build_wheels_on_Linux 0 error(s), 0 warning(s) build_wheels_on_macos 0 error(s), 0 warning(s) test_ci 0 error(s), 1 warning(s) compile_cron_hadoop313 0 error(s), 0 warning(s) e2e_1_cron_azure
