Thanks David and others. Our DAG has multiple sources which are not connected to each other. If one of them fails, I believe Flink can restart a single region for defaultscheduler. but it is not the same case for adaptive scheduler. Do you think somehow the adaptive scheduler supports region pipelines for streaming jobs ? If it can, local recovery makes sense at that time I believe.
Thanks On Wed, Apr 12, 2023 at 2:15 AM David Morávek <david.mora...@gmail.com> wrote: > Hi Talat, > > For most streaming pipelines, we have to restart the whole pipeline no > matter the scheduler used because they're a single pipelined region. One > limitation of AdaptiveScheduler is the lack of support for local recovery. > This will be addressed in Flink 1.18 [1]. > > [1] https://issues.apache.org/jira/browse/FLINK-21450 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_FLINK-2D21450&d=DwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=BkW1L6EF7ergAVYDXCo-3Vwkpy6qjsWAz7_GD7pAR8g&m=io1uqfos6mZy0_MjhX-EMn6Kv1O-J51WEdXoLFL2-JdSmUURlkVSK9Jo06K7PXbt&s=duMv5JJ1nSCwJNlA1eG3uaopaPpgwqrBfBnbegFEb7s&e=> > > Best, > D. > > On Tue, Apr 11, 2023 at 4:35 AM Weihua Hu <huweihua....@gmail.com> wrote: > >> Hi, >> >> AFAIK, the reactive mode always restarts the whole pipeline now. >> >> Best, >> Weihua >> >> >> On Tue, Apr 11, 2023 at 8:38 AM Talat Uyarer via user < >> user@flink.apache.org> wrote: >> >>> Hi All, >>> >>> We use Flink 1.13 with reactive mode for our streaming jobs. When we >>> have an issue/exception on our pipeline. Flink rescheduled all tasks. Is >>> there any way to reschedule only task that had exceptions ? >>> >>> Thanks >>> >>
