Hi Sharif, You could not catch exceptions globally.
For exceptions that can be explicitly ignored for your business, you need to add a try-catch in the operators. For exceptions that are not catched, Flink will trigger a recovery from failure automatically[1]. [1] https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/ops/state/task_failure_recovery/ Best, Weihua On Tue, May 23, 2023 at 12:24 PM Sharif Khan via user <user@flink.apache.org> wrote: > Thanks for your response. > > For simplicity, I want to capture exceptions in a centralized manner and > log them for further analysis, without interrupting the job's execution or > causing it to restart. > > On Tue, May 23, 2023 at 6:31 AM Shammon FY <zjur...@gmail.com> wrote: > >> Hi Sharif, >> >> I would like to know what do you want to do with the exception after >> catching it? There are different ways for different requirements, for >> example, Flink has already reported these exceptions. >> >> Best, >> Shammon FY >> >> >> On Mon, May 22, 2023 at 4:45 PM Sharif Khan via user < >> user@flink.apache.org> wrote: >> >>> Hi, community. >>> Can anyone please let me know? >>> >>> 1. What is the best practice in terms of handling exceptions in Flink >>> jobs? >>> >>> 2. Is there any way to catch exceptions globally in Flink jobs? >>> Basically, I want to catch exceptions from any operators in one place >>> (globally). >>> >>> my expectation is let's say I have a pipeline >>> source-> operator(A) -> operator(B) -> operator(C) -> sink. >>> I don't want to write a try-catch for every operator. Is it possible to >>> write one try-catch for the whole pipeline? >>> >>> I'm using the Python version of the Flink API. version 1.16 >>> >>> Thanks in advance. >>> >>> [image: SELISE] >>> >>> SELISE Group >>> Zürich: The Circle 37, 8058 Zürich-Airport, Switzerland >>> Munich: Tal 44, 80331 München, Germany >>> Dubai: Building 3, 3rd Floor, Dubai Design District, Dubai, United Arab >>> Emirates >>> Dhaka: Midas Center, Road 16, Dhanmondi, Dhaka 1209, Bangladesh >>> Thimphu: Bhutan Innovation Tech Center, Babesa, P.O. Box 633, Thimphu, >>> Bhutan >>> >>> Visit us: www.selisegroup.com >>> >>> *Important Note: This e-mail and any attachment are confidential and may >>> contain trade secrets and may well also be legally privileged or otherwise >>> protected from disclosure. If you have received it in error, you are on >>> notice of its status. Please notify us immediately by reply e-mail and then >>> delete this e-mail and any attachment from your system. If you are not the >>> intended recipient please understand that you must not copy this e-mail or >>> any attachment or disclose the contents to any other person. Thank you for >>> your cooperation.* >>> >> > > -- > > Md. Sharif Khan, BSc in CSE, DIU > > Software Engineer > > Mobile: +880 1741976078 > > [image: SELISE] > > SELISE Group > Zürich: The Circle 37, 8058 Zürich-Airport, Switzerland > Munich: Tal 44, 80331 München, Germany > Dubai: Building 3, 3rd Floor, Dubai Design District, Dubai, United Arab > Emirates > Dhaka: Midas Center, Road 16, Dhanmondi, Dhaka 1209, Bangladesh > Thimphu: Bhutan Innovation Tech Center, Babesa, P.O. Box 633, Thimphu, > Bhutan > > Visit us: www.selisegroup.com > > *Important Note: This e-mail and any attachment are confidential and may > contain trade secrets and may well also be legally privileged or otherwise > protected from disclosure. If you have received it in error, you are on > notice of its status. Please notify us immediately by reply e-mail and then > delete this e-mail and any attachment from your system. If you are not the > intended recipient please understand that you must not copy this e-mail or > any attachment or disclose the contents to any other person. Thank you for > your cooperation.* >
