Re: Throwing Recoverable Exceptions from Tasks

2021-01-05 Thread Arvid Heise
A typical solution to your issue is to use an ELK stack to collect the logs and define some filters on log events. If it's specific to input data issues, I also found side-outputs useful to store invalid data points. Then, you can simply monitor the side topic (assuming Kafka) and already have the

Re: Throwing Recoverable Exceptions from Tasks

2020-12-28 Thread Chesnay Schepler
There is no way to have an exception appear in the REST API without restarting the job; that field is exactly defined as the exception causing the job to fail. Using asynchronous by itself is fine, so long as you don't wait for any confirmation. In any case you could remedy the issue by writin

Throwing Recoverable Exceptions from Tasks

2020-12-27 Thread Chirag Dewan
Hi, I am building an alerting system where based on some input events I need to raise an alert from the user defined aggregate function.  My first approach was to use an asynchronous REST API to send alerts outside the task slot. But this obviously involves IO from within the task and if I under