[DISCUSS] Initial Implementation of MailIO - Bounded Read with SDF

2024-11-12 Thread LDesire
Hi all, I've implemented basic functionality for MailIO, following the suggestions provided in the previous discussions. Current implementation includes: - Bounded read operation for email messages - Integration with Splittable DoFn (SDF) for parallel processing The implementation is currentl

Re: [YAML] Reprocessing failed records

2024-11-12 Thread Robert Bradshaw via dev
I created https://github.com/apache/beam/pull/33094 . We can continue to iterate on the best way to do this, but it'd be good to make this at least possible. I added some more justification in the PR description. On Mon, Oct 28, 2024 at 9:58 AM Robert Bradshaw wrote: > > On Tue, Oct 22, 2024 at 1

Re: [DISCUSS] Potential Use Cases for MailIO Connector

2024-11-12 Thread Piotr Wiśniowski
Hi, Had theoretical poc project in the past with quite similar functionalities needed. Bounded read makes sense, and can be threatened as special case of unbounded read. The second I could imagine is doing the same (reading emails for downstream processing like some logic triggers or ml categorizat

[DISCUSS] Potential Use Cases for MailIO Connector

2024-11-12 Thread LDesire
Hello, I am currently working on developing a MailIO connector for Apache Beam. While I have made progress implementing bounded read functionality, I'm somewhat uncertain about the practical use cases where users would need the MailIO connector. The use cases I've considered are: - Bounded Re

Beam High Priority Issue Report (46)

2024-11-12 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P0 Issues: https://github.com/apache/beam/issues/33075 [Failing Tests]: Ma