aeroyorch opened a new issue, #63924:
URL: https://github.com/apache/airflow/issues/63924

   ### Description
   
   I would like to propose adding batching controls to `MessageQueueTrigger`.
   
   Currently, a DAG triggered by a messaging-based asset may receive multiple 
events in `triggering_asset_events`, but this does not seem to be configurable 
in a general way at the `MessageQueueTrigger` level, even if some providers may 
support provider-specific behavior.
   
   It would be useful to support, for example, a parameter to control the 
maximum batch size and another one to control the maximum wait time before 
flushing.
   
   ### Use case/motivation
   
   This would make `MessageQueueTrigger` easier to use in cases where message 
batching needs to be bounded and more predictable.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to