maheshguptags commented on issue #12988: URL: https://github.com/apache/hudi/issues/12988#issuecomment-2743216536
Hi @danny0405 I tried your suggestion and implemented the index like ingestion job and it worked. So summary is : we will need to enable concurrency to delete the data by flink-hudi batch stream with index that are built on ingestion job. added below config ``` ,'hoodie.index.type'= 'BUCKET','hoodie.index.bucket.engine' = 'SIMPLE','hoodie.bucket.index.num.buckets'='16','hoodie.bucket.index.hash.field'='x' ``` Also dag has been updated with 3 bucket_writer. <img width="1051" alt="Image" src="https://github.com/user-attachments/assets/1b65dd46-3d77-4a68-a6c0-c8403626f97f"; /> it is scanning exactly number are required records that match the conditions. <img width="1203" alt="Image" src="https://github.com/user-attachments/assets/6c133ad7-91dc-417c-8685-3496622263be"; /> Thank you @danny0405 for your support. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
