hgudladona commented on issue #13356: URL: https://github.com/apache/hudi/issues/13356#issuecomment-2923234088
The default value for the timeout (for view manager and marker api) is 5m, we have lowered it to 30s to fail fast to secondary view for timeline view clients. We also have task retries set to 5, which also consistently fails on retry. In this case the number of executor tasks are ~1200 in the write stage, which is very reasonable and this works most of the time. We did test the marker API ourself with K6 and able to produce 30K RPS without fail, but this timeout happens periodically. I found this in Hudi JIRA https://issues.apache.org/jira/browse/HUDI-5670 which has the same stacktrace. One other possibility that the driver is responsive is a contention on a lock at higher load, which is also hard to reproduce. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
