This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 34f991846f2 [fix](be) Avoid per-fd stat when counting process fds in
metrics (#67945)
34f991846f2 is described below
commit 34f991846f2437c6ee75f302c2f2eb2c69b62ff6
Author: Yancy <[email protected]>
AuthorDate: Wed Sep 16 11:06:06 2026 +0800
[fix](be) Avoid per-fd stat when counting process fds in metrics (#67945)
[fix](be) Avoid per-fd stat when counting process fds in metrics
### What problem does this PR solve?
Issue Number: close #67777
Problem Summary: `DorisMetrics::_update_process_fd_num()` counts entries
under `/proc/self/fd` with `entry.is_regular_file()`, which follows
every symlink with a `stat()` syscall. With ~200k open fds this takes
~0.6s per run, and the hook runs under the `MetricRegistry` lock every
15 s, stalling `/metrics` scrapes for hundreds of milliseconds. This PR
counts the directory entries directly (readdir only), which is about 5x
cheaper while still reflecting the number of open fds. The remaining
stalls (hooks running under the registry lock, stream load blocking
webserver workers) are tracked as follow-ups.
### Release note
None
### Check List (For Author)
- Test: No need to test (with reason) - behavior preserved (fd count),
only the per-entry stat syscalls are removed
- Behavior changed: No
- Does this need documentation: No
Co-authored-by: Asthenia0412 <[email protected]>
---
be/src/common/metrics/doris_metrics.cpp | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/be/src/common/metrics/doris_metrics.cpp
b/be/src/common/metrics/doris_metrics.cpp
index 37bbd024210..66f34fa573a 100644
--- a/be/src/common/metrics/doris_metrics.cpp
+++ b/be/src/common/metrics/doris_metrics.cpp
@@ -527,11 +527,15 @@ void DorisMetrics::_update_process_fd_num() {
process_fd_num_used->set_value(0);
return;
}
- int64_t count =
- std::count_if(dict_iter, std::filesystem::end(dict_iter), [](const
auto& entry) {
- std::error_code error_code;
- return entry.is_regular_file(error_code) && !error_code;
- });
+ // Entries under /proc/self/fd are symlinks, and every is_regular_file()
call
+ // follows the link with a stat(), which is O(fds) syscalls and can stall
+ // /metrics for hundreds of milliseconds when there are many open fds.
+ // Count the entries directly (readdir only) to avoid the per-entry stat.
+ int64_t count = 0;
+ for (const auto& entry : dict_iter) {
+ (void)entry;
+ ++count;
+ }
process_fd_num_used->set_value(count);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
