Gabriel39 commented on PR #435:
URL: https://github.com/apache/doris-thirdparty/pull/435#issuecomment-6967335487

   Review of `b3ee845`:
   
   **[P1] The retryable Metadata loader can block permanently after an 
exception with pthread_once-backed runtime combinations**
   
   At 
[`ReaderImpl::readMetadata()`](https://github.com/apache/doris-thirdparty/blob/b3ee845417888bc60a009f1610d41219e4b5fb54/c%2B%2B/src/Reader.cc#L1000),
 the `std::call_once` callback performs I/O, decompression, allocation, and 
protobuf parsing, all of which can throw. The implementation relies on 
exceptional execution resetting the once flag so another reader can retry.
   
   I tested this through the SDK: export a cold statistics handle, destroy the 
parent, inject one Metadata read exception in the first child, catch that 
exception, and retry through a second child sharing the handle. The first 
exception was caught, but the second child blocked in `__pthread_once_slow`. A 
sequential diagnostic control that bypassed `call_once` completed the retry. 
This is **toolchain/runtime dependent**: a separate system compiler/runtime 
control succeeded, and I have not reproduced it in a running Doris BE.
   
   The underlying compatibility issue is documented in [GCC's explanation of 
the pthread_once-backed 
implementation](https://gcc.gnu.org/pipermail/libstdc%2B%2B-cvs/2021q1/035565.html).
 This minimal program also distinguishes affected runtime combinations:
   
   ```cpp
   #include <cstdio>
   #include <mutex>
   #include <stdexcept>
   
   int main() {
     std::once_flag flag;
     try {
       std::call_once(flag, [] {
         throw std::runtime_error("injected metadata read failure");
       });
     } catch (const std::runtime_error&) {
       std::puts("first failure caught");
       std::fflush(stdout);
     }
     std::call_once(flag, [] { std::puts("retry succeeded"); });
   }
   ```
   
   On the affected combination, the second call never returns. Since siblings 
share the flag, a recoverable read or parse failure can leave subsequent 
readers waiting indefinitely.
   
   Please consider a mutex-protected loading state, publishing the immutable 
Metadata and marking it loaded only after successful parsing. Add a 
failure-then-retry test using two readers with the same handle; the new test 
currently exercises successful cold/warm loads only.
   
   Normal-path validation passed: 119 selected ASAN tests, 25 repetitions of 
the new concurrent test, and all 45 valid intervals of an eight-stripe fixture 
with mixed pruning decisions.
   
   One downstream integration note: the currently published `add70a4` head of 
apache/doris#70664 still passes only the serialized tail and calls the 
full-file `getNeedReadStripes(opts)`. It needs to propagate the shared 
statistics handle and use the interval overload, including its relative result 
indexing, before the intended Doris optimization takes effect.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to