Hi Keith, Thanks for the review and the kind words. Good points!
> 1. Have we considered timeout, liveness or retry semantics? ... It might be > worth adding configurable timeouts. Without this, user will have to resort to > wrapping the kernel with watchdog. The concern is valid, but I think it’s better not to add a reader-level timeout, to stay aligned with fluss-java. In fluss-java, LogScanner::poll(Duration) and BatchScanner::pollBatch(Duration) take a per-call timeout and return empty on timeout; higher-level retry and cancellation are left to the caller. StarRocks and Flink already manage query-level timeouts around connector reads, so a reader-level timeout would just duplicate that. Transient RPC errors are retried by the underlying client; a stuck or dead server shows up as ConnectionError. For slow PK tail reads, the caller can wrap stream.next() with their own timeout. > 2. On PK hash overlay, the proposal relies on the premise that tail is > bounded and small. There's no defined behaviour when that premise fails. > Could we specify a failure mode? Good point, the OOM risk is real. But I think it’s better to leave this out of V1 for now and track it as future work. I have noted this in the FIP. What do you think? > 3. Can we include in the proposal on the credential and security story for > object storage? ... calling it out in the proposal would make it clear. Added a "Security and credentials" section to the FIP. In short: fluss-lake doesn't manage credentials itself. Auth for object storage (S3, OSS, GCS, etc.) is delegated to the underlying storage client. Credentials are resolved at reader construction time, never logged, and never put into splits. Splits only carry storage-relative paths. All changes are on the Confluence page. Appreciate the review! Best regards, Jim > 2026年7月31日 下午6:58，Keith Lee <[email protected]> 写道： > > Hello Jim, > > Thank you for the FIP, the general direction looks solid. I have the > following questions: > > 1. Have we considered timeout, liveness or retry semantics? A hung storage > request or dead server would stall the stream indefinitely without timeout. > This also apply for PK splits where merge must first materialise the whole > changeling tail before emitting anything. It might be worth adding > configurable timeouts. Without this, user will have to resort to wrapping > the kernel with watchdog. > 2. On PK hash overlay, the proposal relies on the premise that tail is > bounded and small. There's no defined behaviour when that premise fails. > Could we specify a failure mode? e.g. configurable memory / tail-size cap > surfacing an error such as FlakeLakeError("snapshot too stale") rather than > risking OOMing the host process? > 3. Can we include in the proposal on the credential and security story for > object storage? I appreciate that the proposal might be following Java side > implementation, calling it out in the proposal would make it clear. > > It's a great FIP, looking forward to your response. > > Best regards > Keith
