contrueCT commented on PR #3127: URL: https://github.com/apache/hugegraph/pull/3127#issuecomment-5200755918
Thanks for the implementation. I reviewed the current patch, and it still seems to follow the original design that I raised concerns about in issue #3071. The main problem is that the chunking logic is still implemented in `HugeTask.asArray()`. At this point, the result has already been fully serialized, and `HugeTask.set()` has already applied the total result-size check, so an oversized result may still fail before the chunking logic is reached. More importantly, this is no longer the shared persisted-result path. `TaskAndResultScheduler.save()` stores task metadata through `HugeTask.asArrayWithoutResult()` and persists the actual result separately through `HugeTaskResult.asArray()`. Since `HugeTaskResult` is unchanged in this PR, distributed/HStore deployments will still store the result as a single BLOB, and the new chunking logic in `HugeTask.asArray()` may not be used for the actual result at all. The current patch also only adds the chunked write path. There is no corresponding schema update or read/reassembly support for `~task_result_0`, `~task_result_1`, and `~task_result_chunk_count`, so even the legacy `HugeTask` persistence path would not be complete. I think the design should first be revised around the actual result-storage abstraction, most likely `HugeTaskResult`, and clearly define: * how chunked and legacy single-BLOB results are distinguished; * how `task.result_size_limit` applies to the total result versus each chunk; * how chunks are read and reassembled safely; * how local and distributed task schedulers share the same behavior; * how old persisted task results remain readable. Tests should cover the real persisted-result path, distributed/HStore behavior, chunk reassembly, incomplete chunks, and backward compatibility with the old single-property format. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
