vaijosh opened a new pull request, #3061:
URL: https://github.com/apache/hugegraph/pull/3061
## Why
This PR introduces a unified cloud storage capability for HugeGraph’s
distributed stack, so store data can be synchronized to cloud object storage
for stronger durability and recovery options.
The focus is not a provider-specific implementation detail, but a
cloud-capable architecture with a default S3-compatible provider and extension
points for additional providers.
## What this PR delivers
### 1) Cloud storage support in distributed runtime
- Enables cloud sync for store-side RocksDB in distributed deployments
(`backend=hstore` flow).
- Supports both:
- **cloud-first mode**: commit waits for cloud sync (stronger durability)
- **async mode**: local/Raft commit first, periodic cloud reconciliation
### 2) Pluggable provider architecture
- Introduces provider/client abstractions for cloud backends.
- Uses `ServiceLoader` discovery so new cloud providers can be added via
plugin JARs.
- Keeps built-in `s3` provider as default (S3-compatible API model), while
allowing future providers without core rewrites.
### 3) Cloud-neutral configuration model
- Standardizes configuration naming around cloud-neutral keys (`cloud_*`).
- Aligns server-side propagation and store-side consumption of cloud
settings.
- Cleans up older S3-specific naming to keep the config surface consistent
and provider-agnostic.
### 4) Operational docs and examples
- Updates architecture and usage docs to describe cloud storage behavior
consistently.
- Adds/updates sample plugin guidance and SPI wiring for provider developers.
- Aligns docker/dev scripts and templates with cloud storage configuration.
## User Impact
* **Resilience against Ephemeral Infrastructure:** By treating cloud object
storage as the decoupled, durable source of truth, HugeGraph can seamlessly
adapt to cloud-native environments (e.g., Kubernetes). If an instance or pod is
unexpectedly terminated, rescheduled, or suffers from local disk/EBS
detachment, the system prevents catastrophic data loss. New instances can
instantly rehydrate or recover their state directly from the cloud checkpoint.
* **Flexible Durability SLAs:** Operators running distributed deployments
(`backend=hstore`) can choose their optimal trade-off between throughput and
durability:
* **Cloud-First Mode (`sync`):** Guarantees zero data loss by ensuring
local Raft commits inline with cloud storage flushes before returning success
to the caller.
* **Asynchronous Mode (`async`):** Minimizes latency by performing
background reconciliation to the cloud storage bucket within a bounded time
horizon.
## Compatibility / migration
- Default provider remains `s3` (S3-compatible).
- Configuration surface is now cloud-neutral (`cloud_*`).
- Environments still using legacy S3-specific keys should migrate to the new
cloud-neutral keys.
## Validation
- Maven compile/build verification on affected modules.
- Cloud plugin wiring validated through ServiceLoader smoke checks.
- Documentation and config templates updated to match runtime behavior.
## Reviewer focus areas
- Cloud config key consistency across server/store/docs
- Provider abstraction and ServiceLoader integration
- Cloud-first vs async durability semantics
- Recovery/rehydration behavior and operational clarity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
