Hi all, I'd like to propose an improvement to the object storage feature and get feedback before I start to think about developing something.
Current behaviour Object storage credentials are provisioned per CloudStack account, not per bucket. The first bucket creates a backing user (e.g. a radosgw-admin user on the RGW provider) and its access/secret key is stored against the account. Every subsequent bucket in that account reuses the same key pair. Deleting and recreating a bucket returns the same keys, and there's no API to rotate or regenerate them. Problem - A leaked key exposes every bucket the account owns, not just one. - You can't delegate access to a single bucket without handing over access to all of them. - There's no safe way (or any way at all, from what I can see) to rotate a credential, since there's only one and revoking it breaks every client at once. Proposal 1. Per-bucket credentials: each bucket gets its own backing identity rather than sharing the account's. This contains the impact of a leaked key to a single bucket. 2. Two key pairs per bucket, with rotate and revoke: two independent slots so an operator can rotate one while clients continue on the other, then cut over and revoke the old one. This mirrors Azure's two-key model and AWS's two-access-keys-per-user. I'd keep the backing primitive as plain users (RGW/MinIO/Cloudian all support multiple keys per user), rather than the newer IAM-account constructs, to keep the change contained and provider-agnostic. Compatibility / migration - Existing buckets are untouched on upgrade. The bucket gains a nullable credential association; when null, it falls back to the account key exactly as today. - New buckets get their own credentials by default. - Operators opt in to migrating existing buckets via an explicit action (single and bulk), so they choose when any disruption happens rather than the upgrade forcing it. - Every supported S3 backend would need code changes which would be larger than I can take on practically, but also because I only have access to Ceph. I could develop for Ceph, and the others would need to be a roadmap item. Alternatively, other people familiar with those systems could develop code and test. Ideally everything would move together, but I'm not confident that we could have someone contribute for the MinIO and Cloudian backends, and I also see that there's an open PR for Dell ECS support (https://github.com/apache/cloudstack/pull/12124), so that would need updating as well. Future direction (I'm proposing it be out of scope here, for now) The longer-term destination might be an IAM / service-account style model where identities are decoupled from buckets and bound via policy, which is where AWS, Azure, GCP and now Ceph (accounts in Squid) all sit. That requires a significant amount of change in CloudStack and the introduction of IAM and granular policy, so I'm deliberately not tackling it now. This proposal is designed as a step towards it rather than against it: once credentials are their own entity, a many-to-many with permissions is an extension rather than a rewrite. Rough scope is a moderate amount of change touching the object store driver interface, the ceph provider plugin (ideally all the three provider plugins), the bucket schema, a new API command + UI tab, and an upgrade path. Does this seem like a reasonable direction? Thanks, Mitch
