mw-0 opened a new issue, #14251:
URL: https://github.com/apache/cloudstack/issues/14251
### problem
When CKS provisions a Kubernetes cluster with external etcd, the generated
kubeadm configuration points the API server at etcd over plaintext HTTP with
no client-certificate authentication. The etcd client/peer endpoints are
served as `http://` and the kubeadm `external` etcd block has empty
`caFile`, `certFile`, and `keyFile` values:
etcd:
external:
caFile: ""
certFile: ""
keyFile: ""
endpoints:
- http://10.x.x.x:2379
- http://10.x.x.x:2379
- http://10.x.x.x:2379
httpEndpoints:
- http://10.x.x.x:2379
- http://10.x.x.x:2379
- http://10.x.x.x:2379
Because etcd is the Kubernetes API server's unconditionally-trusted source of
truth, any workload that can reach TCP 2379 can read and write cluster state
without authentication and over an unencrypted channel. This is effectively a
full-cluster-compromise primitive: unauthenticated read of all Secrets and
unauthenticated write of arbitrary objects.
As per Kubernetes Documentation here:
https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#securing-etcd-clusters
### versions
CKS (CloudStack Kubernetes Service) — external etcd configuration
### The steps to reproduce the bug
# 1. Reachability to the etcd client port
nc -vz 10.x.x.x 2379
# 2. etcd answers over cleartext HTTP (no TLS)
curl -v http://10.x.x.x:2379/health
# -> {"health":"true"} over plain HTTP
# 3. No client certificate required
ETCDCTL_API=3 etcdctl \
--endpoints=http://10.x.x.x:2379,http://10.x.x.x:2379,http://10.x.x.x:2379
\
endpoint status --write-out=table
ETCDCTL_API=3 etcdctl --endpoints=http://10.x.x.x:2379 member list
--write-out=table
# 4. Write access, proven non-destructively under a namespace k8s does not
use,
# then cleaned up (does NOT touch the /registry/ prefix):
ETCDCTL_API=3 etcdctl --endpoints=http://10.x.x.x:2379 \
put /pentest/poc "unauthenticated write proof"
ETCDCTL_API=3 etcdctl --endpoints=http://10.x.x.x:2379 get --prefix /pentest/
ETCDCTL_API=3 etcdctl --endpoints=http://10.x.x.x:2379 del --prefix /pentest/
# 5. Cleartext confirmation on the wire
tcpdump -i any -A -c 40 host 10.x.x.x and port 2379
### What to do about it?
**1. Immediate fix — mutual TLS on external etcd (this issue)**
CKS should provision external etcd with TLS and client-certificate
authentication, and populate the kubeadm `external` block with real key
material instead of empty strings:
etcd:
external:
caFile: /etc/kubernetes/pki/etcd/ca.crt
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
endpoints:
- https://10.x.x.x:2379
- https://10.x.x.x:2379
- https://10.x.x.x:2379
**2. Follow-up — automated certificate rotation (separate PR / issue)**
Introducing etcd mTLS adds certificates that expire, so the fix is only
durable
if renewal is automated. Without it, clusters will silently drift toward an
outage when the etcd CA or the `apiserver-etcd-client` certificate lapses,
and
operators may be tempted to revert to HTTP to "make it work again" — which
would
reintroduce this exact vulnerability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
