Public bug reported:

When upgrading runc to the latest version from focal-updates, Kubernetes, using 
containerd, fails to start new containers after a certain amount of container 
have been reached a certain amount (more than 100 ~ 150 containers).
With the previous version of runc, I was able to run more than 340 containers 
on a single server without any issue.

I got those logs in containerd (journalctl -u containerd)
```
May 05 00:48:17 node6 containerd[1368656]: 
time="2021-05-05T00:48:17.260349619Z" level=warning msg="could not use 
snapshotter devmapper in metadata plugin" error="devmapper not configured"
May 05 00:48:21 node6 containerd[1368656]: 
time="2021-05-05T00:48:21.807947009Z" level=error msg="get state for 
71005cc34ac16f6976a7aebd72faa734792639b003c5dec67f666449cf6eb8d7" 
error="context deadline exceeded: unknown"
May 05 00:48:23 node6 containerd[1368656]: 
time="2021-05-05T00:48:23.810434820Z" level=error msg="get state for 
71005cc34ac16f6976a7aebd72faa734792639b003c5dec67f666449cf6eb8d7" 
error="context deadline exceeded: unknown"
May 05 00:48:25 node6 containerd[1368656]: 
time="2021-05-05T00:48:25.813453574Z" level=error msg="get state for 
c6497bbf82f668f8b8b534204cd2efaa30a97f884da3dd5fca4f51489298f855" 
error="context deadline exceeded: unknown"
May 05 00:48:27 node6 containerd[1368656]: 
time="2021-05-05T00:48:27.816285268Z" level=error msg="get state for 
c6497bbf82f668f8b8b534204cd2efaa30a97f884da3dd5fca4f51489298f855" 
error="context deadline exceeded: unknown"
May 05 00:48:29 node6 containerd[1368656]: 
time="2021-05-05T00:48:29.823052797Z" level=error msg="get state for 
f90f054966ab6b78304797d420a1f948c85e5b6e425037417f8966c08cd9a0f1" 
error="context deadline exceeded: unknown"
May 05 00:48:31 node6 containerd[1368656]: 
time="2021-05-05T00:48:31.826412096Z" level=error msg="get state for 
f90f054966ab6b78304797d420a1f948c85e5b6e425037417f8966c08cd9a0f1" 
error="context deadline exceeded: unknown"
```

This version of runc triggered the problem:
```
runc (1.0.0~rc93-0ubuntu1~20.04.1) focal; urgency=medium

  * Backport version 1.0.0~rc93-0ubuntu1 from Hirsute (LP: #1919322,
    LP: #1916485).

 -- Lucas Kanashiro <kanash...@ubuntu.com>  Tue, 16 Mar 2021 15:34:35 -0300
```

```
# runc -v
runc version spec: 1.0.2-dev
go: go1.13.8
libseccomp: 2.5.1
```


Reverting to the previous version of runc solved the problem, and I was able to 
run more than 340 pods / containers without any error.
```
apt-get install runc=1.0.0~rc10-0ubuntu1

# runc -v
runc version spec: 1.0.1-dev
```

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: runc 1.0.0~rc93-0ubuntu1~20.04.1
ProcVersionSignature: Ubuntu 5.4.0-72.80-generic 5.4.101
Uname: Linux 5.4.0-72-generic x86_64
ApportVersion: 2.20.11-0ubuntu27.16
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed May  5 12:06:30 2021
SourcePackage: runc
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: runc (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug focal uec-images

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1927219

Title:
  context deadline exceeded: unknown in containerd with latest runc
  version

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/runc/+bug/1927219/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to