Hi Bogdan,
This looks very much like a cpu scheduler lockup, as many of the processes
belonging to the container are in R state but not running.
Can you try resetting the cpulimit for the container in question,
something like
vzctl set $CTID --cpulimit 0
and see if anything changes?
Also, take a look at cpu.stat for some of the processes that is in such
state?
Say, this one:
root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00 vzctl
exec 111 ps
cat /proc/vz/fairsched/107398/cpu.stat
If throttled_time is big, it means my hypothesis makes sense.
I am also ccing Vladimir, who knows a thing or two about our fair cpu
scheduler.
Kir.
On 02/04/2016 05:48 AM, Bogdan-Stefan Rotariu wrote:
Hi there,
We are having issues with one container that cannot be
stopped/suspended or killed, all commands remain in Sleep or Running
Sleep.
Any ideea how to stop this container withour rebooting the main machine?
We did try to kill all proceeses, they do not die.
CTID NPROC STATUS IP_ADDR HOSTNAME
111 100 running a.b.c.d server.name
[3839648.976835] CPT ERR: ffff8803dd109000,111 :foreign process
14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
[3839648.976842] CPT ERR: ffff8803dd109000,111 :suspend is impossible
now.
[3839649.977756] CPT ERR: ffff8803dd109000,111 :foreign process
14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
[3839649.977764] CPT ERR: ffff8803dd109000,111 :suspend is impossible
now.
[3839650.978718] CPT ERR: ffff8803dd109000,111 :foreign process
14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
[3839650.978726] CPT ERR: ffff8803dd109000,111 :suspend is impossible
now.
[3839665.639557] CPT ERR: ffff880839216000,111 :foreign process
14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
[3839665.639564] CPT ERR: ffff880839216000,111 :suspend is impossible
now.
[3839666.640019] CPT ERR: ffff880839216000,111 :foreign process
14243/9892(bash) inside CT (e.g. vzctl enter or vzctl exec).
root 19890 0.0 0.0 25460 376 ? Rs 03:34 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 39626 0.0 0.0 25460 376 ? Rs 03:44 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 65503 0.0 0.0 27560 412 ? Rs 11:59 0:00
vzctl enter 111
root 65508 0.0 0.0 27560 416 ? Rs 11:59 0:00
vzctl enter 111
root 65522 0.0 0.0 27560 416 ? Rs 11:59 0:00
vzctl enter 111
root 73329 0.0 0.0 25460 372 ? Rs 12:00 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 73371 0.0 0.0 25460 380 ? Rs 12:00 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 74865 0.0 0.0 25464 408 ? Rs 12:00 0:00
vzctl stop 111
root 75864 0.0 0.0 25464 412 ? Rs 12:04 0:00
vzctl stop 111
root 85384 0.0 0.0 25464 404 ? Rs 12:08 0:00
vzctl stop 111
root 96674 0.0 0.0 25464 412 ? Rs 12:12 0:00
vzctl stop 111
root 96787 0.0 0.0 25464 408 ? Rs 12:13 0:00
vzctl stop 111 --fast
root 107300 0.0 0.0 27560 412 ? Rs 12:18 0:00
vzctl enter 111
root 107398 0.0 0.0 25460 396 ? Rs 12:19 0:00
vzctl exec 111 ps
root 116638 0.0 0.0 108168 1368 ? S 12:21 0:00 sh
-c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
'MemFree' | awk '{print $2}'
root 116639 0.0 0.0 25460 1024 ? S 12:21 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 116642 0.0 0.0 25460 364 ? S 12:21 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 116643 0.0 0.0 25460 384 ? Rs 12:21 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 116650 0.0 0.0 25460 384 ? Rs 12:21 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117653 0.0 0.0 25460 380 ? Rs 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117746 0.0 0.0 108168 1368 ? S 12:22 0:00 sh
-c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
'MemFree' | awk '{print $2}'
root 117747 0.0 0.0 108168 1368 ? S 12:22 0:00 sh
-c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
'MemFree' | awk '{print $2}'
root 117748 0.0 0.0 25460 1016 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117749 0.0 0.0 25460 1020 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117754 0.0 0.0 25460 360 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117755 0.0 0.0 25460 356 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117756 0.0 0.0 25460 380 ? Rs 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 117757 0.0 0.0 25460 376 ? Rs 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 118191 0.0 0.0 108168 1372 ? S 12:22 0:00 sh
-c /usr/sbin/vzctl exec 111 cat /proc/meminfo | grep --max-count=1
'MemTotal' | awk '{print $2}'
root 118192 0.0 0.0 25460 1020 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 118195 0.0 0.0 25460 360 ? S 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 118196 0.0 0.0 25460 380 ? Rs 12:22 0:00
/usr/sbin/vzctl exec 111 cat /proc/meminfo
root 126585 0.0 0.0 25464 408 ? Rs 12:25 0:00
vzctl stop 111
root 129412 0.0 0.0 25464 352 ? Rs 12:26 0:00
vzctl stop 111
root 138146 0.0 0.0 25464 404 ? Rs 12:28 0:00
vzctl stop 111
root 147844 0.0 0.0 25464 408 ? Rs 12:33 0:00
vzctl stop 111
root 157178 0.0 0.0 25464 412 ? Rs 12:36 0:00
vzctl stop 111
root 158300 0.0 0.0 25464 400 ? Rs 12:39 0:00
vzctl stop 111
root 179962 0.0 0.0 25464 408 ? Rs 12:49 0:00
vzctl stop 111
root 180039 0.0 0.0 25464 408 ? Rs 12:49 0:00
vzctl stop 111
root 220918 0.0 0.0 25464 412 ? Rs 13:04 0:00
vzctl stop 111
root 240631 0.0 0.0 25464 408 ? Rs 13:14 0:00
vzctl stop 111
root 247169 0.0 0.0 25464 412 ? Rs 13:15 0:00
vzctl stop 111
root 250371 0.0 0.0 25464 400 ? Rs 13:19 0:00
vzctl stop 111 --fast
_______________________________________________
Users mailing list
Users@openvz.org
https://lists.openvz.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@openvz.org
https://lists.openvz.org/mailman/listinfo/users
