I've added Nova to the projects here because currently we're at a stale
mate where there seems to be a single case during unrescue that tirggers
this. Patc is proposed but looks like it won't be accepted, want to
make sure we link this and keep it tracked although it is different than
the original bug posted by Vish, I believe it's related.
** Project changed: cinder => nova
** Changed in: nova
Milestone: kilo-3 => None
** Also affects: cinder
Importance: Undecided
Status: New
** Changed in: cinder
Status: New => In Progress
** Changed in: cinder
Importance: Undecided => Critical
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1373513
Title:
Lvm hang during tempest tests
Status in Cinder:
In Progress
Status in OpenStack Compute (Nova):
In Progress
Bug description:
Managed to trigger a hang in lvm create
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.704164]
INFO: task lvm:14805 blocked for more than 120 seconds.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.705096]
Not tainted 3.13.0-35-generic #62-Ubuntu
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.705839]
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706871] lvm
D ffff8801ffd14440 0 14805 14804 0x00000000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706876]
ffff880068f9dae0 0000000000000082 ffff8801a14bc800 ffff880068f9dfd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706879]
0000000000014440 0000000000014440 ffff8801a14bc800 ffff8801ffd14cd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706881]
0000000000000000 ffff88004063c280 0000000000000000 ffff8801a14bc800
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706883]
Call Trace:
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706895]
[<ffffffff81722a6d>] io_schedule+0x9d/0x140
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706914]
[<ffffffff811fac94>] do_blockdev_direct_IO+0x1ce4/0x2910
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706918]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706920]
[<ffffffff811fb915>] __blockdev_direct_IO+0x55/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706922]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706924]
[<ffffffff811f61f6>] blkdev_direct_IO+0x56/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706926]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706937]
[<ffffffff8115106b>] generic_file_aio_read+0x69b/0x700
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706947]
[<ffffffff811cca78>] ? path_openat+0x158/0x640
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706953]
[<ffffffff810f3c92>] ? from_kgid_munged+0x12/0x20
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706955]
[<ffffffff811f667b>] blkdev_aio_read+0x4b/0x70
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706958]
[<ffffffff811bc99a>] do_sync_read+0x5a/0x90
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706960]
[<ffffffff811bd035>] vfs_read+0x95/0x160
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706962]
[<ffffffff811bdb49>] SyS_read+0x49/0xa0
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706966]
[<ffffffff8172ed6d>] system_call_fastpath+0x1a/0x1f
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.706968]
INFO: task lvs:14822 blocked for more than 120 seconds.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.707774]
Not tainted 3.13.0-35-generic #62-Ubuntu
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.708507]
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709535] lvs
D ffff8801ffc14440 0 14822 14821 0x00000000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709537]
ffff880009ffdae0 0000000000000082 ffff8800095e1800 ffff880009ffdfd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709539]
0000000000014440 0000000000014440 ffff8800095e1800 ffff8801ffc14cd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709541]
0000000000000000 ffff880003d59900 0000000000000000 ffff8800095e1800
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709543]
Call Trace:
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709547]
[<ffffffff81722a6d>] io_schedule+0x9d/0x140
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709549]
[<ffffffff811fac94>] do_blockdev_direct_IO+0x1ce4/0x2910
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709551]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709554]
[<ffffffff811fb915>] __blockdev_direct_IO+0x55/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709555]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709557]
[<ffffffff811f61f6>] blkdev_direct_IO+0x56/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709559]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709562]
[<ffffffff8115106b>] generic_file_aio_read+0x69b/0x700
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709564]
[<ffffffff811cca78>] ? path_openat+0x158/0x640
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709566]
[<ffffffff810f3c92>] ? from_kgid_munged+0x12/0x20
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709568]
[<ffffffff811f667b>] blkdev_aio_read+0x4b/0x70
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709570]
[<ffffffff811bc99a>] do_sync_read+0x5a/0x90
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709572]
[<ffffffff811bd035>] vfs_read+0x95/0x160
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709573]
[<ffffffff811bdb49>] SyS_read+0x49/0xa0
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709575]
[<ffffffff8172ed6d>] system_call_fastpath+0x1a/0x1f
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.709578]
INFO: task lvs:14984 blocked for more than 120 seconds.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.710411]
Not tainted 3.13.0-35-generic #62-Ubuntu
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.711153]
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712193] lvs
D ffff8801ffc94440 0 14984 14983 0x00000000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712196]
ffff880101711ae0 0000000000000086 ffff880007d2b000 ffff880101711fd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712198]
0000000000014440 0000000000014440 ffff880007d2b000 ffff8801ffc94cd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712200]
0000000000000000 ffff880017fd2300 0000000000000000 ffff880007d2b000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712202]
Call Trace:
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712219]
[<ffffffff81722a6d>] io_schedule+0x9d/0x140
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712221]
[<ffffffff811fac94>] do_blockdev_direct_IO+0x1ce4/0x2910
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712224]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712226]
[<ffffffff811fb915>] __blockdev_direct_IO+0x55/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712228]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712230]
[<ffffffff811f61f6>] blkdev_direct_IO+0x56/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712231]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712234]
[<ffffffff8115106b>] generic_file_aio_read+0x69b/0x700
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712236]
[<ffffffff811cca78>] ? path_openat+0x158/0x640
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712238]
[<ffffffff810f3c92>] ? from_kgid_munged+0x12/0x20
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712249]
[<ffffffff811f667b>] blkdev_aio_read+0x4b/0x70
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712256]
[<ffffffff811bc99a>] do_sync_read+0x5a/0x90
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712264]
[<ffffffff811bd035>] vfs_read+0x95/0x160
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712270]
[<ffffffff811bdb49>] SyS_read+0x49/0xa0
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712276]
[<ffffffff8172ed6d>] system_call_fastpath+0x1a/0x1f
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.712278]
INFO: task vgs:15086 blocked for more than 120 seconds.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.713149]
Not tainted 3.13.0-35-generic #62-Ubuntu
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.713870]
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714919] vgs
D ffff8801ffc94440 0 15086 15061 0x00000000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
ffff88014411fae0 0000000000000082 ffff8801f2f7e000 ffff88014411ffd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
0000000000014440 0000000000014440 ffff8801f2f7e000 ffff8801ffc94cd8
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
0000000000000000 ffff880017fd2580 0000000000000000 ffff8801f2f7e000
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
Call Trace:
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
[<ffffffff81722a6d>] io_schedule+0x9d/0x140
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
[<ffffffff811fac94>] do_blockdev_direct_IO+0x1ce4/0x2910
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714921]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714923]
[<ffffffff811fb915>] __blockdev_direct_IO+0x55/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714924]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714926]
[<ffffffff811f61f6>] blkdev_direct_IO+0x56/0x60
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714928]
[<ffffffff811f5b00>] ? I_BDEV+0x10/0x10
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714930]
[<ffffffff8115106b>] generic_file_aio_read+0x69b/0x700
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714932]
[<ffffffff811cca78>] ? path_openat+0x158/0x640
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714934]
[<ffffffff810f3c92>] ? from_kgid_munged+0x12/0x20
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714936]
[<ffffffff811f667b>] blkdev_aio_read+0x4b/0x70
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714938]
[<ffffffff811bc99a>] do_sync_read+0x5a/0x90
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714940]
[<ffffffff811bd035>] vfs_read+0x95/0x160
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714942]
[<ffffffff811bdb49>] SyS_read+0x49/0xa0
Sep 22 21:19:01 devstack-trusty-hpcloud-b5-2344254 kernel: [ 3120.714944]
[<ffffffff8172ed6d>] system_call_fastpath+0x1a/0x1f
Link to logs: http://logs.openstack.org/22/119522/9/check/check-
tempest-dsvm-postgres-full/46a5f8c/logs/syslog.txt.gz#_Sep_22_21_19_01
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1373513/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
