Public bug reported:
Attempting to create a snapshot of a shutdown instance fails. It seems
nova assumes the instance exists and is running when attempting to
create the snapshot.
# Steps to reproduce
$ openstack server create \
--os-compute-api-version=2.latest --flavor m1.tiny --image
cirros-0.5.1-x86_64-disk \
--nic none --wait test.server
$ openstack server stop test.server
$ openstack server image create test.server
# Expected result
A snapshot of the instance root disk should be created.
# Actual result
The snapshot is not created. Attempts to resume the instance fail:
$ openstack server start test.server
Cannot 'start' instance aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda while it is in
task_state image_pending_upload (HTTP 409) (Request-ID:
req-39d4bd58-366b-4b93-8d7d-72a487183088)
# Additional details
I see the following in the logs:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance instance-0000000c disappeared
while taking snapshot of it: [Error Code 42] Domain not found: no domain with
matching uuid 'aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda' (instance-0000000c)
nova-compute[20898]: DEBUG nova.compute.manager [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance disappeared during snapshot
{{(pid=20898) _snapshot_instance /opt/stack/nova/nova/compute/manager.py:3874}}
Compare with logs from snapshot of a running guest:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: DEBUG nova.privsep.utils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Path
'/opt/stack/data/nova/instances' supports direct I/O {{(pid=20898)
supports_direct_io /opt/stack/nova/nova/privsep/utils.py:64}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Running cmd (subprocess):
qemu-img convert -t none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581
{{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:371}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] CMD "qemu-img convert -t
none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581"
returned: 0 in 0.403s {{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:408}}
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot extracted, beginning image upload
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot image upload complete
nova-compute[20898]: INFO nova.compute.manager [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Took 2.44 seconds to snapshot the
instance on the hypervisor.
We see the same issue if the issues is suspended ('openstack server
suspend'). There are no issues if the instance is paused, however
('openstack server pause') but
** Affects: nova
Importance: Medium
Status: Confirmed
** Tags: libvirt snapshot
** Tags added: libvirt snapshot
** Changed in: nova
Status: New => Confirmed
** Changed in: nova
Importance: Undecided => Medium
** Description changed:
Attempting to create a snapshot of a shutdown instance fails. It seems
nova assumes the instance exists and is running when attempting to
create the snapshot.
# Steps to reproduce
- $ openstack server create \
- --os-compute-api-version=2.latest --flavor m1.tiny --image
cirros-0.5.1-x86_64-disk \
- --nic none --wait test.server
- $ openstack server stop test.server
- $ openstack server image create test.server
+ $ openstack server create \
+ --os-compute-api-version=2.latest --flavor m1.tiny --image
cirros-0.5.1-x86_64-disk \
+ --nic none --wait test.server
+ $ openstack server stop test.server
+ $ openstack server image create test.server
# Expected result
A snapshot of the instance root disk should be created.
# Actual result
The snapshot is not created. Attempts to resume the instance fail:
- $ openstack server start test.server
- Cannot 'start' instance aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda while it is in
task_state image_pending_upload (HTTP 409) (Request-ID:
req-39d4bd58-366b-4b93-8d7d-72a487183088)
+ $ openstack server start test.server
+ Cannot 'start' instance aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda while it is in
task_state image_pending_upload (HTTP 409) (Request-ID:
req-39d4bd58-366b-4b93-8d7d-72a487183088)
# Additional details
I see the following in the logs:
- nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
- nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance instance-0000000c disappeared
while taking snapshot of it: [Error Code 42] Domain not found: no domain with
matching uuid 'aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda' (instance-0000000c)
- nova-compute[20898]: DEBUG nova.compute.manager [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance disappeared during snapshot
{{(pid=20898) _snapshot_instance /opt/stack/nova/nova/compute/manager.py:3874}}
+ nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
+ nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance instance-0000000c disappeared
while taking snapshot of it: [Error Code 42] Domain not found: no domain with
matching uuid 'aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda' (instance-0000000c)
+ nova-compute[20898]: DEBUG nova.compute.manager [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance disappeared during snapshot
{{(pid=20898) _snapshot_instance /opt/stack/nova/nova/compute/manager.py:3874}}
Compare with logs from snapshot of a running guest:
- nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
- nova-compute[20898]: DEBUG nova.privsep.utils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Path
'/opt/stack/data/nova/instances' supports direct I/O {{(pid=20898)
supports_direct_io /opt/stack/nova/nova/privsep/utils.py:64}}
- nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Running cmd (subprocess):
qemu-img convert -t none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581
{{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:371}}
- nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] CMD "qemu-img convert -t
none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581"
returned: 0 in 0.403s {{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:408}}
- nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot extracted, beginning image upload
- nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot image upload complete
- nova-compute[20898]: INFO nova.compute.manager [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Took 2.44 seconds to snapshot the
instance on the hypervisor.
+ nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
+ nova-compute[20898]: DEBUG nova.privsep.utils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Path
'/opt/stack/data/nova/instances' supports direct I/O {{(pid=20898)
supports_direct_io /opt/stack/nova/nova/privsep/utils.py:64}}
+ nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Running cmd (subprocess):
qemu-img convert -t none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581
{{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:371}}
+ nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] CMD "qemu-img convert -t
none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581"
returned: 0 in 0.403s {{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:408}}
+ nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot extracted, beginning image upload
+ nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot image upload complete
+ nova-compute[20898]: INFO nova.compute.manager [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Took 2.44 seconds to snapshot the
instance on the hypervisor.
+
+ Things also work if the instance is paused ('openstack server pause')
** Summary changed:
- Snapshot of stopped instance fails
+ Snapshot of stopped, suspended instance fails
** Description changed:
Attempting to create a snapshot of a shutdown instance fails. It seems
nova assumes the instance exists and is running when attempting to
create the snapshot.
# Steps to reproduce
$ openstack server create \
--os-compute-api-version=2.latest --flavor m1.tiny --image
cirros-0.5.1-x86_64-disk \
--nic none --wait test.server
$ openstack server stop test.server
$ openstack server image create test.server
# Expected result
A snapshot of the instance root disk should be created.
# Actual result
The snapshot is not created. Attempts to resume the instance fail:
$ openstack server start test.server
Cannot 'start' instance aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda while it is in
task_state image_pending_upload (HTTP 409) (Request-ID:
req-39d4bd58-366b-4b93-8d7d-72a487183088)
# Additional details
I see the following in the logs:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance instance-0000000c disappeared
while taking snapshot of it: [Error Code 42] Domain not found: no domain with
matching uuid 'aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda' (instance-0000000c)
nova-compute[20898]: DEBUG nova.compute.manager [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance disappeared during snapshot
{{(pid=20898) _snapshot_instance /opt/stack/nova/nova/compute/manager.py:3874}}
Compare with logs from snapshot of a running guest:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: DEBUG nova.privsep.utils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Path
'/opt/stack/data/nova/instances' supports direct I/O {{(pid=20898)
supports_direct_io /opt/stack/nova/nova/privsep/utils.py:64}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Running cmd (subprocess):
qemu-img convert -t none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581
{{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:371}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] CMD "qemu-img convert -t
none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581"
returned: 0 in 0.403s {{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:408}}
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot extracted, beginning image upload
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot image upload complete
nova-compute[20898]: INFO nova.compute.manager [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Took 2.44 seconds to snapshot the
instance on the hypervisor.
- Things also work if the instance is paused ('openstack server pause')
+ We see the same issue if the issues is suspended ('openstack server
+ suspend'). There are no issues if the instance is paused, however
+ ('openstack server pause') but
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1888414
Title:
Snapshot of stopped, suspended instance fails
Status in OpenStack Compute (nova):
Confirmed
Bug description:
Attempting to create a snapshot of a shutdown instance fails. It seems
nova assumes the instance exists and is running when attempting to
create the snapshot.
# Steps to reproduce
$ openstack server create \
--os-compute-api-version=2.latest --flavor m1.tiny --image
cirros-0.5.1-x86_64-disk \
--nic none --wait test.server
$ openstack server stop test.server
$ openstack server image create test.server
# Expected result
A snapshot of the instance root disk should be created.
# Actual result
The snapshot is not created. Attempts to resume the instance fail:
$ openstack server start test.server
Cannot 'start' instance aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda while it is in
task_state image_pending_upload (HTTP 409) (Request-ID:
req-39d4bd58-366b-4b93-8d7d-72a487183088)
# Additional details
I see the following in the logs:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance instance-0000000c disappeared
while taking snapshot of it: [Error Code 42] Domain not found: no domain with
matching uuid 'aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda' (instance-0000000c)
nova-compute[20898]: DEBUG nova.compute.manager [None
req-0b7dfe74-d465-4c2b-90e3-54ed26ea4244 demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Instance disappeared during snapshot
{{(pid=20898) _snapshot_instance /opt/stack/nova/nova/compute/manager.py:3874}}
Compare with logs from snapshot of a running guest:
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Skipping quiescing instance: QEMU guest
agent is not enabled.
nova-compute[20898]: DEBUG nova.privsep.utils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Path
'/opt/stack/data/nova/instances' supports direct I/O {{(pid=20898)
supports_direct_io /opt/stack/nova/nova/privsep/utils.py:64}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] Running cmd (subprocess):
qemu-img convert -t none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581
{{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:371}}
nova-compute[20898]: DEBUG oslo_concurrency.processutils [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] CMD "qemu-img convert -t
none -O qcow2 -f qcow2
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581.delta
/opt/stack/data/nova/instances/snapshots/tmpal0gmbcx/d64bd94655da448495d69b274ca14581"
returned: 0 in 0.403s {{(pid=20898) execute
/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py:408}}
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot extracted, beginning image upload
nova-compute[20898]: INFO nova.virt.libvirt.driver [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Snapshot image upload complete
nova-compute[20898]: INFO nova.compute.manager [None
req-79e59ed6-4558-42be-a016-09ff2c4d60cb demo admin] [instance:
aa82c7a9-dbc1-4c7b-b3f1-8dc6b83e1bda] Took 2.44 seconds to snapshot the
instance on the hypervisor.
We see the same issue if the issues is suspended ('openstack server
suspend'). There are no issues if the instance is paused, however
('openstack server pause') but
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1888414/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
