Hi there,
I’m having an issue enabling Glance scrubber in Icehouse:
$ service openstack-glance-scrubber status
openstack-glance-scrubber dead but subsys locked
I’ve tried:
- checking the PID, but there are no running processes found:
$ cat /var/run/glance/glance-scrubber.pid
2834
$ kill -9 2834
bash: kill: (2834) - No such process
- removing the lockfile and restarting the process, but it goes back to “dead
but subsys locked” immediately:
$ service openstack-glance-scrubber status
openstack-glance-scrubber dead but subsys locked
My Glance scrubber config is as follows:
[DEFAULT]
# Show more verbose log output (sets INFO log level output)
verbose = True
# Show debugging output in logs (sets DEBUG log level output)
debug = False
# Log to this directory
log_dir=/var/log/glance
# Format string to use for log messages with context. (string
# value)
logging_context_format_string=%(asctime)s.%(msecs)03d %(levelname)s %(message)s
pid[%(process)s] %(request_id)s %(instance)s
# Format string to use for log messages without context.
# (string value)
logging_default_format_string=%(asctime)s.%(msecs)03d %(levelname)s %(message)s
pid[%(process)s]
# Data to append to log format when level is DEBUG. (string
# value)
logging_debug_format_suffix=%(funcName)s %(pathname)s:%(lineno)d
# Prefix each line of exception output with this format.
# (string value)
logging_exception_prefix=%(asctime)s.%(msecs)03d TRACE instance[%(instance)s]
# Should we run our own loop or rely on cron/scheduler to run us
daemon = False
# Loop time between checking the db for new items to schedule for delete
wakeup_time = 300
# Directory that the scrubber will use to remind itself of what to delete
# Make sure this is also set in glance-api.conf
scrubber_datadir = /data/var/lib/glance/scrubber
# Only one server in your deployment should be designated the cleanup host
cleanup_scrubber = False
# pending_delete items older than this time are candidates for cleanup
cleanup_scrubber_time = 86400
# Address to find the registry server for cleanups
registry_host = os-controller.wd5-cedev21.az2.eng.pdx.wd
# Port the registry server is listening on
registry_port = 9191
# ============ Filesystem Store Options ========================
# Directory that the Filesystem backend store
# writes image data to
filesystem_store_datadir = /data/var/lib/glance/images
When I attempted to launch “glance-scrubber”, the output to scrubber.log is as
follows:
2016-06-27 13:34:36.953 WARNING Deprecated: glance.store.rbd.Store not found in
`known_store`. Stores need to be explicitly enabled in the configuration file.
pid[3983]
2016-06-27 13:34:36.954 WARNING Failed to configure store correctly: Store
gridfs could not be configured correctly. Reason: Missing dependencies: pymongo
Disabling add method. pid[3983]
2016-06-27 13:34:36.955 WARNING Deprecated: glance.store.gridfs.Store not found
in `known_store`. Stores need to be explicitly enabled in the configuration
file. pid[3983]
2016-06-27 13:34:36.993 WARNING Failed to configure store correctly: Store
cinder could not be configured correctly. Reason: Cinder storage requires a
context. Disabling add method. pid[3983]
2016-06-27 13:34:36.994 WARNING Deprecated: glance.store.cinder.Store not found
in `known_store`. Stores need to be explicitly enabled in the configuration
file. pid[3983]
2016-06-27 13:34:37.000 ERROR Could not find swift_store_auth_address in
configuration options. pid[3983]
2016-06-27 13:34:37.001 WARNING Failed to configure store correctly: Store
swift could not be configured correctly. Reason: Could not find
swift_store_auth_address in configuration options. Disabling add method.
pid[3983]
2016-06-27 13:34:37.001 WARNING Deprecated: glance.store.swift.Store not found
in `known_store`. Stores need to be explicitly enabled in the configuration
file. pid[3983]
2016-06-27 13:34:37.004 WARNING Failed to configure store correctly: Store s3
could not be configured correctly. Reason: Could not find s3_store_host in
configuration options. Disabling add method. pid[3983]
2016-06-27 13:34:37.005 WARNING Deprecated: glance.store.s3.Store not found in
`known_store`. Stores need to be explicitly enabled in the configuration file.
pid[3983]
2016-06-27 13:34:37.007 INFO Initializing scrubber with configuration:
{'cleanup_time': 86400, 'registry_host':
'os-controller.wd5-cedev21.az2.eng.pdx.wd', 'cleanup': False,
'scrubber_datadir': '/data/var/lib/glance/scrubber', 'registry_port': 9191}
pid[3983]
Despite glance-scrubber not working, there is not a massive discrepancy between
active glance images & those stored locally on the filesystem:
$ ls /data/var/lib/glance/images/ | wc -l
1094
$ glance image-list --all-tenants | grep active | wc -l
1083
My questions are:
- i) Based upon my config and stack trace, does anyone have any ideas on why
the Glance scrubber process would be stuck in a "dead but subsys locked” state
- ii) If glance-scrubber is not working, are these images being cleaned up some
other way (i.e. through Glance itself)?
Thanks.
-- mickm
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
