I am developing a python application (using rbd.py) that requires querying
information about tens of thousands of rbd images. I have noticed that the
number of threads in my process grow linearly with each Image object that
is created.
After creating about 800 Image objects (that all share a single ioctx), my
process already has more than 2000 threads. I get the thread count using
`ps huH p <pid> | wc -l`.
If I call close() on each Image object after operating on it then the
threads are cleaned up. However, I want to cache these objects and reuse
them since it is expensive to create tens of thousands of these objects all
the time.
Is it correct for librbd to create 4-5 threads for each Image object
created?
For example, I'm doing something similar to:
-------------
import rbd
import rados
cluster = rados.Rados(conffile='my_ceph.conf')
cluster.connect()
ioctx = cluster.open_ioctx('mypool')
# With each object, new threads are created until close() is called
image0 = rbd.Image(ioctx, 'myimage0')
image1 = rbd.Image(ioctx, 'myimage1')
...
image9000 = rbd.Image(ioctx, 'myimage9000')
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
