Hello, Thanks for Your help but it not helps. I checked that for sure on each swift node there is a lot of free space. What can confirm that is fact that when I try to create image with size about 1.7GB and I have swift_store_large_object_size set to 1GB than there is error (always after send first chunk to swift (200MB). When I only change swift_store_large_object_size to 2GB and restart glance-api than the same image is created correctly (it is then in one big object).
--- Best regards Sławek Kapłoński [email protected] Dnia wtorek, 30 września 2014 22:28:11 Timur Nurlygayanov pisze: > Hi Slawek, > > we faced the same error and this is issue with Swift. > We can see 100% disk usage on the Swift node during the file upload and > looks like Swift can't send info about status of the file loading in time. > > On our environments we found the workaround for this issue: > 1. Set swift_store_large_object_size = 200 in glance.conf. > 2. Add to Swift proxy-server.conf: > > [DEFAULT] > ... > node_timeout = 90 > > Probably we can set this value as default value for this parameter instead > of '30'? > > > Regards, > Timur > > > On Tue, Sep 30, 2014 at 7:41 PM, Sławek Kapłoński <[email protected]> > > wrote: > > Hello, > > > > I can't find that upload from was previous logs but I now try to upload > > same image once again. In glance there was exactly same error. In swift > > logs I have: > > > > Sep 30 17:35:10 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y > > 30/Sep/2014/15/35/10 HEAD /v1/AUTH_7ef5a7661ccd4c069e3ad387a6dceebd/glance > > HTTP/1.0 204 > > Sep 30 17:35:16 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y > > 30/Sep/2014/15/35/16 PUT /v1/AUTH_7ef5a7661ccd4c069e3ad387a6dcee > > bd/glance/fa5dfe09-74f5-4287-9852-d2f1991eebc0-00001 HTTP/1.0 201 - - > > > > Best regards > > Slawek Kaplonski > > > > W dniu 2014-09-30 17:03, Kuo Hugo napisał(a): > >> Hi , > >> > >> Could you please post the log of related requests in Swift's log ??? > >> > >> Thanks // Hugo > >> > >> 2014-09-30 22:20 GMT+08:00 Sławek Kapłoński <[email protected]>: > >> Hello, > >> > >>> I'm using openstack havana release and glance with swift backend. > >>> Today I found that I have problem when I create image with url in > >>> "--copy-from" when image is bigger than my > >>> "swift_store_large_object_size" because then glance is trying to > >>> split image to chunks with size given in > >>> "swift_store_large_object_chunk_size" and when try to upload first > >>> chunk to swift I have error: > >>> > >>> 2014-09-30 15:05:29.361 18023 ERROR glance.store.swift [-] Error > >>> during chunked upload to backend, deleting stale chunks > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift Traceback > >>> (most recent call last): > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift File > >>> "/usr/lib/python2.7/dist-packages/glance/store/swift.py", line 384, > >>> in add > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift > >>> > >>> content_length=content_length) > >>> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift File > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1234, > >>> in put_object > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift > >>> > >>> response_dict=response_dict) > >>> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift File > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1143, > >>> in _retry > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift > >>> > >>> reset_func(func, *args, **kwargs) > >>> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift File > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1215, > >>> in _default_reset > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift % > >>> (container, obj)) > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift > >>> ClientException: put_object('glance', > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no > >>> ability to reset contents for reupload. > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift > >>> 2014-09-30 15:05:29.362 18023 ERROR glance.store.swift [-] Failed > >>> to add object to Swift. > >>> Got error from Swift: put_object('glance', > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no > >>> ability to reset contents for reupload. > >>> 2014-09-30 15:05:29.362 18023 ERROR glance.api.v1.upload_utils [-] > >>> Failed to upload image 9f56ccec-deeb-4020-95ba-ca7bf1170056 > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> Traceback (most recent call last): > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> File > >>> > >>> "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py", > >>> line 101, in upload_data_to_store > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> store) > >>> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> File "/usr/lib/python2.7/dist-packages/glance/store/__init__.py", > >>> > >>> line 333, in store_add_to_backend > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> (location, size, checksum, metadata) = store.add(image_id, data, > >>> > >>> size) > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> File "/usr/lib/python2.7/dist-packages/glance/store/swift.py", > >>> > >>> line 447, in add > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> > >>> raise glance.store.BackendException(msg) > >>> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils > >>> BackendException: Failed to add object to Swift. > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils Got > >>> error from Swift: put_object('glance', > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no > >>> ability to reset contents for reupload. > >>> > >>> Does someone of You got same error and know what is solution of it? > >>> I was searching about that in google but I not found anything what > >>> could solve my problem. > >>> > >>> -- > >>> Best regards > >>> Sławek Kapłoński > >>> [email protected] > >>> > >>> _______________________________________________ > >>> Mailing list: > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1] > >>> Post to : [email protected] > >>> Unsubscribe : > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1] > >> > >> Links: > >> ------ > >> [1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > > > -- > > Pozdrawiam > > Sławek Kapłoński > > [email protected] > > > > _______________________________________________ > > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/ > > openstack > > Post to : [email protected] > > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/ > > openstack > > _______________________________________________ > > OpenStack-dev mailing list > > [email protected] > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
