Re: [openstack-dev] [Openstack] Glance on swift problem

Sławek Kapłoński Wed, 01 Oct 2014 01:06:09 -0700

Hello,

W dniu 2014-09-30 20:28, Timur Nurlygayanov napisał(a):

Hi Slawek,


we faced the same error and this is issue with Swift.
We can see 100% disk usage on the Swift node during the file upload
and looks like Swift can't send info about status of the file loading
in time.

On our environments we found the workaround for this issue:

1. Set  swift_store_large_object_size = 200 in glance.conf.

I have it set to 1024 - should it be lower? Value which I have is toobig now?


2. Add to Swift proxy-server.conf:

[DEFAULT]
 ...
 node_timeout = 90

Probably we can set this value as default value for this parameter
instead of '30'?

Regards,

Timur

On Tue, Sep 30, 2014 at 7:41 PM, Sławek Kapłoński
<sla...@kaplonski.pl> wrote:

Hello,

I can't find that upload from was previous logs but I now try to
upload same image once again. In glance there was exactly same
error. In swift logs I have:

Sep 30 17:35:10 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y
30/Sep/2014/15/35/10 HEAD
/v1/AUTH_7ef5a7661ccd4c069e3ad387a6dceebd/glance HTTP/1.0 204
Sep 30 17:35:16 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y
30/Sep/2014/15/35/16 PUT

/v1/AUTH_7ef5a7661ccd4c069e3ad387a6dceebd/glance/fa5dfe09-74f5-4287-9852-d2f1991eebc0-00001

HTTP/1.0 201 - -

Best regards
Slawek Kaplonski

W dniu 2014-09-30 17:03, Kuo Hugo napisał(a):

Hi ,

Could you please post the log of related requests in Swift's log
???

Thanks // Hugo

2014-09-30 22:20 GMT+08:00 Sławek Kapłoński
<sla...@kaplonski.pl>:

Hello,

I'm using openstack havana release and glance with swift backend.
Today I found that I have problem when I create image with url in
"--copy-from" when image is bigger than my
"swift_store_large_object_size" because then glance is trying to
split image to chunks with size given in
"swift_store_large_object_chunk_size" and when try to upload first
chunk to swift I have error:

2014-09-30 15:05:29.361 18023 ERROR glance.store.swift [-] Error
during chunked upload to backend, deleting stale chunks
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift Traceback
(most recent call last):
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
"/usr/lib/python2.7/dist-packages/glance/store/swift.py", line 384,
in add
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   
 content_length=content_length)
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
"/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
1234,
in put_object
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   
 response_dict=response_dict)
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
"/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
1143,
in _retry
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   
 reset_func(func, *args, **kwargs)
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
"/usr/lib/python2.7/dist-packages/swiftclient/client.py", line
1215,
in _default_reset
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift     %
(container, obj))
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
ClientException: put_object('glance',
'9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
ability to reset contents for reupload.
2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
2014-09-30 15:05:29.362 18023 ERROR glance.store.swift [-] Failed
to add object to Swift.
Got error from Swift: put_object('glance',
'9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
ability to reset contents for reupload.
2014-09-30 15:05:29.362 18023 ERROR glance.api.v1.upload_utils [-]
Failed to upload image 9f56ccec-deeb-4020-95ba-ca7bf1170056
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
Traceback (most recent call last):
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils 
 File
"/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py",
line 101, in upload_data_to_store
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils   
 store)
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils 
 File "/usr/lib/python2.7/dist-packages/glance/store/__init__.py",
line 333, in store_add_to_backend
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils   
 (location, size, checksum, metadata) = store.add(image_id, data,
size)
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils 
 File "/usr/lib/python2.7/dist-packages/glance/store/swift.py",
line 447, in add
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils   
 raise glance.store.BackendException(msg)
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
BackendException: Failed to add object to Swift.
2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils Got
error from Swift: put_object('glance',
'9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
ability to reset contents for reupload.

Does someone of You got same error and know what is solution of it?
I was searching about that in google but I not found anything what
could solve my problem.

--
Best regards
Sławek Kapłoński
sla...@kaplonski.pl

_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
[1]
Post to     : openst...@lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
[1]

Links:
------
[1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
[1]


 --
 Pozdrawiam
 Sławek Kapłoński
 sla...@kaplonski.pl

 _______________________________________________
 Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
 Post to     : openst...@lists.openstack.org
 Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]

 _______________________________________________
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev [2]

--

Timur,
QA Engineer
OpenStack Projects
Mirantis Inc

Links:
------
[1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
[2] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Pozdrawiam
Sławek Kapłoński
sla...@kaplonski.pl

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] Glance on swift problem

Reply via email to