[Openstack] Understanding the Swift Ring, Ruby Edition
Hey folks, I've posted an article related to the swift ring format, and how to read it from Ruby. It's a work in progress and, since I'm not a Swift expert by any means, it'll be full of inaccuracies probably. I hope it can be of use to someone else in the meantime. http://rubiojr.rbel.co/hack/2013/02/06/understanding-the-swift-ring-ruby-edition Fixes and corrections greatly appreciated. Cheers. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [OpenStack][Swift] Cifrado 0.1 Swift CLI available
Hey folks, I've released a new OpenStack Swift command line client: http://rubiojr.rbel.co/cifrado Somewhat experimental, but here's the feature list for the first release: * Uploading/downloading files and directories to/from OpenStack Swift. * Regular list/delete/stat commands to manipulate objects and containers. * Asymmetric/Symmetric transparent encryption/decryption of files when uploading/downloading using GnuPG. * Segmented uploads (splitting the file in multiple segments) with (partial) resume support. * Bandwidth limits when uploading/downloading stuff. * Music streaming (streams mp3/ogg files available in a container) and plays them using mplayer/vlc/totem if locally installed. * Video streaming (streams video files available in a container). * Bootstrap a Swift All-In-One server in a cloud provider (DigitalOcean is the only provider one supported ATM). * Ruby 1.8.7, 1.9.X and 2.0 compatibility. Enjoy! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [OpenStack][Swift] Swift Storage nodes leave db.pending files after failed requests
Howdy folks, While populating an empty Swift test cluster with swift-dispersion-populate some container creation requests failed, leaving 'db.pending' files. This is standard operation I think (http://docs.openstack.org/developer/swift/overview_architecture.html#updaters) however the pending files are never removed (the updaters aren't picking up those changes?) and I can see periodic errors in the storage node error log: root@swift-002:/srv/node# find|grep pending ./cd6b760c-bf47-4733-b7bb-23659dd8ee1d/accounts/523880/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db.pending ./7e0cfcfc-7c6e-41a4-adc8-e4173147bd2a/accounts/523880/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db.pending ./7cf651d7-e2f7-4dc4-ada0-7cd0dc832449/containers/498724/d1d/bd66b837c7b9a95dec68b3ca05a75d1d/bd66b837c7b9a95dec68b3ca05a75d1d.db.pending Apr 23 11:06:26 swift-001 account-server ERROR __call__ error with PUT /d3829495-9f10-4075-a558-f99fb665cfe2/464510/AUTH_aeedb7bdcb8846599e2b6b87bb8a947f/dispersion_916bc3a9cc1548aca41be6633c1bd194 : #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 333, in __call__#012res = method(req)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1558, in wrapped#012return func(*a, **kw)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 520, in _timing_stats#012resp = func(ctrl, *args, **kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 112, in PUT#012req.headers['x-bytes-used'])#012 File "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 1431, in put_container#012raise DatabaseConnectionError(self.db_file, "DB doesn't exist")#012DatabaseConnectionError: DB connection error (/srv/node/d3829495-9f10-4075-a558-f99fb665cfe2/accounts/464510/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db, 0):#012DB doesn't exist The test cluster has two storage nodes with 12 drives each, packaged Swift 1.8.0 from Ubuntu Cloud Archive and the account/container configuration is the following: https://gist.github.com/rubiojr/6ea3d0ea0c4d00949d33 The cluster is fully operational and there are no other known issues ATM. I can easily reproduce the problem by emptying the cluster and populating it again with swift-dispersion-populate. I've exhausted all the possibilities (bug reports, mailing lists, google, etc) so I decided to ask before start digging the source code trying to gather evidence to open a bug report. If anyone can shed some light on the issue that would be great. Thanks! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Swift] Swift Storage nodes leave db.pending files after failed requests
For the sake of completion, the whole thing was a false alarm, has nothing to do with db.pending files and it has been documented here: https://bugs.launchpad.net/swift/+bug/1172358 On Tue, Apr 23, 2013 at 11:39 AM, Sergio Rubio wrote: > Howdy folks, > > While populating an empty Swift test cluster with > swift-dispersion-populate some container creation requests failed, > leaving 'db.pending' files. > > This is standard operation I think > (http://docs.openstack.org/developer/swift/overview_architecture.html#updaters) > however the pending files are never removed (the updaters aren't > picking up those changes?) and I can see periodic errors in the > storage node error log: > > > root@swift-002:/srv/node# find|grep pending > ./cd6b760c-bf47-4733-b7bb-23659dd8ee1d/accounts/523880/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db.pending > ./7e0cfcfc-7c6e-41a4-adc8-e4173147bd2a/accounts/523880/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db.pending > ./7cf651d7-e2f7-4dc4-ada0-7cd0dc832449/containers/498724/d1d/bd66b837c7b9a95dec68b3ca05a75d1d/bd66b837c7b9a95dec68b3ca05a75d1d.db.pending > > > > > Apr 23 11:06:26 swift-001 account-server ERROR __call__ error with PUT > /d3829495-9f10-4075-a558-f99fb665cfe2/464510/AUTH_aeedb7bdcb8846599e2b6b87bb8a947f/dispersion_916bc3a9cc1548aca41be6633c1bd194 > : #012Traceback (most recent call last):#012 File > "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 333, > in __call__#012res = method(req)#012 File > "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 1558, > in wrapped#012return func(*a, **kw)#012 File > "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 520, in > _timing_stats#012resp = func(ctrl, *args, **kwargs)#012 File > "/usr/lib/python2.7/dist-packages/swift/account/server.py", line 112, > in PUT#012req.headers['x-bytes-used'])#012 File > "/usr/lib/python2.7/dist-packages/swift/common/db.py", line 1431, in > put_container#012raise DatabaseConnectionError(self.db_file, "DB > doesn't exist")#012DatabaseConnectionError: DB connection error > (/srv/node/d3829495-9f10-4075-a558-f99fb665cfe2/accounts/464510/adc/e2cfc6be58be71d5ad6111364fff0adc/e2cfc6be58be71d5ad6111364fff0adc.db, > 0):#012DB doesn't exist > > > The test cluster has two storage nodes with 12 drives each, packaged > Swift 1.8.0 from Ubuntu Cloud Archive and the account/container > configuration is the following: > https://gist.github.com/rubiojr/6ea3d0ea0c4d00949d33 > > The cluster is fully operational and there are no other known issues > ATM. I can easily reproduce the problem by emptying the cluster and > populating it again with swift-dispersion-populate. > > I've exhausted all the possibilities (bug reports, mailing lists, > google, etc) so I decided to ask before start digging the source code > trying to gather evidence to open a bug report. > > If anyone can shed some light on the issue that would be great. > > Thanks! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
