Re: [Openstack] swift proxys and dedecated ceilometer

2013-07-24 Thread Julien Danjou
On Wed, Jul 24 2013, Axel Christiansen wrote: > To be sure, the ceilometer-agent-central and ceilometer-collector do > go on the proxys. And just the ceilometer-api ist for the dedecated > ceilometer-host? No, they all can go to your dedicated host. Ceilometer agent central access Swift via its A

Re: [Openstack] swift proxys and dedecated ceilometer

2013-07-24 Thread Axel Christiansen
Hi Julien, thx a lot for the real quick help. Yes, only swift. These meters: storage.objects storage.objects.size storage.objects.containers storage.objects.incoming.bytes storage.objects.outgoing.bytes To be sure, the ceilometer-agent-central and ceilometer-collector do go on the proxys. And

Re: [Openstack] swift proxys and dedecated ceilometer

2013-07-24 Thread Julien Danjou
On Wed, Jul 24 2013, Axel Christiansen wrote: Hi Axel, If you want to only meter Swift, you just need: > ceilometer-agent-central > ceilometer-collector And to retrieve your data: > ceilometer-api -- Julien Danjou /* Free Software hacker * freelance consultant http://julien.danjou.info */

Re: [Openstack] [Swift] Is it possible to use tempauth and tempurl together?

2013-07-17 Thread Clay Gerrard
Yes tempurl and and tempauth are tessted together, your pipeline looks good. "allow_overrides = false" in the tempauth section could cause this, but it's default is true. Are you creating the tempurl's with "bin/swift-temp-url" ??? clayg@swift:~$ swift post -m temp-url-key:asdf clayg@swift:~$ s

Re: [Openstack] Swift proxy logs

2013-07-16 Thread CHABANI Mohamed El Hadi
Yes, i followed the instructions in the SAIO, and i've either proxy.log, proxy.error or /message !! except the /syslog in which there is logs but no signs for proxy activities (even i do authentication, or upload/download) !! 2013/7/16 Chuck Thier > Hello, > > If you followed the rsyslog instru

Re: [Openstack] Swift proxy logs

2013-07-16 Thread Chuck Thier
Hello, If you followed the rsyslog instructions in the SAIO, then the proxy logs will be in /var/log/swift/proxy.log and proxy.error. If not, then it will be either in /var/log/syslog or /var/log/messages, depending on your server distro. -- Chuck On Tue, Jul 16, 2013 at 4:57 AM, CHABANI Moham

Re: [Openstack] [SWIFT] Account reaping - gradually

2013-07-15 Thread Samuel Merritt
On 7/15/13 6:52 AM, Leandro Reox wrote: Hi guys, I was wondering if there any plans or maybe already exist a feature or a flag for the account reaping process to make it gradually. We had one case with an account with millions of files, that was plain deleted, an that triggers the deletion of ev

Re: [Openstack] [Swift] Try to get the final status of PUT and timeout

2013-07-12 Thread Peter Portante
Can you use the transaction ID in the object server log that is in error to correlate with the proxy server logs to see how the proxy server responded for that REST API request? On Fri, Jul 12, 2013 at 12:46 PM, Samuel Merritt wrote: > On 7/12/13 12:34 AM, Jonathan Lu wrote: > >> Hi, all stacke

Re: [Openstack] [Swift] Try to get the final status of PUT and timeout

2013-07-12 Thread Samuel Merritt
On 7/12/13 12:34 AM, Jonathan Lu wrote: Hi, all stackers, I test the cluster and get such error msg: Jul 12 11:23:57 ubuntu-12 proxy-server ERROR with Object server 192.168.1.161:60 00/xvdb1 re: Trying to get final status of PUT to /v1/AUTH_test/test_8/e9a243cb0

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Axel Christiansen
Hi Hugo, as i mentioned, my ring was wrong. At least i beleve using the wrong ports for the ring services(account, container, object) breaks the ring, right? Thats what broke my setup, i beleve. Axel Am 12.07.13 14:09, schrieb Kuo Hugo: > Hi Alex , > > Did you re-check the drives informati

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Kuo Hugo
Hi Alex , Did you re-check the drives information in the ring? Would you like to show it? +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/7/12 Axel Christiansen > Hello. > > > my issue is solved. What did i do wrong, got wrong from google ;) > > I mixed up the default po

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Axel Christiansen
Hello. my issue is solved. What did i do wrong, got wrong from google ;) I mixed up the default ports for the ring building. The proxy-server option "allow_account_management = true" and "account_autocreate = true" where set to false. All the best. Axel Am 12.07.13 12:50, schrieb Axel Chri

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Axel Christiansen
Thank you. That looks all right. Switching to user swift on a storage node, cd-ing to a mountpoint (/srv/node/sdb1/) and creating a file works. I checked the mount points and rights twice. Here is a little larger snippet from the log server: http://paste.openstack.org/show/40222/ someone with

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Kuo Hugo
Agree with Jonathan +1 Change the owner of disk mount point to the relevant user which you set in /etc/swift/*. +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/7/12 Jonathan Lu > Hi, > I once met such problem because I forget to change the own of the > directory of t

Re: [Openstack] swift storage, getting it working

2013-07-12 Thread Jonathan Lu
Hi, I once met such problem because I forget to change the own of the directory of the mounted device to swift:swift. On 2013/7/12 16:44, Axel Christiansen wrote: Hello List, i got stock getting a swift store running. the base components, a proxy some storage nodes are prepared. The keys

Re: [Openstack] [Swift] Breakpoint resume with tempurl

2013-07-11 Thread John Dickinson
Swift supports Range requests, so you are able to make a GET request with the Range header starting with the first byte you didn't fetch the first time. --John On Jul 10, 2013, at 8:10 PM, Jonathan Lu wrote: > Hi, all stackers, > > I have realized downloading large object with tempurl mi

Re: [Openstack] Swift tempurl

2013-07-10 Thread Morten Møller Riis
This turned out to be a known issue with swath. I upgraded to a newer version and the problem was resolved :) Mvh / Best regards Morten Møller Riis Gigahost ApS m...@gigahost.dk On Jul 11, 2013, at 10:52 AM, Morten Møller Riis wrote: > I use swauth, and thus authtoken is not present in my p

Re: [Openstack] Swift tempurl

2013-07-10 Thread Morten Møller Riis
I use swauth, and thus authtoken is not present in my pipeline. This might be why I'm seeing the problem actually. Mvh / Best regards Morten Møller Riis Gigahost ApS m...@gigahost.dk On Jul 11, 2013, at 4:04 AM, Shrinand Javadekar wrote: > I had a similar problem. See [1]. The fix was to ad

Re: [Openstack] Swift tempurl

2013-07-10 Thread Shrinand Javadekar
I had a similar problem. See [1]. The fix was to add delay_auth_decision = 1 in the proxy-server.conf [filter:authtoken] ... delay_auth_decision = 1 -Shri [1] https://answers.launchpad.net/swift/+question/225614 On Wed, Jul 10, 2013 at 4:43 AM, Morten Møller Riis wrote: > I've been looking at

Re: [Openstack] Swift tempurl

2013-07-10 Thread Morten Møller Riis
I've been looking at tempurl.py and found that the problem occurs here: def _get_key(self, env, account): """ Returns the X-Account-Meta-Temp-URL-Key header value for the account, or None if none is set. :param env: The WSGI environment for the request.

Re: [Openstack] [SWIFT] raising network traffic on the storage node

2013-07-09 Thread Klaus Schürmann
Nachricht- Von: Openstack [mailto:openstack-bounces+klaus.schuermann=mediabeam@lists.launchpad.net] Im Auftrag von Robert van Leeuwen Gesendet: Dienstag, 9. Juli 2013 09:09 An: openstack@lists.launchpad.net Betreff: Re: [Openstack] [SWIFT] raising network traffic on the storage node > If

Re: [Openstack] [SWIFT] raising network traffic on the storage node

2013-07-09 Thread Robert van Leeuwen
> If the replication traffic is responsible for this raising network traffic > for only 1.200.000 objects, how much traffic I can > expect if I have 100.000.000 objects stored? > The average size of my mailobjects are 120 kB. > It's planned to use all 12 hard drive slots of my DELL R720xd with 4

Re: [Openstack] [SWIFT] raising network traffic on the storage node

2013-07-08 Thread Klaus Schürmann
Juli 2013 19:22 An: Klaus Schürmann Cc: openstack@lists.launchpad.net Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node On Mon, 8 Jul 2013 13:50:38 + Klaus Schürmann wrote: > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > st

Re: [Openstack] [SWIFT] raising network traffic o the storage node

2013-07-08 Thread Pete Zaitcev
On Mon, 8 Jul 2013 13:50:38 + Klaus Schürmann wrote: > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluster. > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.

Re: [Openstack] swift

2013-07-08 Thread Pete Zaitcev
On Mon, 8 Jul 2013 23:59:40 +0800 Peter Cheung wrote: > Hi AllHow you guy admin swift? by command line or by www.swiftstack.com > admin portal? Sounds like a great question for the ops list http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators -- P

Re: [Openstack] [SWIFT] raising network traffic o the storage node

2013-07-08 Thread Kuo Hugo
:04 > An: Klaus Schürmann > Cc: openstack@lists.launchpad.net > Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node > > Can you zoom in past the spike, most recent 2 or three weeks and see > how it looks? > > My guess is that the proxy traffic is also risin

Re: [Openstack] [SWIFT] raising network traffic o the storage node

2013-07-08 Thread Klaus Schürmann
Portante [mailto:peter.a.porta...@gmail.com] Gesendet: Montag, 8. Juli 2013 16:04 An: Klaus Schürmann Cc: openstack@lists.launchpad.net Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node Can you zoom in past the spike, most recent 2 or three weeks and see how it looks? My

Re: [Openstack] [SWIFT] raising network traffic o the storage node

2013-07-08 Thread Peter Portante
Can you zoom in past the spike, most recent 2 or three weeks and see how it looks? My guess is that the proxy traffic is also rising. On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann wrote: > Hi, > > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluste

Re: [Openstack] Swift Client

2013-07-05 Thread CHABANI Mohamed El Hadi
Okey thank's, i will check this approach. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp

Re: [Openstack] Swift Client

2013-07-05 Thread Jonathan Lu
Hi, Maybe you need the latest snapshot build, which you can download on the Cyberduck website , "Connect without SSL" section. If that still not work, you can check the config file. If the url of auth is /auth/v1.0, then you sho

Re: [Openstack] Swift Client

2013-07-05 Thread CHABANI Mohamed El Hadi
To use Cyberduck of course. I thing i should also use a different middleware instead of Tempauth, how can i replace it by devauth or swauth ? 2013/7/5 CHABANI Mohamed El Hadi > Hi Hugo, > > I didn't use Owncloud, i tried before Cyberduck and had a problem such > 'unrecongnized SSL message' ! >

Re: [Openstack] Swift Client

2013-07-05 Thread CHABANI Mohamed El Hadi
Hi Hugo, I didn't use Owncloud, i tried before Cyberduck and had a problem such 'unrecongnized SSL message' ! My purpose is just to test ! i'm using a SAIO installation, with an Ubuntu 12.04, with tempauth middleware exactly according to : http://docs.openstack.org/developer/swift/development_sai

Re: [Openstack] Swift Client

2013-07-04 Thread Kuo Hugo
--- I ever used --- 1) CyberDuck 2) Owncloud (What's the problem in your test?) 3) Gladinet client 4) SwiftStack Web Console 5) Most of AWS S3 client tools, but you need to enable Swift3 middleware support 6) OpenStack DashBoard 7) Maldivica gateway -- I never use but should work -- 8) SME Hope

Re: [Openstack] [Swift] Updating best practices for XFS inode size

2013-07-04 Thread Chmouel Boudjnah
On Mon, Jul 1, 2013 at 9:02 AM, Robert van Leeuwen wrote: >> I would like to thank the XFS folks at Redhat for letting us know about the >> improvements in XFS, >> and the XFS team in general for the great work they have done. > Thanks for the heads up. > Do you, or any of the Redhat people, kno

Re: [Openstack] Swift objects permissions

2013-07-01 Thread CHABANI Mohamed El Hadi
Thank's for the replay, in addition to this, i had a problem in the object name...it works now ! 2013/7/1 Hua ZZ Zhang > currently Swift R/W ACL is for container level. make sure that you set ACL > on the container "test". > if you also want to authorize listing that container, add ACL like " >

Re: [Openstack] Swift objects permissions

2013-07-01 Thread Hua ZZ Zhang
currently Swift R/W ACL is for container level. make sure that you set ACL on the container "test". if you also want to authorize listing that container, add ACL like " .r:*,.rlistings". see detailed doc here: http://docs.openstack.org/developer/swift/misc.html#id5 -Edward Zhang

Re: [Openstack] [Swift] Updating best practices for XFS inode size

2013-07-01 Thread Robert van Leeuwen
> If you are using recent Linux Kernels, using the default inode size no longer > has any impact to write (PUT) performance through swift. > > I would like to thank the XFS folks at Redhat for letting us know about the > improvements in XFS, > and the XFS team in general for the great work they

Re: [Openstack] Swift / Keystone authentication

2013-06-27 Thread CHABANI Mohamed El Hadi
ify --ready > > # or a module with notify() method: > > # onready = keystone.common.systemd > > ** ** > > ** ** > > *From:* CHABANI Mohamed El Hadi [mailto:chabani.mohamed.h...@gmail.com] > *Sent:* Wednesday, June 26, 2013 6:25 AM > *To:* Jamie Lennox

Re: [Openstack] Swift / Keystone authentication

2013-06-26 Thread CHABANI Mohamed El Hadi
ing into a problem and cannot start. Check > > the /var/log/syslog file. > > > > > > > > From: CHABANI Mohamed El Hadi > > [mailto:chabani.mohamed.h...@gmail.com] > > Sent: Tuesday, June 25, 2013 8:46 AM > > To:

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Jamie Lennox
gt; Cc: Виль Суркин; openstack@lists.launchpad.net > > > Subject: Re: [Openstack] Swift / Keystone authentication > > > Yes you are right :) ! > > When trying 'service keystone status' i get : keyst

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread tudor
chabani.mohamed.h...@gmail.com [7]] >> SENT: Tuesday, June 25, 2013 8:46 AM >> TO: Miller, Mark M (EB SW Cloud - R&D - Corvallis) >> CC: Виль Суркин; openstack@lists.launchpad.net [8] >> >> SUBJECT: Re: [Openstack] Swift / Keystone authentication >> >

Re: [Openstack] [Swift] Load Average node storage

2013-06-25 Thread Brent Roskos
I wouldn't think 3 replica's across 1 zone is a viable configuration. Given you are starting with 4 servers, you should really have 4 zones. Then all 3 copies of an object can still go somewhere even when aone server is down. When you add more servers, just add them in groups of 4 to keep it sim

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread CHABANI Mohamed El Hadi
c:* Виль Суркин; openstack@lists.launchpad.net > > *Subject:* Re: [Openstack] Swift / Keystone authentication > > ** ** > > Yes you are right :) ! > > When trying *'service keystone status'* i get : *keystone > stop/waiting*..however > when i do *ser

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Miller, Mark M (EB SW Cloud - R&D - Corvallis)
@lists.launchpad.net Subject: Re: [Openstack] Swift / Keystone authentication When i tried with port : 5000 and even 35357 i got : Authorization Failed: [Errno 111] Connection refused These 2 ports are not open i guess, should i open them manually ? (the keystone service is start/running) 2013/6/25

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread CHABANI Mohamed El Hadi
k-bounces+mark.m.miller= > hp@lists.launchpad.net] *On Behalf Of *CHABANI Mohamed El Hadi > *Sent:* Tuesday, June 25, 2013 7:48 AM > *To:* Виль Суркин > > *Cc:* openstack@lists.launchpad.net > *Subject:* Re: [Openstack] Swift / Keystone authentication > > ** ** > &

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Miller, Mark M (EB SW Cloud - R&D - Corvallis)
Subject: Re: [Openstack] Swift / Keystone authentication Remo Mattei : wich log file ? because in keystone.log there is nothing ! 2013/6/25 Remo Mattei mailto:r...@mattei.org>> Hi, can you do a netstat -ant and see what it says? Also I would check the logs, /var/log and look for any erro

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Miller, Mark M (EB SW Cloud - R&D - Corvallis)
Mattei Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Swift / Keystone authentication Remo Mattei : wich log file ? because in keystone.log there is nothing ! 2013/6/25 Remo Mattei mailto:r...@mattei.org>> Hi, can you do a netstat -ant and see what it says? Also I would check th

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread CHABANI Mohamed El Hadi
*Remo Mattei* : wich log file ? because in keystone.log there is nothing ! 2013/6/25 Remo Mattei > Hi, > can you do a netstat -ant and see what it says? Also I would check the > logs, /var/log and look for any errors? If you are using Redhat base > (Fedora, CentOS etc, you may have to open up

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread CHABANI Mohamed El Hadi
*Remo Mattei* : when i cheched with netstat -an : the port 5000 is not open either the port 35357, i don't know why ! and no i'm on Ubuntu 12.04, they are supposed to be open when the keystone run no ?! *Виль Суркин :* No i didn't add any additional configurations, i'm looking now how to add the

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Виль Суркин
Did you create services and endpoints in keystone? If yes, how you can do this? =) -- Виль Суркин вторник, 25 июня 2013 г. в 18:48, CHABANI Mohamed El Hadi написал: > When i tried with port : 5000 and even 35357 i got : Authorization Failed: > [Errno 111] Connection refused > > These 2 po

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Remo Mattei
Hi, can you do a netstat -ant and see what it says? Also I would check the logs, /var/log and look for any errors? If you are using Redhat base (Fedora, CentOS etc, you may have to open up the iptables since by default it does not have those ports open). My 2 cents. Remo On Jun 25, 2013, a

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread CHABANI Mohamed El Hadi
When i tried with port : 5000 and even 35357 i got : *Authorization Failed: [Errno 111] Connection refused* These 2 ports are not open i guess, should i open them manually ? (the keystone service is start/running) 2013/6/25 Виль Суркин > You should authenticate in keystone service, not in swi

Re: [Openstack] Swift / Keystone authentication

2013-06-25 Thread Виль Суркин
You should authenticate in keystone service, not in swift-proxy. Keystone's Public API usually binds to TCP port 5000. -- Виль Суркин вторник, 25 июня 2013 г. в 18:38, CHABANI Mohamed El Hadi написал: > Hi, > > I'm trying to install Swift from this Guide > (http://docs.openstack.org/trun

Re: [Openstack] [Swift] Load Average node storage

2013-06-23 Thread Robert van Leeuwen
> load average: 24.06, 24.02, 24.00 > There is something, can i do to reduce the load? > is it normal? I guess it depends on what is causing the load. (are there processes eating all cpu cycles?) What we see is that the object-auditor and object-replicator can cause quite a bit of load because

Re: [Openstack] Swift cleaning tenant after deletion on Keystone

2013-06-20 Thread Brent Roskos
The account-reaper generally takes care of this. Have you seen this? http://docs.openstack.org/developer/swift/overview_reaper.html Regards, Brent On Thu, Jun 20, 2013 at 6:06 AM, Heiko Krämer wrote: > Hey Hugo, > > ok, thx for your quick answer! > > Greetings > Heiko > > > On 20.06.2013 1

Re: [Openstack] Swift cleaning tenant after deletion on Keystone

2013-06-20 Thread Heiko Krämer
Hey Hugo, ok, thx for your quick answer! Greetings Heiko On 20.06.2013 11:56, Kuo Hugo wrote: > Hi Heiko, > > All objects won't be deleted if the tenant been deleted in Keystone. > > Hugo > > +Hugo Kuo+ > h...@swiftstack.com > tonyt...@gmail.com >

Re: [Openstack] Swift cleaning tenant after deletion on Keystone

2013-06-20 Thread Kuo Hugo
Hi Heiko, All objects won't be deleted if the tenant been deleted in Keystone. Hugo +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/6/20 Heiko Krämer > Heyho guys, > > I've a short question because I can't find anything in the docs. > > Will Swift cleanup himself if I d

Re: [Openstack] Swift Object Storage authentication

2013-06-19 Thread CHABANI Mohamed El Hadi
During the installation of Swift, i didn't touch any Keystone configuration, so do you think that i should install Keystone separately according to : http://docs.openstack.org/grizzly/openstack-compute/install/apt/content/install-keystone.html? and configure Swift to work with it ? 2013/6/19 Kuo

Re: [Openstack] Swift Object Storage authentication

2013-06-19 Thread CHABANI Mohamed El Hadi
Hi Hugo, Here is my proxy-sever.conf http://paste.openstack.org/show/38944/ (i've already attached a pic in the first mail :) ) Thanks 2013/6/19 Kuo Hugo > Hi CHABANI , > > Would you please show me the proxy-server.conf ? > > You can paste on http://paste.openstack.org/ or gist. > > Cheer

Re: [Openstack] Swift Object Storage authentication

2013-06-19 Thread Kuo Hugo
Hi CHABANI , Would you please show me the proxy-server.conf ? You can paste on http://paste.openstack.org/ or gist. Cheers +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/6/19 CHABANI Mohamed El Hadi > Hi all, > > I'm trying to install Swift Object Storage accordin

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-18 Thread ZHOU Yuan
Jonathan, Yes we have 10,000 containers and we're using COSBench to do the tests. Sincerely, Yuan On Wed, Jun 19, 2013 at 9:24 AM, Jonathan Lu wrote: > Hi, Zhou, > BTW, in test case 2, the number of container is 10,000 or just 10? > > > Jonathan Lu > > On 2013/6/18 19:18, ZHOU Yuan wrot

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-18 Thread ZHOU Yuan
Jonathan, we happen to use SN similar with yours and I could share you some performance testing data here: 1. 100 container with 1 objects(base) The performance is quite good and can hit HW bottleneck 2. 10kcontainer with 100M objects The performance is not so good, which dropped 80% compared

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Jonathan Lu
Hi, Huang Thanks a lot. I will try this test. One more question: In the 3 following situation, will the base line performance be quite different? 1. only 1 sontaienr with 10m objects; 2. 100,000 objects per container at 100 containers 3. 1,000 objects per con

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Huang Zhiteng
On Tue, Jun 18, 2013 at 12:35 PM, Jonathan Lu wrote: > Hi, Huang > Thanks for you explanation. Does it mean that the storage cluster of > specific processing ability will be slower and slower with more and more > objects? Is there any test about the rate of the decline or is there any > lowe

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Jonathan Lu
Hi, Huang Thanks for you explanation. Does it mean that the storage cluster of specific processing ability will be slower and slower with more and more objects? Is there any test about the rate of the decline or is there any lower limit? For example, my environment is: Swift

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Kuo Hugo
Hi Huang, Storage nodes will run out of memory for caching inode eventually right? Have you ever measure the upper limit of caching capability of your storage nodes? Hi Jonathan, The default reclaiming time is 7 days. Did you wait for 7 days or just change the setting to 1 min in conf file ?

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Jonathan Lu
Hi Hugo, I know the tombstone mechanism. In my opinion after the reclaim time, the object of xxx.tombstone will be deleted at all. Is that right? Maybe I misunderstand the doc :( ... We try to "colddown" the swift system ( just wait for the reclaiming) and test, but the result is not sa

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Kuo Hugo
Hi Jonathan , How did you perform "delete all the objects in the storage" ? Those deleted objects still consume inodes in tombstone status until the reclaim time. Would you mind to compare the result of $> sudo cat /proc/slabinfo | grep xfs , before/after set the vfs_cache_pressure spongebob@

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Huang Zhiteng
On Tue, Jun 18, 2013 at 10:42 AM, Jonathan Lu wrote: > On 2013/6/17 18:59, Robert van Leeuwen wrote: > >> I'm facing the issue about the performance degradation, and once I >>> glanced that changing the value in /proc/sys >>> /vm/vfs_cache_pressure will do a favour. >>> Can anyone explain to me w

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Jonathan Lu
On 2013/6/17 18:59, Robert van Leeuwen wrote: I'm facing the issue about the performance degradation, and once I glanced that changing the value in /proc/sys /vm/vfs_cache_pressure will do a favour. Can anyone explain to me whether and why it is useful? Hi, When this is set to a lower value th

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Robert van Leeuwen
> I'm facing the issue about the performance degradation, and once I glanced > that changing the value in /proc/sys > /vm/vfs_cache_pressure will do a favour. > Can anyone explain to me whether and why it is useful? Hi, When this is set to a lower value the kernel will try to keep the inode/den

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Peter Portante
Can you say more about your environment and setup, and then about the measurements you are taking to categorize the performance issue? On Mon, Jun 17, 2013 at 3:25 AM, Jonathan Lu wrote: > Hi, all, > I'm facing the issue about the performance degradation, and once I > glanced that changing th

Re: [Openstack] [Swift] Cache pressure tuning

2013-06-17 Thread Hugo
Check the slab usage of xfs_* might help . $ sudo slabtop Hope it help 從我的 iPhone 傳送 Jonathan Lu 於 2013/6/17 下午3:25 寫道: > Hi, all, > I'm facing the issue about the performance degradation, and once I > glanced that changing the value in /proc/sys/vm/vfs_cache_pressure will do a > favou

Re: [Openstack] [Swift] Upgrade Swift from ver1.4 to ver1.8

2013-06-10 Thread John Dickinson
This is quite possible to do, and in fact one thing that we keep in mind with every Swift release: deployers must be able to upgrade without any lapse in availability. The basic steps are: - stop background processes - upgrade packages (system and/or swift) - restart processes - start background

Re: [Openstack] [Swift] Swift load balancing

2013-06-07 Thread John Dickinson
The given options (DNS, SW load balancer, and HW load balancer) are all things I've seen people use in production Swift clusters. As mentioned in another reply, DNSRR isn't really load balancing, but it can be used if nothing else is available. One thing to consider when choosing a load balance

Re: [Openstack] [Swift] Swift load balancing

2013-06-07 Thread Heiko Krämer
Hey Kotwani, we are using an SW loadbalancer but L3 (keepalived). DNS round robin are not a load balancer :) if one node is done, some connections will arrive the down host that's not the right way i think. HTTP Proxy are an option but you make a bottleneck of your connection to WAN because all u

Re: [Openstack] [Swift] Swift load balancing

2013-06-06 Thread Chu Duc Minh
If you choose to use DNS round robin, you can set TTL small and use a script/tool to continous check proxy nodes to reconfigure DNS record if one proxy node goes down, and vice-versa. If you choose to use SW load-balancer, I suggest HAProxy for performance (many high-traffic websites use it) and N

Re: [Openstack] Swift performance issues with requests

2013-06-05 Thread Jonathan Lu
Hi, Hugo, and all list, I'm also facing the issue about the performance degradation of swift that Hugo had once mentioned in his Message #14935. I'm in tesing with 50KB-size_object putting, and after several turn of test the throughout dropped dramatically. Have you found the sol

Re: [Openstack] [Swift] Intermittent error 403 "Access was denied to this resource"

2013-06-04 Thread Chmouel Boudjnah
On Tue, Jun 4, 2013 at 2:41 PM, Andrii Loshkovskyi wrote: > Thank you for answer. > > Chmouel, do you mean the auth_token on Keystone or swift proxy server? > > from /etc/keystone/keystone.conf > > [filter:token_auth] > paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory > > fro

Re: [Openstack] Swift performance issues with requests

2013-06-04 Thread Rick Jones
openstack@lists.launchpad.net Betreff: Re: [Openstack] Swift performance issues with requests On 05/31/2013 04:55 AM, Klaus Schürmann wrote: May 31 10:33:08 swift-proxy1 proxy-logging 10.4.2.99 10.4.2.99 31/May/2013/08/33/08 GET /v1/AUTH_provider1/129450/829188397.31 HTTP/1.0 200 - Wget/1.12%20%28li

Re: [Openstack] [Swift] Intermittent error 403 "Access was denied to this resource"

2013-06-04 Thread Andrii Loshkovskyi
Thank you for answer. Chmouel, do you mean the auth_token on Keystone or swift proxy server? from /etc/keystone/keystone.conf [filter:token_auth] paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory from /etc/swift/proxy-server.conf [filter:authtoken] paste.filter_factory = k

Re: [Openstack] [Swift] Intermittent error 403 "Access was denied to this resource"

2013-06-04 Thread Chmouel Boudjnah
I have seen this when keystone is too busy for validating tokens. getting keystone behind apache or scaling up keystone make things a better (and make sure you are using swift memcache connection in auth_token). Chmouel. On Mon, Jun 3, 2013 at 8:15 PM, Andrii Loshkovskyi wrote: > Hello, > > I w

Re: [Openstack] Swift performance issues with requests

2013-06-04 Thread Klaus Schürmann
, 31. Mai 2013 19:17 An: Klaus Schürmann Cc: openstack@lists.launchpad.net Betreff: Re: [Openstack] Swift performance issues with requests On 05/31/2013 04:55 AM, Klaus Schürmann wrote: > May 31 10:33:08 swift-proxy1 proxy-logging 10.4.2.99 10.4.2.99 > 31/May/2013/08/33/08 GET /v1/AUTH_pro

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Hua ZZ Zhang
"Openstack" , m@lists.launchpad Subject .net> Re: [Openstack] [Swift] Storage Server Redirection

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Hua ZZ Zhang
lists.launchpad.net" , Openstack Subject RE: [Openstack] [Swift] Storage

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Luse, Paul E
E Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Swift] Storage Server Redirection On Mon, Jun 3, 2013 at 8:41 AM, Luse, Paul E wrote: > Hi Peter, > > I'd ask that Edward may want to chime in on object variation as I may not > understand that one correctly - again I&#x

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Peter Portante
y server, specifically in a form of the object server controllers would be a good place for this kind of thing. > > Thanks > Paul > > > -Original Message- > From: Peter Portante [mailto:peter.a.porta...@gmail.com] > Sent: Monday, June 03, 2013 3:50 AM > To: Luse

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Luse, Paul E
...@gmail.com] Sent: Monday, June 03, 2013 3:50 AM To: Luse, Paul E Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Swift] Storage Server Redirection Hi Paul, Can you explain more about the two use cases referenced? Object Variation seems like a nice idea, but can you clarify how the m

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Luse, Paul E
Thanks for the response John, see below... Thx Paul -Original Message- From: John Dickinson [mailto:m...@not.mn] Sent: Sunday, June 02, 2013 11:29 PM To: Luse, Paul E Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Swift] Storage Server Redirection On May 31, 2013, at 4:53

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Luse, Paul E
nt: Monday, June 03, 2013 3:41 AM To: Hua ZZ Zhang Cc: Luse, Paul E; Openstack; openstack@lists.launchpad.net Subject: Re: [Openstack] [Swift] Storage Server Redirection Hi Paul, I would like to echo Hua's and John's concern about 5xx code uses. Using 3xx codes should be fine. Can y

Re: [Openstack] Swift performance issues with requests

2013-06-03 Thread Kuo Hugo
Hi Klaus, How's the disk space usage now? Cheers +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/5/31 Klaus Schürmann > Hi, > > when I test my new swift cluster I get a strange behavior with GET and PUT > requests. > Most time it is really fast. But sometimes it takes a

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Peter Portante
Hi Paul, Can you explain more about the two use cases referenced? Object Variation seems like a nice idea, but can you clarify how the mechanism would work to get at the variations? Any examples? From the current descriptions, it seems like the LFS Patch could help here without adding anything to

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Peter Portante
Hi Paul, I would like to echo Hua's and John's concern about 5xx code uses. Using 3xx codes should be fine. Can you give a sufficient argument to consider 5xx? Thanks, -peter On Mon, Jun 3, 2013 at 5:49 AM, Hua ZZ Zhang wrote: >>> 2) The basic idea is that an object server (via middleware or o

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-03 Thread Hua ZZ Zhang
>> 2) The basic idea is that an object server (via middleware or otherwise) will be given the ability to respond to a request to indicate ‘not me but I know who should handle this’. I’m thinking this makes more sense as a 5xx response with additional information (partition, nodes) about the route

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-02 Thread John Dickinson
On May 31, 2013, at 4:53 PM, "Luse, Paul E" wrote: > I’m looking at tacking this item: > > https://blueprints.launchpad.net/swift/+spec/support-storage-server-redirects > > and wanted to get some feedback on the following observations/thoughts: > > 1) This is a capability that would be che

Re: [Openstack] Swift performance issues with requests

2013-05-31 Thread Rick Jones
On 05/31/2013 04:55 AM, Klaus Schürmann wrote: May 31 10:33:08 swift-proxy1 proxy-logging 10.4.2.99 10.4.2.99 31/May/2013/08/33/08 GET /v1/AUTH_provider1/129450/829188397.31 HTTP/1.0 200 - Wget/1.12%20%28linux-gnu%29 provider1%2CAUTH_tke6408efec4b2439091fb6f4e75911602 - 283354 - txd4a3a4bf3f38

Re: [Openstack] Swift performance issues with requests

2013-05-31 Thread Rick Jones
On 05/31/2013 04:55 AM, Klaus Schürmann wrote: Hi, when I test my new swift cluster I get a strange behavior with GET and PUT requests. Most time it is really fast. But sometimes it takes a long time to get the data. Here is an example with the same request which took one time 17 seconds: ...

Re: [Openstack] Swift questions

2013-05-22 Thread Chuck Thier
On Wed, May 22, 2013 at 7:54 PM, Mark Brown wrote: > Thanks Chuck. > > Just one more question about rebalancing. Have there been measurements on > how much it affects performance when a rebalance is in progress? I would > assume its an operation that puts some load on the system, while also > keep

Re: [Openstack] Swift questions

2013-05-22 Thread Mark Brown
From: Chuck Thier To: Mark Brown Cc: Openstack Sent: Wednesday, May 22, 2013 7:26 AM Subject: Re: [Openstack] Swift questions Hey Mark, On Wed, May 22, 2013 at 8:59 AM, Mark Brown wrote: Thank you for the responses Chuck. > > >As part of a rebalance, the replicator, I would assum

Re: [Openstack] Swift questions

2013-05-22 Thread Chuck Thier
Hey Mark, On Wed, May 22, 2013 at 8:59 AM, Mark Brown wrote: > Thank you for the responses Chuck. > > As part of a rebalance, the replicator, I would assume, copies the object > from the old partition to the new partition, and then deletes it from the > old partition. Is that a fair assumption? >

  1   2   3   4   5   6   7   >