Re: [Openstack] Help - swift with low IOPS

2013-09-04 Thread Zhou, Yuan
j Cc: openstack@lists.openstack.org Subject: Re: [Openstack] Help - swift with low IOPS Hi, If all your requests are PUT, then the expected network bandwidth should be: 3000*500/1024*8 = 11718.75Mbps , which means more than 10Gbps According to your environment, there are 2

Re: [Openstack] Help - swift with low IOPS

2013-09-04 Thread Jonathan Lu
Hi, If all your requests are PUT, then the expected network bandwidth should be: 3000*500/1024*8 = 11718.75Mbps , which means more than 10Gbps According to your environment, there are 2 proxies ( giga network?) and that is far away from the expectation. For example, in our

Re: [Openstack] Help - swift with low IOPS

2013-09-04 Thread pangj
Hi, How does a setup make swift have the capability of 3000 TPS, for the average size 500KB of files? What architecture, hardware, network etc will be expected? Thanks for any suggestion. On 2013-9-3 17:11, pangj wrote: Thanks for the info. I have checked this one days ago. Given the case,

Re: [Openstack] Help - swift with low IOPS

2013-09-03 Thread Chuck Thier
Hi Pangj, First, make sure you have an updated version of swift-bench. There was a bug where it was relying on python-swiftclient to setup eventlet, but when eventlet was removed from swiftclient, that caused swift-bench to not run requests concurrently. There are quite a few things that should

Re: [Openstack] Help - swift with low IOPS

2013-09-03 Thread Robert van Leeuwen
>> http://engineering.spilgames.com/openstack-swift-lots-small-files/ > > there are many containers (more than 100 millions), but objects under > each container are few (less than 20). There is only one account for > TempAuth. Hi, For each countainer there is a sql-lite database. Creating 100 mil

Re: [Openstack] Help - swift with low IOPS

2013-09-03 Thread pangj
On 2013-9-3 14:10, Jonathan Lu wrote: Robert van Leeuwen once post a report of their experience in the mail list and I think it's such a good sharing. http://engineering.spilgames.com/openstack-swift-lots-small-files/ Hi, Thanks for the info. I have checked this one days ago. Given the ca

Re: [Openstack] Help - swift with low IOPS

2013-09-03 Thread Kuo Hugo
Using RamDisk to simulate spindles +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/9/3 Jonathan Lu > Hi, pangj > While putting small objects, the bottleneck will not be the network > bandwidth, and the server's cpu/mem will be low. It will be a matter of > inode and c

Re: [Openstack] Help - swift with low IOPS

2013-09-02 Thread Jonathan Lu
Hi, pangj While putting small objects, the bottleneck will not be the network bandwidth, and the server's cpu/mem will be low. It will be a matter of inode and cache. You can check the iowait of your storage node server and it may be a "filesystem" problem. As a estimate of the performance

Re: [Openstack] Help - swift with low IOPS

2013-09-02 Thread pangj
For each test file the size is 4KB. Network connection is 1Gb. But the bandwidth never got full, it's about only 70 - 80 Mb at peak. And all servers' cpu/mem usage are low. Thanks. On 2013-9-3 13:32, Jonathan Lu wrote: Hi, What is the average size of the objects? And your network condition?

Re: [Openstack] Help - swift with low IOPS

2013-09-02 Thread Jonathan Lu
Hi, What is the average size of the objects? And your network condition? Maybe the bottleneck of performance is not Swift, but the network or other things. Best Regards Jonathan Lu On 2013/9/2 19:51, pangj wrote: Hi, We have benched the swift, the cluster is with one proxy server, one