Re: [Openstack] Improved browser-based access to Swift

2013-07-18 Thread John Dickinson
Those both sound pretty cool. If you submit a patch (via gerrit) to the https://github.com/openstack/swift/blob/master/doc/source/associated_projects.rst document, then you can have these projects listed on http://docs.openstack.org/developer/swift/associated_projects.html --John On Jul 18,

Re: [Openstack] [Swift] Breakpoint resume with tempurl

2013-07-11 Thread John Dickinson
Swift supports Range requests, so you are able to make a GET request with the Range header starting with the first byte you didn't fetch the first time. --John On Jul 10, 2013, at 8:10 PM, Jonathan Lu wrote: > Hi, all stackers, > > I have realized downloading large object with tempurl mi

Re: [Openstack] Debug level / Swift

2013-07-03 Thread John Dickinson
You can set log_level = DEBUG in the [DEFAULT] section of your config files. https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L19 --John On Jul 3, 2013, at 7:37 AM, CHABANI Mohamed El Hadi wrote: > Hi guys, > > I want to active the Debug level on Swift because

Re: [Openstack] Availability of metrics from SWIFT - Object Storage

2013-06-27 Thread John Dickinson
Swift itself also reports well over 130 unique metrics about itself via StatsD, including the bandwidth for each client request (like ceilometer does). You can monitor these with any standard statsd listener, and monitoring this data doesn't require integration with any other openstack project (

Re: [Openstack] [OpenStack] SWIFT Object Store spanning multiple data centers

2013-06-16 Thread John Dickinson
The global clusters feature is Swift is very new and just now being finished up. We are finishing up the last part of it and will have it completed in our next release (tentatively scheduled for June 27). The last part is the affinity write (ie don't write to a WAN region). The regions concept

Re: [Openstack] Load Balancers for Swift with nginx and pound

2013-06-14 Thread John Dickinson
Also, just as general info, don't use nginx in front of Swift. nginx buffers request bodies, and that can become very problematic when uploading content into the Swift cluster (especially since the body could be up to 5GB--not too many of those requests and you'll overload your nginx box). Seco

Re: [Openstack] Folsom Keystone and Grizzly Swift

2013-06-11 Thread John Dickinson
gt; be compatible with folsom and have the fix you need for caching (the > cache=swift.cache parameter to have in auth_token). You will need a > recent version of swift tho to get everything working well. > > On Tue, Jun 11, 2013 at 4:39 PM, John Dickinson wrote: >> Yes it will

Re: [Openstack] Folsom Keystone and Grizzly Swift

2013-06-11 Thread John Dickinson
Yes it will "work", but I would not recommend it for production systems. There were some changes in Keystone during the Grizzly release that are pretty much required for Keystone to be usable with Swift (specifically around Keystone's ability to support token caching). My recommendation would b

Re: [Openstack] curl with swift

2013-06-10 Thread John Dickinson
Sure, easy to do (curl is what I normally use anyway). To auth (for auth v1. v2 and keystone will be different): curl -i -H "X-Auth-User: foo" -H "X-Auth-Key: bar" http://swift/auth/v1.0 The 2 headers you need to look for are X-Storage-URL and X-Auth-Token. After that, use the X-Auth-Token to t

Re: [Openstack] [Swift] Upgrade Swift from ver1.4 to ver1.8

2013-06-10 Thread John Dickinson
This is quite possible to do, and in fact one thing that we keep in mind with every Swift release: deployers must be able to upgrade without any lapse in availability. The basic steps are: - stop background processes - upgrade packages (system and/or swift) - restart processes - start background

Re: [Openstack] [Swift] Swift load balancing

2013-06-07 Thread John Dickinson
The given options (DNS, SW load balancer, and HW load balancer) are all things I've seen people use in production Swift clusters. As mentioned in another reply, DNSRR isn't really load balancing, but it can be used if nothing else is available. One thing to consider when choosing a load balance

Re: [Openstack] [Swift] Storage Server Redirection

2013-06-02 Thread John Dickinson
On May 31, 2013, at 4:53 PM, "Luse, Paul E" wrote: > I’m looking at tacking this item: > > https://blueprints.launchpad.net/swift/+spec/support-storage-server-redirects > > and wanted to get some feedback on the following observations/thoughts: > > 1) This is a capability that would be che

Re: [Openstack] Understanding Failure Recovery in Swift

2013-05-09 Thread John Dickinson
Replication will be logged and you can look for a "Object replication complete" log message. For a more detailed look at how swift handles failures, you can watch this video: http://mirror.linux.org.au/linux.conf.au/2013/mp4/Playing_with_OpenStack_Swift.mp4 (I need to post my blog post that ha

Re: [Openstack] Configuring tempurl in Openstack Swift

2013-04-29 Thread John Dickinson
There are no plans to change the current implementation of tempurl. In order to generate a tempurl that can be validated without relying on a network call to a centralized authority service, the owner and the service must have a shared secret. In the tempurl feature, this shared secret is set as

Re: [Openstack] Short introduction to running swift with quotas

2013-04-19 Thread John Dickinson
Looks great! Thanks. --John On Apr 19, 2013, at 8:16 AM, Heiko Krämer wrote: > Hi Guys, > > I've written a short guide to enable the quotas in Swift (1.8.0). > > http://honeybutcher.de/2013/04/account-quotas-swift-1-8-0/ > > > I hope it's helpfully. > > Greetings > Heiko > >

Re: [Openstack] [DevStack] Does a Swift/Keystone only install require AMQP?

2013-02-19 Thread John Dickinson
nothing in swift requires rabbit, qpid, or zeromq --john On Feb 19, 2013, at 4:53 PM, Everett Toews wrote: > Hi All, > > When I was doing a Swift/Keystone only install with DevStack I used the > following in my localrc > > disable_all_services > enable_service key swift mysql > > Then

Re: [Openstack] [SWIFT] code reading

2013-02-13 Thread John Dickinson
This part of the code gets the account info, but if the account isn't found and account autocreation is set (eg for tempauth and keystone), then we need to create the account first before returning the info. However, since there are multiple replicas of the account in the cluster, we can't simp

Re: [Openstack] [swift]

2013-02-11 Thread John Dickinson
This is the result of `python ./setup.py develop`. That command sets up the scripts to reference your local source so that active dev work is immediately reflected. `python ./setup.py install` would actually copy the scripts to the "right" locations. --John On Feb 11, 2013, at 9:02 AM, Kun

[Openstack] [Swift] Swift 1.7.6 and python-swiftclient 1.3.0 released

2013-01-24 Thread John Dickinson
I'm happy to announce that Swift 1.7.6 has been released. This release is the work of twenty five contributors. As always, you can upgrade to this release with no downtime for your clients. You can find download links at https://launchpad.net/swift and the Launchpad release tracking at https://lau

[Openstack] [Swift] Candidate Swift 1.7.6 cut

2013-01-18 Thread John Dickinson
We've cut he milestone-proposed branch for Swift 1.7.6. It's scheduled to be released next Thursday January 24. Please take a look and let us know of any issues you find ASAP. Candidate tarball: http://tarballs.openstack.org/swift/swift-milestone-proposed.tar.gz The full proposed changelog for

Re: [Openstack] [SWIFT] Change the partition power to recreate the RING

2013-01-14 Thread John Dickinson
ies & Standards > Building 501/B205 > liton...@us.ibm.com > > John Dickinson ---01/11/2013 04:28:47 PM---If effect, this would > be a complete replacement of your rings, and that is essentially a whole new c > > From: John Dickinson > To: Alejandro Comisario , &g

Re: [Openstack] [SWIFT] Change the partition power to recreate the RING

2013-01-11 Thread John Dickinson
If effect, this would be a complete replacement of your rings, and that is essentially a whole new cluster. All of the existing data would need to be rehashed into the new ring before it is available. There is no process that rehashes the data to ensure that it is still in the correct partition

[Openstack] [Swift] new regular meeting for Swift

2013-01-08 Thread John Dickinson
I've scheduled a bi-weekly Swift team meeting for Swift contributors. Starting tomorrow, we will meet in #openstack-meeting every other Wednesday at 11am Pacific, 1pm Central, 1900 UTC. For tomorrow's meeting, we will be discussing Swift's next release and the current outstanding patches (http

Re: [Openstack] Swift issues with Swift3

2013-01-08 Thread John Dickinson
The "use" line in the swift3 section is wrong. Your config tells the proxy to load that module from the swift project (ie from swift import swift3). However, that won't work because swift3 isn't shipped with swift. Instead you need to have something like "use = egg:swift3#middleware". For more

Re: [Openstack] swift -- object layout on storage

2013-01-04 Thread John Dickinson
It's pretty simple. Swift uses the underlying filesystem to store the data on disk, and so you can use normal FS tools to find and inspect your data. For the object server, the "magic" happens here: https://github.com/openstack/swift/blob/master/swift/obj/server.py#L117 The end result is that t

Re: [Openstack] Error while enabling s3 support in swift

2012-11-28 Thread John Dickinson
> "/usr/local/lib/python2.7/dist-packages/swift3-1.0.0-py2.7.egg/swift3/middleware.py", > line 67, in > > from swift.common.swob import Request, Response > ImportError: No module named swob > > > I couldnt find a way to install this swob module. > Any

Re: [Openstack] Error while enabling s3 support in swift

2012-11-19 Thread John Dickinson
check out the README at https://github.com/fujita/swift3 for the correct proxy server config section. --john On Nov 19, 2012, at 11:22 PM, Shashank Sahni wrote: > Hi, > > I'm trying to install swift 1.7.4 on Ubuntu 12.04. In order to enable the s3 > support, I added the appropriate paramete

Re: [Openstack] PoC Swift

2012-11-19 Thread John Dickinson
You can run your swift cluster with 2 replicas. This is set when the ring is first created. You can use `swift-ring-builder` (ie with no arguments) to get usage help. Note that there is not (yet) an easy way to change your replica count once the ring has been created. If you have more than one

Re: [Openstack] [Swift] account-level and container-level usage information

2012-11-13 Thread John Dickinson
You could use a project like slogging (http://github.com/notmyname/slogging) to run inside the cluster and aggregate that information from account dbs and logs. --John On Nov 13, 2012, at 12:15 PM, Pete Zaitcev wrote: > On Mon, 12 Nov 2012 20:48:35 -0800 > Ning Zhang wrote: > >> Is there an

Re: [Openstack] Running other services on swift nodes

2012-11-04 Thread John Dickinson
From the Swift perspective, there isn't any reason other services can't be run on the Swift boxes. I'd check for a few things, though. 1) Make sure dependencies aren't in conflict. Thanks to the work of the CI team, this should be mostly sane. 2) Obviously, monitor your systems and don't overlo

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
> Have you thought anything like it? > If not, im wondering how, for example rackspace handles this kind of issues ( > ignoring all the CDN thing ) > > Cheers. > > --- > Alejandrito > > On Thu, Nov 1, 2012 at 12:55 PM, John Dickinson wrote: > Thi

Re: [Openstack] [SWIFT] Rack-awareness

2012-11-01 Thread John Dickinson
This is already supported in Swift with the concept of availability zones. Swift will place each replica in different availability zones, if possible. If you only have one zone, Swift will place the replicas on different machines. If you only have one machine, Swift will place the replicas on di

Re: [Openstack] Nova middleware for enabling CORS?

2012-10-30 Thread John Dickinson
Since the CORS support in Swift allows the preflight OPTIONS response to be different on a per-container basis (which is correct in a multi-tenant system), the CORS support was added directly into Swift's proxy server rather than as middleware. In order to fulfill the OPTIONS request, container

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-29 Thread John Dickinson
I can write files to the mounted filesystem directly before, > during, and after the Swift error occurs. So the problem must be some kind > of timeout that is causing the object server to think that something is wrong > with the disk. > > I'll keep digging... >

Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers

2012-10-26 Thread John Dickinson
A 507 is returned by the object servers in 2 situations: 1) the drives are full or 2) the drives have been unmounted because of disk error. It's highly likely that you simply have full drives. Remember that the usable space in your cluster is 1/N where N = replica count. As an example, with 3 r

[Openstack] Swift 1.7.5 release plan

2012-10-25 Thread John Dickinson
Fast on the heels of a productive summit in San Diego, we are getting ready to release Swift 1.7.5. Our current schedule is to cut the QA release on November 5 and, assuming it passes all QA tests, prepare the final release on November 8. This is quite a solid release with a ton of bug fixes and

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
arify, because i really want to understand what my real metrics are > so i can know where to tune in case i need to. > Thanks ! > > --- > Alejandrito > > > On Wed, Oct 24, 2012 at 3:28 PM, John Dickinson wrote: > Sorry for the delay. You've got an interesting pr

Re: [Openstack] [SWIFT] Proxies Sizing for 90.000 / 200.000 RPM

2012-10-24 Thread John Dickinson
Sorry for the delay. You've got an interesting problem, and we were all quite busy last week with the summit. First, the standard caveat: Your performance is going to be highly dependent on your particular workload and your particular hardware deployment. 3500 req/sec in two different deploymen

Re: [Openstack] [openstack-dev] [Swift] community meeting Oct 1

2012-10-01 Thread John Dickinson
Reminder for today's meeting. On Sep 24, 2012, at 10:59 PM, John Dickinson wrote: > As we finish up Folsom and head into the Grizzly summit, I'd like to have a > Swift community meeting. > > Who: The Swift community (users, deployers, contributors, core devs) > When

Re: [Openstack] Swift reliability

2012-09-26 Thread John Dickinson
The 404s on object PUTs are probably related to the timeout errors you are seeing on the container servers. This may be because of IO contention on your hardware (eg overtaxed drives). How does the disk IO look on your physical hardware? The disk full errors may be because you are running out o

[Openstack] [Swift] community meeting Oct 1

2012-09-24 Thread John Dickinson
As we finish up Folsom and head into the Grizzly summit, I'd like to have a Swift community meeting. Who: The Swift community (users, deployers, contributors, core devs) When: October 1, 2012 at 8pm (UTC), 3pm (Central), 1pm (Pacific) Where: #openstack-meeting on freenode (IRC) Agenda: http://w

Re: [Openstack] [swift] make swift.common.utils.streq_const_time more efficient

2012-09-13 Thread John Dickinson
The intended purpose of this string comparison is to explicitly compare every character. Doing it this way guards against timing attacks (http://en.wikipedia.org/wiki/Timing_attack). --John On Sep 13, 2012, at 12:06 AM, Mike Green wrote: > def streq_const_time(s1, s2): > > if len(s1) !=

[Openstack] [swift] 1.7.0-final pushed

2012-09-12 Thread John Dickinson
e the Essex release, but I wanted to publicly thank everyone who contributed to Swift during the Folsom release cycle. The following people have code contributions in Swift during the Folsom release cycle. Thanks again! Greg Holt John Dickinson Darrell Bishop Samuel Merritt Florian Hines David

Re: [Openstack] [OpenStack][Swift][Replicator] Does object replicator push "exist" object to handoff node while a node/disk/network fails ?

2012-09-06 Thread John Dickinson
you can force a replicator to push to a handoff node by unmounting the drive one of the primary replicas is on. --John On Sep 6, 2012, at 9:00 AM, Kuo Hugo wrote: > Hi folks , John and Chmouel , > > I did post a question about this long time ago. And my test result is match > to Chmouel's a

[Openstack] Swift PTL candidacy

2012-09-04 Thread John Dickinson
I am running to continue my position as Swift PTL. I have been involved with swift since the project started. I am an active contributor, reviewer, and community participant. I have lead meetups about swift, given conference presentations on swift, and am active in IRC helping those who have qu

[Openstack] [swift] important upgrade note for swift deployers

2012-08-28 Thread John Dickinson
Swift 1.7 = The next release of Swift will be version 1.7. This will be our release for OpenStack Folsom, and is scheduled land mid-September. There is an important change for deployers in this release. This email has the details so you can begin planning your upgrade path. What's the cha

Re: [Openstack] Question about undo delete in Swift with Object Version

2012-08-22 Thread John Dickinson
With the current implementation of versioning in Swift, this isn't possible. It's better to think of the feature as "versioned writes". --John On Aug 22, 2012, at 12:13 AM, ZHOU Yuan wrote: > Hi stackers, > > I'm trying to understand the object version feature in Swift. > In the current impl

Re: [Openstack] OpenStack Summit Tracks & Topics

2012-08-14 Thread John Dickinson
SwiftStack would like to do an intro session on swift on Monday. We'll also be signing up for several workshops. --John On Aug 14, 2012, at 9:57 AM, Lauren Sell wrote: > We have space available Monday if someone wants to run a getting started > track in the morning (or all day), but it would

[Openstack] [Swift] Where do we go from here?

2012-08-14 Thread John Dickinson
Swift has many exciting features coming in the OpenStack Folsom Release this fall, but where do we go from here? What's next for Swift in grizzly? I've got some ideas. I'd like to mention them and see where you the community will take them. I've written up most of them into quick one- line bluepri

[Openstack] [Swift] community update and what's coming in Folsom

2012-08-13 Thread John Dickinson
We just released Swift 1.6.0 last Monday ( https://lists.launchpad.net/openstack/msg15505.html ). We've got a lot of great features and improvements in it, and I wanted to take some time to update the wider community about where Swift is. Swift 1.4.8 was included with the last OpenStack realease (

Re: [Openstack] Swift + keystone integration

2012-08-11 Thread John Dickinson
Make sure that the endpoint stored in keystone is returning the right hostname/domain name and port (8080 based on your config). --John On Aug 11, 2012, at 12:58 PM, Miguel Alejandro González wrote: > Hello > > I have 3 nodes with ubuntu 12.04 server and installed openstack with packages >

[Openstack] [swift] Operational knowledge sharing

2012-08-10 Thread John Dickinson
In a standard swift deployment, the proxy server is running behind a load balancer and/or an SSL terminator. At SwiftStack, we discovered an issue that may arise from some config parameters in this layer, and we'd like to share it with other swift deployers. Symptom: Users updating metadata (i

[Openstack] Swift 1.6.0 released

2012-08-06 Thread John Dickinson
I'm happy to announce that Swift 1.6.0 has been released. You can get the tarball at https://launchpad.net/swift/folsom/1.6.0. As always, you can upgrade your production Swift clusters to this new version with no downtime to your clients. The complete changelog for this release is at https://g

[Openstack] next swift release just around the corner

2012-07-23 Thread John Dickinson
The next swift release is scheduled for public release next Monday (July 30). That means we've got a little bit of work to do this week to get it ready. In order to allow Cloud Files QA time to check it, we need to have packages built by the middle of the day Wednesday. This means all outstandin

Re: [Openstack] [Swift] LFS patch (Ia32c9c34)

2012-07-18 Thread John Dickinson
Nexenta's LFS patch (https://review.openstack.org/#/c/7524/) has languished for a while, and I'd like to address that. First, thank you for your patch submission. This patch adds new functionality that potentially can allow swift to be deployed in more places. The original version of the patch,

Re: [Openstack] Statsd on SWIFT 1.4.8

2012-07-05 Thread John Dickinson
statsd integration was added in swift 1.5.0 On Jul 5, 2012, at 9:07 AM, Leandro Reox wrote: > On swift essex stable 1.4.8, is the capacity to send statistics to a statsd > server available? Its not in the docs > > Regards > ___ > Mailing list: https:

Re: [Openstack] [Swift] [Storage node] Lots of timeouts in load test after several hours around 1, 000, 0000 operations

2012-07-01 Thread John Dickinson
I hope you are able to get an answer. I'm traveling this week, so I won't have a chance to look in to it. I hope some of the other core devs will have a chance to help you find an answer. --John On Jul 1, 2012, at 2:03 PM, Kuo Hugo wrote: > Hi all , > > I did several loading tests for swif

Re: [Openstack] Swift Probetests

2012-06-26 Thread John Dickinson
The probe tests are internal whole-system type of tests that test functionality never exposed through normal integration testing. They exist at the level between unit tests and functional tests. For example, one of the probetests makes sure that asynchronous container updates actually happen. Un

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
On Jun 20, 2012, at 11:02 AM, Victor Rodionov wrote: > > Also, I want ask do you think it's good idea to store object ACL in object > metadata? I'd suggest looking at container-level ACLs rather than object-level. But either way, the data does need to be stored in the metadata in swift itself

Re: [Openstack] [Swift] S3 like ACL for Swift

2012-06-20 Thread John Dickinson
Yes, this could be good for swift. ACLs in swift do need to be stored in swift (for scale reasons), but their implementation is dependent on the particular auth system that you are using. The auth middleware is responsible for determining if a request is granted access to a particular swift ent

Re: [Openstack] [Swift] Interested in implementing swift ring builder server

2012-06-19 Thread John Dickinson
It looks like Florian (at Rackspace) is working on that blueprint. He just assigned it to himself. I'm happy to hear that you have some extra devs for swift work. I'd love to help coordinate some swift goals with you. Off the top of my head, here are a few things that could be worked on: 1) Ha

[Openstack] Swift 1.5.0 release

2012-06-05 Thread John Dickinson
I'm pleased to announce the release of swift 1.5.0. This is a big release with some big changes. I recommend that deployers upgrade, and, as always, you can upgrade your production cluster with no downtime. Swift docs: http://swift.openstack.org Swift code: http://github.com/openstack/swift Openst

Re: [Openstack] Opening up bug triaging rights

2012-05-14 Thread John Dickinson
On May 14, 2012, at 10:06 AM, Thierry Carrez wrote: > Hello everyone, > > Currently the bug triaging rights for a given PROJECT (ability to set > status and importance of bugs, but also ability to nominate a bug for a > past series) is restricted to the corresponding PROJECT-bugs team, which > i

[Openstack] [Swift] swift news and plans

2012-05-04 Thread John Dickinson
TL;DR: removing code from swift, associated projects doc, swift 1.5.0 I want to let the openstack community know of some recent changes within swift and how those changes will affect the next version of swift. Swift has a growing developer community and a rapidly expanding deployed base. While t

Re: [Openstack] [Openstack-operators] Question Regarding Swift Distribution Statistics

2012-05-02 Thread John Dickinson
On your proxy server, use swift-get-nodes to see which servers your object is on. With no arguments or --help you will get usage info. --John On May 2, 2012, at 3:48 PM, Duncan McGreggor wrote: > cc'ing openstack list > > On Wed, May 2, 2012 at 4:45 PM, Richard Raseley wrote: >> I am trying

Re: [Openstack] HTTP status value naming normalization

2012-04-21 Thread John Dickinson
I like what you are trying to do here. Can you please submit this as a patch through gerrit so we can get the rest of the core devs to look at it? --John On Apr 20, 2012, at 12:14 PM, Victor Rodionov wrote: > There are many place in Swift code where used hard coded values, such > as response s

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
I should also mention the summit session talking about this very topic led by Everett Toews. It's (currently) scheduled for 9am on wednesday. http://summit.openstack.org/sessions/view/81 --John On Apr 12, 2012, at 8:51 PM, John Dickinson wrote: > Swift keeps total bytes, contai

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

2012-04-12 Thread John Dickinson
Swift keeps total bytes, container, and object count (eventually) up-to-date in the account metadata. There are also log processing tools (like slogging - http://github.com/notmyname/slogging) that can provide usage information (including bandwidth) based on swift logs. While I think that it's

Re: [Openstack] Where does Keystone middleware for Swift belong?

2012-04-11 Thread John Dickinson
I do not think that these pieces of middleware belong in the core swift repo. 1) Including them in swift would require swift to depend on keystone for full testing. 2) (When the middleware was created) Keystone's API was in a state of constant flux. Keystone has changed quite a bit since then,

Re: [Openstack] Distributed rate-limiting

2012-03-17 Thread John Dickinson
If you're looking for wsgi rate limiting, take a look at swift's rate limiting middleware. It's distributed, simple (uses memcached to keep track of what's going on), supports multiple tiers of rate limiting (with interpolation between them), and white/black lists. While swift's rate limiting i

Re: [Openstack] Swift: NAS or DAS?

2012-03-16 Thread John Dickinson
Generally, you would introduce latency in the storage system by using a NAS attached to a storage drive. Also, at scale, your costs will be dominated by drive, so you will want to optimize the storage nodes for dense, cheap storage. --John On Mar 16, 2012, at 8:32 AM, Michaël Van de Borne wrot

Re: [Openstack] Swift concept architecture

2012-03-12 Thread John Dickinson
On Mar 11, 2012, at 12:16 AM, Dmitry Ukov wrote: > Hi all, > I want to introduce some ideas about Swift. > > Let’s assume we have huge amount of data stored in Swift (e.g. 10Pb). This > data are dynamically changed by users. So we need to reduce network load > caused by replication and intensi

Re: [Openstack] swift and two data-centres - hierarchical zones?

2012-03-12 Thread John Dickinson
On Mar 10, 2012, at 5:58 AM, John Leach wrote: > > I think what I need here is hierarchical zones - I'd define one parent > zone per data-centre, and then multiple child zones within each > (representing racks or whatever). > > Swift would be configured to write 3 replicas in 3 child zones, aimi

[Openstack] Swift 1.4.7 released

2012-03-09 Thread John Dickinson
I'm happy to announce swift 1.4.7 has been released today. Changelog: Full set of commits since the last release:

[Openstack] Swift patches for the essex release

2012-03-07 Thread John Dickinson
The final swift release for openstack essex is coming quickly. Swift 1.4.8 is tentatively scheduled for March 22nd. To get your patches into this release, please submit them for review by next week so we have enough time to review and QA them. --John smime.p7s Description: S/MIME cryptograp

[Openstack] swift authors list

2012-02-24 Thread John Dickinson
Stefano has asked that swift start keeping email addresses in our AUTHORS file. I'll be adding these soon, but some contributors have more than one email. If you have a particular email address you'd like to have me add to your name, please let me know. Otherwise, I will use one from git log. N

Re: [Openstack] [Swift] Cannot create container.

2012-02-24 Thread John Dickinson
a 507 response means that the drive was unmounted. If you are running this in a VM (like the SAIO), then you need to disable to mount check. Docs for this are in the SAIO docs. --John On Feb 24, 2012, at 9:13 AM, Leander Bessa wrote: > Hello, > > I'm trying to set up a custom swift server in

Re: [Openstack] Swift container ACLs and container visibility question

2012-02-23 Thread John Dickinson
It all depends on the auth system you are using. Below is for swauth and tempauth: Are the users using the same shared storage? If so, set them up as .admin users with the same storage endpoint. If they are not using the same shared storage, then you may be stuck. The ACL support in swauth and

Re: [Openstack] swprobe: swift middleware for sending metrics to graphite using statsd

2012-02-21 Thread John Dickinson
That's great. Have you by any chance seen https://github.com/pandemicsyn/swift-informant? It's something similar that we've been playing with at Rackspace. --John On Feb 21, 2012, at 10:36 AM, Jasper Capel wrote: > Hi all, > > I'm announcing a piece of Swift middleware, swprobe [1], designed

Re: [Openstack] python-swiftclient?

2012-02-13 Thread John Dickinson
On Feb 13, 2012, at 8:29 AM, Chmouel Boudjnah wrote: > > > What do you think if we : > > - split swift.common.client to its own. > - have bin/swift import that package and shipped with it. > - have a comprehensive test suite covering the CLI and the library. > - have some proper PIP release for

Re: [Openstack] Doubts about virtual CPUs and Swift storage capacity.

2012-02-11 Thread John Dickinson
On Feb 11, 2012, at 7:29 AM, Jorge Luiz Corrêa wrote: > > 2) About Swift, how do I determine the total usable storage capacity of the > system? For example: I have 3 nodes with 5 HDs of 1 TB each one. Straightly I > have 15 TB of space. If I use raid 1 I can say that I'm going to have 7,5 TB >

[Openstack] Swift 1.4.6 release

2012-02-10 Thread John Dickinson
I'm happy to announce that swift 1.4.6 has been released today. We've added some great new features in this release and fixed several outstanding bugs. The full changelog is below, but I'd like to highlight a few key points. Swift 1.4.6 includes new middleware that adds the ability to upload objec

Re: [Openstack] Swift Container Name Restrictions

2012-01-24 Thread John Dickinson
On Jan 24, 2012, at 5:22 PM, Matthew Wodrich wrote: > Hi Folks, > > I'm trying to write some scripts to work with Swift containers, but I don't > actually know what the restrictions on container names are. Does anyone know > what the specification is, or where I can read up on it? > > For ex

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
The storage volumes referenced in the ring are identified by an IP, port, and mount point. So, it is possible to use network attached storage for swift (as long as it still supports xattrs). However, I don't know if this has ever really been tried (especially in production), and I'd be surprised

Re: [Openstack] Swift supported file system

2012-01-23 Thread John Dickinson
That functionality is left up to a client. For example, you could use FUSE to spoof swift as a filesystem or you could use a client like Cyberduck or even write your own. Last week someone on this mailing list talked about adding webDAV support to swift. All of these "work" in that they present

Re: [Openstack] (no subject)

2012-01-19 Thread John Dickinson
look in syslog on your proxy server to see what caused the error. --John On Jan 19, 2012, at 6:28 PM, Khaled Ben Bahri wrote: > Hi all, > > I tryed to install OpenStack swift, > > after creating and configuring all nodes, when i want to check that swift > works, > I execute this command : >

Re: [Openstack] swift object versions

2012-01-12 Thread John Dickinson
On Thu, Jan 12, 2012 at 1:42 PM, John Dickinson wrote: > I've recently started working on adding versioning to objects stored in swift > (https://github.com/notmyname/swift/tree/version_manifest). Since this is > such a requested feature, I thought it would be a good idea to get ea

[Openstack] swift object versions

2012-01-12 Thread John Dickinson
I've recently started working on adding versioning to objects stored in swift (https://github.com/notmyname/swift/tree/version_manifest). Since this is such a requested feature, I thought it would be a good idea to get early input. I've created an etherpad (http://etherpad.openstack.org/swift-ob

[Openstack] swift essex status update

2012-01-11 Thread John Dickinson
he improvements we've made to swift, and I expect some more exciting things to come before our final essex release is made. As always, patches welcome! John Dickinson Swift Project Technical Lead notmyname on IRC smime.p7s Description: S/MIME cryptographic signature ___

Re: [Openstack] Using Gerrit to verify the CLA

2012-01-09 Thread John Dickinson
https://rackspace.echosign.com/verifier On Jan 9, 2012, at 8:08 AM, Mark McLoughlin wrote: > Hey, > > On Thu, 2012-01-05 at 10:02 -0800, James E. Blair wrote: >> This change is in place; membership in openstack-cla is required in >> order to submit changes to Gerrit. >> >> All of the -core gro

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-06 Thread John Dickinson
The best, technical description of the ring was written by the person who had the biggest role in writing it for swift: http://www.tlohg.com/p/building-consistent-hashing-ring.html --John smime.p7s Description: S/MIME cryptographic signature ___ Ma

[Openstack] swift release 1.4.5

2012-01-04 Thread John Dickinson
It's time again for a swift release. We have cut the swift 1.4.5 release and it's headed to QA. We expect it to be validated by the end of this week and it should land for public use early next week. Below is the changelog for this release. The highlights are the swift-orphans and swift-oldies

Re: [Openstack] Several questions about HOW SWIFT WORKS

2012-01-03 Thread John Dickinson
Answers inline. On Jan 3, 2012, at 11:32 AM, Alejandro Comisario wrote: > > So, lets get down to business. > > # 1 we have memcache service running on each proxy, so as far as we know, > memcache actually caches keystone tokens and object paths as the request ( > PUT , GET) enters the proxy,

Re: [Openstack] [RFC] Common config options module

2011-12-06 Thread John Dickinson
Overall, I think it's a great thing to have commonality between the projects on option names and environment variables. I think it's worthwhile to push for that in the swift cli tool in the essex timeframe. On the topic of common config libraries, though, I think the differences are less import

[Openstack] Swift 1.4.4 released

2011-11-24 Thread John Dickinson
release is https://launchpad.net/~swift-core/+archive/release. As always, we welcome all input and patches. The swift code is found on github at https://github.com/openstack/swift and we'd be happy to answer questions on the openstack mailing list or on IRC (#openstack on freenode). -

Re: [Openstack] rpms for rhel5.x to install Open stack Object Storage

2011-11-21 Thread John Dickinson
I suspect there is a communication gap somewhere, but this is certainly not the case. Openstack Object Storage (swift) is not deprecated. Glance provides a bridge between nova and swift, but all three are important, active projects. Sudhaker, I know that rpms exist for swift, but I don't know

Re: [Openstack] Keystone & Swift: swiftauth tenant namespace collisions?

2011-11-20 Thread John Dickinson
I don't think that is exactly right, but my understanding of tenants vs accounts vs users may be lacking. Nonetheless, auth v2.0 support was added to the swift cli tool by Chmouel recently. Have you tried with the code in swift's trunk (also the 1.4.4 release scheduled for Tuesday)? --John On

Re: [Openstack] Update Container Metadata.

2011-11-14 Thread John Dickinson
Updating existing container metadata with a POST is entirely supported and intended behavior. (The same also applies to adding/updating metadata on an account.) --John On Nov 14, 2011, at 8:10 PM, easco wrote: > Anne, > > Thank you for your help, and you're correct, that is the case I am pu

Re: [Openstack] Tutorials of how to install openstack swift into centos 6

2011-11-09 Thread John Dickinson
Awesome. Thanks for putting this together. I know a lot of people have been interested in getting swift running on non-ubuntu systems. Thanks for sharing this with everyone. --John On Nov 9, 2011, at 12:00 AM, pf shineyear wrote: > openstack swift install on centos 6 > > 1. proxy install >

Re: [Openstack] [Openstack-poc] proposal for policy around and management of client libraries

2011-11-08 Thread John Dickinson
On Nov 8, 2011, at 10:54 AM, Thierry Carrez wrote: > > With solution (2), if you look at the issue from Gerrit, GitHub, > Launchpad or Jenkins, those will be separate projects though. The fact > that they share the same PTL is not enough to make them "one". For > example, they will have separate

  1   2   >