Re: [Openstack] [Swift] Use JSON in object xattr

2013-04-06 Thread Michael Barton
On Sat, Apr 6, 2013 at 5:24 PM, Samuel Merritt wrote: Perhaps one of the original developers will chime in here. > All I can really say is that when that decision was made, there was no inkling that portability would ever be a concern. Pickle was fast and effective. Switching it to json should

Re: [Openstack] [swift] how to configure NWR policy

2012-12-10 Thread Michael Barton
Does any of our documentation say NRW are directly configurable? It's not really quite that simple. N is a property of the cluster, chosen when you build the ring. W is always a simple majority of N. R is generally 1 (eventually consisten reads), but the client can request it be bumped to N (> hi

Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Michael Barton
Overall I've never been super enthusiastic about erasure codes for swift. Figuring out which blocks are missing then re-assembling them is a lot more difficult and expensive than what we do now. But if you can come up with a good scheme for identifying missing blocks and it doesn't double the amo

Re: [Openstack] General questions regarding zones and replicas

2012-10-08 Thread Michael Barton
On Mon, Oct 8, 2012 at 8:12 AM, Moritz Krinke wrote: > - I cannot change the number of replicas after creating the ring (e.g. > changing it from 1 to 2) Not really. Logically it's not a difficult operation, there's just no code to do it. There was a patch a while back, but I guess it wasn't eve

Re: [Openstack] make swift.common.utils.streq_const_time more efficient

2012-09-13 Thread Michael Barton
That function's purpose is to compare strings without short-circuiting, to foil timing attacks against token comparisons or similar. On Thu, Sep 13, 2012 at 1:28 AM, Mike Green wrote: > def streq_const_time(s1, s2): > > if len(s1) != len(s2): > return False > result = 0 > for

Re: [Openstack] [Openstack swift][1.6.0] Regarding to swift api for "Controlling a Large List of Containers"

2012-08-28 Thread Michael Barton
On Tue, Aug 28, 2012 at 5:46 AM, Irene.Peng-彭怡欣-研究發展部 > API like below: > Curl –H ‘X-Auth-Token: ’ > http:///?marker=banana&limit=2 > I think you just need to put quotation marks around that URL -- the & is causing the curl command to be backgrounded by the shell, and cutting off the url being sen

Re: [Openstack] shuffle(nodes) in Swift

2012-07-05 Thread Michael Barton
On Thu, Jul 5, 2012 at 11:21 AM, Anatoly Legkodymov wrote: > I propose removing shuffle(nodes) from proxy-server will make memory caching > 3 times more efficient, without loosing in anything else. I don't know, this kind of stuff is pretty use case dependent. If you're not using page cache for

Re: [Openstack] [Swift] Lost of an object data file is not detected

2012-06-24 Thread Michael Barton
On Sun, Jun 24, 2012 at 9:05 PM, 山縣陽 wrote: > But if I removed the data file, no process detected it  and the data > files was not recovered. > Is this a bug?  or is there any way to detect a data file lost case? Nope, there's nothing that'll detect missing data files once the system's in a stead

Re: [Openstack] [keystone] v3 API draft (update and questions to the community)

2012-06-12 Thread Michael Barton
On Tue, Jun 12, 2012 at 3:24 AM, Gabriel Hurley wrote: > To speak on the specific feature of pagination, the problem of 'corruption' > by simultaneous writers is no excuse for not implementing it. You think > Google, Facebook, Flickr, etc. etc. etc. don't have this problem? If you > consume the

[Openstack] gerrit account help

2012-03-09 Thread Michael Barton
Can someone help me? I think my gerrit account is boned. redbo@ubuntu:~/swift$ git review fatal: A Contributor Agreement must be completed before uploading: http://wiki.openstack.org/HowToContribute -- Mike ___ Mailing list: https://la

Re: [Openstack] HA for Swift proxy and object server processes

2012-01-19 Thread Michael Barton
On Thu, Jan 19, 2012 at 3:52 PM, Bob Schatz wrote: > I noticed that if I kill any of the proxy or object server processes they > are not restarted. Worker processes should be restarted automatically (though I haven't looked at or tested that code in a while). There's a parent process that won't

Re: [Openstack] Swift Consistency Guarantees?

2012-01-17 Thread Michael Barton
On Tue, Jan 17, 2012 at 4:55 PM, Nikolaus Rath wrote: > Amazon S3 and Google Storage make very explicit (non-) consistency > guarantees for stored objects. I'm looking for a similar documentation > about OpenStack's Swift, but haven't had much success. I don't think there's any documentation on t

Re: [Openstack] swift enforcing ssl?

2011-12-28 Thread Michael Barton
On Tue, Dec 27, 2011 at 2:11 PM, andi abes wrote: > Does the swift proxy enforce SSL connections if it's configured with a > cert/key file? Or is it assumed that there's an external entity performing > that? The Swift proxy's SSL support is probably only useful for light testing - SSL in python (

Re: [Openstack] Swift slow write performance

2011-12-19 Thread Michael Barton
On Mon, Dec 19, 2011 at 6:21 AM, Rustam Aliyev wrote: > The only thing which looks suspicious to me are these errors: > > Dec 18 04:01:28 ec01 object-server ERROR container update failed with > 10.0.1.3:6001/d01 (saving for async update later): Timeout (3s) (txn: > txdf95ad5a10844ee0b74d70d8a76380

Re: [Openstack] Writes are faster than reads in Swift

2011-12-13 Thread Michael Barton
On Tue, Dec 13, 2011 at 9:21 PM, Huang Zhiteng wrote: > Can anyone explain why Swift doesn't want to utilize page cache _at all_? It's an artifact of the use case swift was built for - heavy on writes, and repeat reads (where a cache would help) are very rare. Having that memory available to cach

Re: [Openstack] Writes are faster than reads in Swift

2011-12-13 Thread Michael Barton
I can't explain it off the top of my head. I don't have a swift installation to play with at the moment, but it's conceivable that posix_fadvise is slower than we expect (drop_cache is called more frequently during reads than writes, iirc). That could be tested by making drop_cache a no-op in the

Re: [Openstack] why swift proxy server use poll not epoll??

2011-12-11 Thread Michael Barton
On Sun, Dec 11, 2011 at 4:08 AM, pf shineyear wrote: > hi all , can any one tell my the reson of swift proxy server http service > use python eventlet poll not epoll? Sure. We had a problem where epoll failed to report a socket close event roughly one out of every bazillion times. Then eventlet

Re: [Openstack] question about X-Timestamp header

2011-11-28 Thread Michael Barton
On Mon, Nov 28, 2011 at 9:04 PM, pf shineyear wrote: > hi all > i think X-Timestamp header is come from the proxy servery to object storage > node, the value is proxy server current time. > if i have 2 or more proxy server run in one cluster, should i comfirm same > "account/container/filename" us

Re: [Openstack] Issues with Packaging and a Proposal

2011-08-24 Thread Michael Barton
On Wed, Aug 24, 2011 at 3:11 PM, Soren Hansen wrote: > Rackspace isn't doing their own packaging because of (lack of) Debian > support. If they did, they'd have realised almost immediately that the > packages actually build on Debian. They're doing it because there'll > supposedly be differences i

Re: [Openstack] [Keystone] legacy_token_auth

2011-08-16 Thread Michael Barton
On Mon, Aug 15, 2011 at 6:44 PM, Nguyen, Liem Manh wrote: > Currently, we have the legacy_token_auth to “help” with integration with > services (like Swift) that still talks auth v1.0.  Will this filter go away > and other services will be speaking pure Keystone when Diablo releases? It would ki

Re: [Openstack] suggestion for data backup/recovery in swift

2011-05-31 Thread Michael Barton
On Tue, May 31, 2011 at 5:54 AM, Rostyslav Slipetskyy wrote: >> The problem is that every version would be stored on the same drives, which >> is >>gonna screw with storage distribution. > > Could you please clarify this? Sure. We've talked about this as a method of adding a "versioning" featur

Re: [Openstack] suggestion for data location compliance in swift

2011-05-31 Thread Michael Barton
On Tue, May 31, 2011 at 8:37 AM, Rostyslav Slipetskyy wrote: > The idea to make ring implementation pluggable seems nice. > At the same time I am thinking that many developers might not will feel > comfortable with modifying existing ring structure, since it *works* > :) Probably, the other viable

Re: [Openstack] suggestion for data backup/recovery in swift

2011-05-30 Thread Michael Barton
(re-sent to list) On Mon, May 30, 2011 at 4:56 PM, Rostyslav Slipetskyy wrote: > It looks like backup/recovery can be easily added to swift without many > changes > in the source code. After successful file upload via PUT method, stored file > receives name equal to timestamp and all the previou

Re: [Openstack] Getting pagination right

2011-05-26 Thread Michael Barton
On Wed, May 25, 2011 at 2:40 PM, Jay Pipes wrote: > The pagination in Swift is not consistent. Inserts into the Swift > databases in between the time of the initial query and the requesting > the "next page" can result in rows from the original first page > getting on the new second page. No, yo

Re: [Openstack] lunr reference iSCSI target driver

2011-05-02 Thread Michael Barton
On Mon, May 2, 2011 at 9:12 PM, Michael Barton wrote: > What I've been playing with is having a manifest that contains hashes > of (4mb) chunks for the volume's backups.  When a user initiates a new > backup, dm-snapshot does its thing and gives me a block device.  I > rea

Re: [Openstack] lunr reference iSCSI target driver

2011-05-02 Thread Michael Barton
non-scientific) tests. All of this is sort of at the proof of concept stage at the moment. -- Michael Barton ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/

Re: [Openstack] Proposing an Identity Service in OpenStack (a.k.a. Auth)

2011-04-18 Thread Michael Barton
On Mon, Apr 18, 2011 at 12:15 PM, Eric Day wrote: > We'll also want to decide if we need a default mechanism for > OpenStack deployments, and if so, what should it be. We had a > discussion previously and I think it was somewhere between token > and HTTP basic w/ SSL. The reason for this is we nee

Re: [Openstack] Logging, 500, Debugging

2011-04-04 Thread Michael Barton
On Mon, Apr 4, 2011 at 4:50 PM, Zed A. Shaw wrote: > I'm currently trying to debug some changes to the Swift proxy server and > finding the logging facilities a little obtuse.  Reading through: Most tracebacks should go to the logs, but yeah, there just isn't a lot of debug-level logging in the

Re: [Openstack] Feature Freeze status

2011-03-31 Thread Michael Barton
I'm gonna +1 Todd. Actually, apache server has a great dev process. They have goals for releases, but people are welcome to submit patches to their mailing list any time, get comments on them, then they're merged if and when people vote them as ready. -- Mike ___

Re: [Openstack] State of OpenStack Auth

2011-03-04 Thread Michael Barton
On Fri, Mar 4, 2011 at 1:43 PM, Greg wrote: > :) Okay, I stand corrected on the tests passing part; that was a jab though > it did take a while to get there. > I now back away from auth once again, as I have so many times before... Well, it still wasn't a great branch. If it had just been a plu

Re: [Openstack] State of OpenStack Auth

2011-03-02 Thread Michael Barton
On Wed, Mar 2, 2011 at 2:38 PM, Soren Hansen wrote: > I'd like to just mention this blog post: > >   http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html > > tl;dr quote: > >    If you stop reading now you only need to remember one thing: >    SSL/TLS is not computationally expensive any

Re: [Openstack] State of OpenStack Auth

2011-03-02 Thread Michael Barton
> a secure channel too, but if not attacks are less severe since they > are limited to reply attacks only (the request and parameters are used > as part of the signature). We can easily support both (and others), > but we need to understand the needs and constraints of each. HMAC is sort of appeal

Re: [Openstack] Entities in OpenStack Auth

2011-03-02 Thread Michael Barton
> Swift > > Swift has the concept of accounts, users, and groups. An account > contains users, and a user can belong to groups. Accounts names have an > abstraction layer, so while you may login with account "example.com", > the account name used within swift is a UUID with a prefix. > > By default

Re: [Openstack] Queue Service, next steps

2011-02-19 Thread Michael Barton
On Fri, Feb 18, 2011 at 9:38 PM, Eric Day wrote: > Hi Mike, > > You make a good point, I apologize for not documenting some of the > ideas sooner. The architecture I had in mind borrows from other > queue systems I've worker on, along with feedback I've gotten from > various conversations while ge

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Michael Barton
On Fri, Feb 18, 2011 at 6:23 PM, Eric Day wrote: > Perhaps I've been assuming some things, but I thought everyone > understood that is what we are looking to build (fault-tolerant, > horizontally scalable, ...). We're certainly not looking to build > a clustered queue (like RabbitMQ) with a REST A

Re: [Openstack] Queue Service, next steps

2011-02-18 Thread Michael Barton
On Fri, Feb 18, 2011 at 5:22 PM, Eric Day wrote: > The main question right now is where to land on the spectrum of service > efficiency vs ease of development (C/C++ on one end and Python on > the other). It seems we're landing in the middle with Erlang. :) Maybe I'm describing a separate project

Re: [Openstack] Queue Service

2011-02-14 Thread Michael Barton
On Mon, Feb 14, 2011 at 7:57 PM, Paul Voccio wrote: > Looking at the swift docs, they reference a container like so: > >  METHOD /v1// HTTP/1.1 Yeah, this has worked out well for us. Delegated access, authentication methods that don't provide the account name otherwise, anonymous access... And

Re: [Openstack] Deprecating nova-objectstore

2011-01-17 Thread Michael Barton
On Mon, Jan 17, 2011 at 1:19 PM, Thierry Carrez wrote: > For serious deployments I guess we would use Swift S3 frontend. For demo It's neat and all, but I don't think you should rely on that plugin existing in all swift installations. -- Mike ___ Mail

Re: [Openstack] Use of IANA-registered ports

2011-01-03 Thread Michael Barton
On Sun, Jan 2, 2011 at 7:57 PM, Ewan Mellor wrote: > We've got some complications though: > http://swift.openstack.org/howto_installmultinode.html says "Auth node: ... > This can be on the same node as a Proxy node" and "Storage nodes: Runs the > swift-account-server, swift-container-server, an

Re: [Openstack] Merge proposals and criteria for approval

2010-12-23 Thread Michael Barton
On Tue, Dec 21, 2010 at 1:51 PM, Michael Barton wrote: > I kind of feel like accepting broken code just because it has good > documentation, test coverage or whatever is backwards. > > But I don't know much about what's going on with nova development.  It > might make se

Re: [Openstack] Merge proposals and criteria for approval

2010-12-21 Thread Michael Barton
On Tue, Dec 21, 2010 at 10:43 AM, Rick Clark wrote: > I think that we should focus on finding regressions at review time not bugs. > > I propose that the following criteria be used for approving code reviews: >  * Architectural soundness >  * regression free >  * Code cleanliness (Pep8 compliance)

Re: [Openstack] OpenStack core components library

2010-08-28 Thread Michael Barton
On Fri, Aug 27, 2010 at 11:05 AM, Jorge Williams < jorge.willi...@rackspace.com> wrote: > Sure, > > We're aiming for consistency with our public APIs so that: > > 1) Collections are handled consistently > a) Pagination works the same across APIs > b) Filtering works the same way across