Re: [openstack-dev] Swift global cluster replication latency...

2014-08-18 Thread Clay Gerrard
Correct, best-effort. There is no guarantee or time boxing on cross-region replication. The best way to manage cross site replication is by tuning your replica count to ensure you have primary copies in each region - eventually. Possibly evaluate if you need write_affinity at all (you can always

Re: [openstack-dev] Gate Status - Friday Edition

2014-01-24 Thread Clay Gerrard
> > > > That's a pretty high rate of failure, and really needs investigation. > That's a great point, did you look into the logs of any of those jobs? Thanks for bringing it to my attention. I saw a few swift tests that would pop, I'll open bugs to look into those. But the cardinality of the fa

Re: [openstack-dev] Gate Status - Friday Edition

2014-01-24 Thread Clay Gerrard
OH yeah that's much better. I had found those eventually but had to dig through all that other stuff :'( Moving forward I think we can keep an eye on that page, open bugs for those tests causing issue and dig in. Thanks again! -Clay On Fri, Jan 24, 2014 at 11:37 AM, Sean Dague wrote: > On 0

Re: [openstack-dev] [swift] a way of checking replicate completion on swift cluster

2014-11-20 Thread Clay Gerrard
You might check if the swift-recon tool has the data you're looking for. It can report the last completed replication pass time across nodes in the ring. On Thu, Nov 20, 2014 at 1:28 AM, Matsuda, Kenichiro < matsuda_keni...@jp.fujitsu.com> wrote: > Hi, > > I would like to know about a way of chec

Re: [openstack-dev] [swift] a way of checking replicate completion on swift cluster

2014-11-24 Thread Clay Gerrard
-- > > * object > > ---------- > # curl http://192.168.1.11:6002/recon/replication | python -mjson.tool > { > "object_replication_last": 1416334368.60865, >

Re: [openstack-dev] [swift] a way of checking replicate completion on swift cluster

2014-12-04 Thread Clay Gerrard
On Tuesday, November 25, 2014 4:37 PM, Matsuda, Kenichiro [mailto: > matsuda_keni...@jp.fujitsu.com] wrote: > > I understood that the logs are necessary to judge whether no failure on > > object-replicator. > > And also, I thought that the recon info of object-replicator having &g

Re: [openstack-dev] (no subject)

2014-05-09 Thread Clay Gerrard
I thought those tracebacks only showed up with old versions of eventlet or and eventlet_debug = true? In my experience that normally indicates a client disconnect on a chucked encoding transfer request (request w/o a content-length). Do you know if your clients are using transfer encoding chunked

Re: [openstack-dev] openstack-swift put performance

2014-03-07 Thread Clay Gerrard
Well... What results did you get? What did you expect? What do you hope to achieve? How are you balancing your client requests across the five nodes? I'm not sure you're going to get anywhere near 2000^H^H requests a second from a single thread (!?) - Swift's performs best against many concurre

Re: [openstack-dev] [SWIFT] SWIFT object caching (HOT content)

2014-03-11 Thread Clay Gerrard
At the HK summit, the topic of hot content came up and seemed to broken into two parts. 1) developing a "caching" storage tier for hot content that would allow proxies to more quickly serve small data requests with even higher rates of concurrent access. 2) developing a mechanism to programmatical

Re: [openstack-dev] [SWIFT] Delete operation problem

2014-04-22 Thread Clay Gerrard
409 on DELETE (object?) is a pretty specific error. That should mean that the timestamp assigned to the delete is earlier than the timestamp of the data file. Most likely mean that you're getting some time-drift on your proxies (but that assumes multi-node) or maybe that you're reusing names betw

Re: [openstack-dev] [Ceph] Why performance of benchmarks with small blocks is extremely small?

2014-09-29 Thread Clay Gerrard
I also have limited experience with Ceph and rados bench - but it looks like you're setting the number of "worker threads" to only 1? (-t 1) I think the default is 16, and most storage distributed storage systems designed for concurrency are going to do a bit better if you exercise more concurren

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clay Gerrard
On Mon, Sep 29, 2014 at 2:53 PM, Chmouel Boudjnah wrote: > > > eventual consistency will only affect container listing and I don't think > there is a need for container listing in that driver. > > well now hold on... if you're doing an overwrite in the face of server failures you could still ge

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clay Gerrard
On Mon, Sep 29, 2014 at 4:15 PM, Clint Byrum wrote: > It would, however, be bad to get a 404 for something that is otherwise > present.. as that will result in an erroneous failure for the client. > > That almost never happens, but is possible if all the primaries are down*, a system than leans

Re: [openstack-dev] Information Dispersal Algorithm implementation in OpenStack

2014-11-10 Thread Clay Gerrard
I've never read that paper before and can not find a free copy online. Based on the abstract it seems to be a parity based algorithm (erasure codes) and would not be directly applicable to replication based dispersion/data placement. There is current work to enable an erasure code scheme for data

Re: [openstack-dev] [swift]Questions on concurrent operations on the same object

2014-11-10 Thread Clay Gerrard
Did you find out anything more on this? There's lots on places in Swift organized around concurrent access to objects - so I think it's probably good that you have that 423 response; your clients will probably see it... When you have multiple replicas the proxy's PUT will return shortly after it

Re: [openstack-dev] [qa] Policy on spelling and grammar

2013-11-11 Thread Clay Gerrard
On Mon, Nov 11, 2013 at 11:02 AM, John Griffith wrote: > > Seems to me there's a middle ground here, but honestly if you're value > add to the review process is catching grammatical or spelling errors > in comments and commit messages I'd argue that in most cases it would > be nice to have more s

Re: [openstack-dev] [Swift] erasure codes, digging deeper

2013-07-18 Thread Clay Gerrard
I'm not sure if I agree with the giving justifications (harder to bill, confusing to users), but I *do* think it could simplify some of the implementation (since users can't create accounts withe the "wrong" storage policy willy nilly). I think eventually some use cases may want mixed accounts (pa

[openstack-dev] clay.gerrard

2013-07-20 Thread Clay Gerrard
http://domeincheck.belgon.nl/pfape/yob.cckwcysypwqaxpjciyf ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Swift] Swift Auth systems and Delay Denial

2013-07-23 Thread Clay Gerrard
I think delay_denial will have to be maintained for awhile for backwards compatibility no matter what happens. I think existing auth middlewares can and often do reject requests outright without forwarding them to swift (no x-auth-token?). I think get_info and the env caching is relatively new, d

Re: [openstack-dev] [Swift] container-sync problems in global data sync senario

2013-08-09 Thread Clay Gerrard
Taking a "snapshot" of swift seems counterintuitive to me. It seems like there would be a number of gaps - no locking to allow for a consistent image, PUT overwrites are immediate, no index of deltas between snapshots. Maybe I misunderstand your goal. I don't believe container sync shouldn't se

Re: [openstack-dev] [swift] ssbench inconsistent results across identical runs

2013-08-12 Thread Clay Gerrard
Three things. 1) make your runs last longer than a few seconds, a sustained load over a longer period will average out some of jitter.. 2) You don't have enough log lines. Is this a single replica cluster? How does that work? 3) If you're benchmarking with the background daemons on, you might

Re: [openstack-dev] [swift] temp auth tokens expire immediately

2013-08-21 Thread Clay Gerrard
Sure *looks* like an issue with memcache... You might try narrowing down your list of servers in the memcache ring so you limit the places things might go wrong and try work you way back starting with something that works. Can you get this working: clayg@swift:~$ echo $ST_USER test:teste

Re: [openstack-dev] [keystone] driver/pluggable base classes and ABCMeta

2013-08-22 Thread Clay Gerrard
I recently took a peek at stevedore [1] which I think Doug has to take credit for in part through work on ceilometer. I bumped into it cause Kurt called it out in his post yesterday on Marconi [2]. ANYWAY, the stevedore documentation also calls out ABC's as a good model for plugins [3], so there

Re: [openstack-dev] [swift] getting middleware connected again

2013-08-29 Thread Clay Gerrard
add-apt-repository -y ppa:swift-core/release ^ is that a thing? How sure are you that you're running 1.9.2.6.g3b48a71 ? https://launchpad.net/~swift-core/+archive/release Try: python -c 'import swift; print swift.__version__' python -c 'import swift.common.middleware.catch_errors; print "SUCCE

Re: [openstack-dev] [tripleo] glance backend: replace swift by file in CI

2016-06-27 Thread Clay Gerrard
There's probably some minimal gain in cross compatibility testing to sticking with the status quo. The Swift API is old and stable, but I believe there was some bug in recent history where some return value in swiftclient changed from a iterable to a generator or something and some aggressive non-

Re: [openstack-dev] [openstack-swift] JOSS package to access Swift - any experience?

2016-01-11 Thread Clay Gerrard
I've used it before in a limited capacity; and still recommend it to java developers who want to use a language binding to connect to swift. As best I can tell the primary maintainers for the project still work at the same company, but there are some pending issues than't haven't received attentio

[openstack-dev] RFC 2616 was *so* 2010

2016-02-05 Thread Clay Gerrard
... really more like 1999, but when OpenStack started back in '10 - RFC 2616 was the boss. Since then (circa '14) we've got 7230 et. al. - a helpful attempt to disambiguate things! Hooray progress! But when someone recently opened this bug I got confused: https://bugs.launchpad.net/swift/+bug/1

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-21 Thread Clay Gerrard
How did you "deleted one data fragment"? Like replication the EC consistency engine uses some sub directory hashing to accelerate replication requests in a consistent system - so if you just rm a file down in an hashdir somewhere you also need to delete the hashes.pkl up in the part dir (or call t

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Clay Gerrard
On Wed, Jul 22, 2015 at 12:24 PM, Changbin Liu wrote: > > But now I wonder: is it "by design" that EC does not handle an accidental > deletion of just the data file? > Well, the design goal was not "do not handle the accidental deletion of just the data file" - it was "make replication fast enou

Re: [openstack-dev] [Openstack] [Swift] Erasure coding reconstructor doesn't work

2015-07-22 Thread Clay Gerrard
On Wed, Jul 22, 2015 at 12:37 PM, Luse, Paul E wrote: > > > Wrt why the replication code seems to work if you delete just a .data > no it doesn't -> https://gist.github.com/clayg/88950d77d25a441635e6 > forces a listing every 10 passes for some reason. Clay? > IIRC the every 10 passes trick

Re: [openstack-dev] [glance] Removing python-swiftclient from requirements.txt

2015-07-28 Thread Clay Gerrard
Doug, I believe our glance friends are not the only project with some open questions on dealing with the "required dependency for optional plugin" use-case. You've made a recommendation to leverage some python tooling functionality that I'm not familiar with. I was hoping I could probe you to el

Re: [openstack-dev] [glance] Removing python-swiftclient from requirements.txt

2015-07-29 Thread Clay Gerrard
So helpful! Thank you. On Wed, Jul 29, 2015 at 7:48 AM, Doug Hellmann wrote: > > There is some documentation in the pbr manual > (http://docs.openstack.org/developer/pbr/#extra-requirements). The > feature is implemented throughout the packaging tool chain now. > > Ah, excellent! PEP 0426 see

Re: [openstack-dev] [packaging][all] setting minimum version of setuptools in setup.py

2015-07-29 Thread Clay Gerrard
I agree an error message is better than breaking for insane reasons. But... maybe as an aside... what about not breaking? How come the openstack ecosystem doesn't have wait for PEP 426 to be approved and for setuptools 17.1 to be widely deployed before it can require/depend on it? Is there no fa

[openstack-dev] [hacking] [style] multi-line imports PEP 0328

2015-08-24 Thread Clay Gerrard
So, I know that hacking has H301 (one import per line) - but say maybe you wanted to import *more* that one thing on a line (there's some exceptions right? sqlalchemy migrations or something?) Anyway - I'm sure there could be a pep8 plugin rule that enforces use of parenthesis for multi line impo

Re: [openstack-dev] [hacking] [style] multi-line imports PEP 0328

2015-08-25 Thread Clay Gerrard
On Tue, Aug 25, 2015 at 8:45 AM, Kevin L. Mitchell < kevin.mitch...@rackspace.com> wrote: > On Mon, 2015-08-24 at 22:53 -0700, Clay Gerrard wrote: > > So, I know that hacking has H301 (one import per line) - but say maybe > > you wanted to import *more* that one thing on

Re: [openstack-dev] Feedback about Swift API - Especially about Large Objects

2015-10-09 Thread Clay Gerrard
A lot of these deficiencies are drastically improved with static large objects - and non-trivial to address (impossible?) with DLO's because of their dynamic nature. It's unfortunate, but DLO's don't really serve your use-case very well - and you should find a way to transition to SLO's [1]. We t

Re: [openstack-dev] [cinder] Some Changes to Cinder Core

2015-05-22 Thread Clay Gerrard
FWIW, as a nod to the great people we've had the privilege of working with as Swift Core Maintainers - we've taking to promoting people who have moved on to Core Emeritus: https://review.openstack.org/#/c/155890/ On Fri, May 22, 2015 at 4:34 PM, Mike Perez wrote: > This is long overdue, but it

Re: [openstack-dev] M Naming Poll ended - results still need to clear legal

2015-06-22 Thread Clay Gerrard
http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_4983776e190c8dbc how is the top pick not the author of the book of five rings [1] -Clay 1. https://en.wikipedia.org/wiki/The_Book_of_Five_Rings On Mon, Jun 22, 2015 at 7:07 AM, Monty Taylor wrote: > Hey all! > > The M naming poll has conclu

Re: [openstack-dev] [devstack] [swift] [ceilometer] installing ceilometermiddleware

2015-06-29 Thread Clay Gerrard
Is Swift the only project that uses the ceilometermiddleware - or just the only project that uses ceilometermiddleware that doesn't already have a oslo.config instance handy? FWIW There's a WIP patch that's trying to bring a *bit* of oslo.config love to the keystone middleware for policy.json [1].

Re: [openstack-dev] Why doesn't Swift cache object data?

2015-06-29 Thread Clay Gerrard
On Fri, Jun 26, 2015 at 8:16 PM, Michael Barton wrote: > > What's the logical difference between having object data in memory on a > memcache server and having it in page cache on an object server? > +1 - about a syscall - i.e. not much - I think memcache does it's own heap management - so it's

Re: [openstack-dev] [Swift] Swift GUI (free or open source)?

2015-01-27 Thread Clay Gerrard
https://github.com/cschwede/django-swiftbrowser is done by a swift core dev You should browse: http://docs.openstack.org/developer/swift/associated_projects.html#associated-projects On Mon, Jan 26, 2015 at 11:50 AM, Adam Lawson wrote: > I'm researching for a web-based visualization that simply

Re: [openstack-dev] [qa][swift] Signature of return values in tempest swift client

2015-02-13 Thread Clay Gerrard
On Fri, Feb 13, 2015 at 2:15 PM, David Kranz wrote: > Swift is different in that most interesting data is in the headers except > for GET methods, and applying the same methodology as the others does not > make sense to me. There are various ways the swift client could be changed > to return one

Re: [openstack-dev] H302 considered harmful

2015-02-27 Thread Clay Gerrard
So, Swift doesn't enforce H302 - and our imports are sorta messy frankly - but it doesn't really bother me, and I do rather enjoy the terseness of not having to spell out the module name. It's not really a chore to maintain, if you don't know where a name came from split the window (or drop a mark

Re: [openstack-dev] auto-abandon changesets considered harmful (was Re: [stable][all] Revisiting the 6 month release cycle [metrics])

2015-03-02 Thread Clay Gerrard
On Mon, Mar 2, 2015 at 8:07 AM, Duncan Thomas wrote: > Why do you say auto-abandon is the wrong tool? I've no problem with the 1 > week warning if somebody wants to implement it - I can see the value. A > change-set that has been ignored for X weeks is pretty much the dictionary > definition of a

Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-11 Thread Clay Gerrard
On Mon, Mar 9, 2015 at 12:27 PM, Weidong Shao wrote: > > I noticed swauth project is not actively maintained. In my local testing, > swauth did not work after I upgraded swift to latest. > > Hrm... I think gholt would be open to patches/support, I know of a number of deployers of Swauth - so mayb

Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-11 Thread Clay Gerrard
On Wed, Mar 11, 2015 at 1:16 PM, Weidong Shao wrote: > > ​the url is encoded in the object hash​! This somehow entangles the data > storage/validity with its account and makes it difficult to migrate the > data. I guess it is too late to debate on the design of this. Do you know > the technical r

Re: [openstack-dev] [OpenStack-Dev] [Cinder] Ubuntu LVM hangs in gate testing

2015-03-16 Thread Clay Gerrard
Can the bits that make those devices invalid and udev out of date call udev admin --settle to just block till things are upto date and hopefully the subseqent vg and pv scans quicker? On Monday, March 16, 2015, John Griffith wrote: > Hey Everyone, > > Thought I'd reach out to the ML to see if so

Re: [openstack-dev] [Swift] HEAD Object API status code

2015-05-06 Thread Clay Gerrard
Can you give an example of an Object HEAD request returning 204? I tried a HEAD of an object with a body and also a HEAD of an object of length 0 and I seem to get 200's... Container's and accounts are a little more interesting story... [2] -Clay 2. https://review.openstack.org/#/c/32647/ On W

Re: [openstack-dev] [swift] Go! Swift!

2015-05-07 Thread Clay Gerrard
On Thu, May 7, 2015 at 3:48 PM, Clint Byrum wrote: > I'm still very curious to hear if anybody has been willing to try to > make Swift work on pypy. yeah, Alex Gaynor was helping out with it for awhile. It worked. And it helped. A little bit. Probably still worth looking at if you're curiou

Re: [openstack-dev] [swift] Go! Swift!

2015-05-07 Thread Clay Gerrard
On Thu, May 7, 2015 at 5:05 PM, Adam Lawson wrote: > what sort of limitations have you discovered that had to do specifically > with the fact we're using Python? > Python is great. Conscious decision to optimize for developer wall time over cpu cycles has made it a great language for 20 years -

Re: [openstack-dev] [kolla][infra] does someone cares about Jenkins? I stopped.

2017-03-02 Thread Clay Gerrard
On Thu, Mar 2, 2017 at 10:41 AM, Paul Belanger wrote: > In fact, the openstack-infra team does mirror a > lot of things today I bumped into this the other day: https://specs.openstack.org/openstack-infra/infra-specs/specs/unified_mirrors.html ... but so far haven't found any specific details

Re: [openstack-dev] [all][TripleO][release][deployment] Packaging problems due to branch/release ordering

2017-04-05 Thread Clay Gerrard
I hate this stuff. Not just pbr (tho I do have a long history of being kicked in the nuts by pbr for no good reason I can ascertain). But when suddenly some process OpenStack invented I've never *heard of in two years* breaks - and overnight me and 100's of other folks have to stop what their doi

Re: [openstack-dev] [devstack] experimenting with systemd for services

2017-04-05 Thread Clay Gerrard
On Wed, Apr 5, 2017 at 1:30 PM, Andrea Frittoli wrote: > > > I just want to say thank you! to you clarkb clayg and everyone involved :) > This is so much better! > > andreaf > > Sean is throwing credit at me where none is due. IIRC I was both in the room and in a very-normal-for-me state of confu

Re: [openstack-dev] [Swift] 404 on geo-replicated clusters with write affinity

2017-05-17 Thread Clay Gerrard
Hi Bruno, On Wed, May 17, 2017 at 3:47 PM, Bruno L wrote: > > I see multiple bugs in launchpad that are related to this issue. > AFAIK, only one bug for this issue is still open, and has the most recent thoughts added from the Forum https://bugs.launchpad.net/swift/+bug/1503161 write a brief

Re: [openstack-dev] [swift] new additional team meeting

2017-05-26 Thread Clay Gerrard
On Fri, May 26, 2017 at 4:06 PM, John Dickinson wrote: > > The new meeting is at a reasonable time for just about everyone, other > than those who live in New York to San Francisco time zones. define *un*-reasonable ;) Regardless we'll have the logs. > > I'd like to thank Mahati for leading

Re: [openstack-dev] [tc][ptls][all] Potential Queens Goal: Migrate Off Paste

2017-06-02 Thread Clay Gerrard
Can we make this (at least) two (community?) goals? #1 Make a thing that is not paste that is better than paste (i.e. > works, ie >= works & is maintained) #2 Have some people/projects "migrate" to it If the goal is just "take over paste maintenance" that's maybe ok - but is that an "OpenStack co

Re: [openstack-dev] [cinder] Target classes in Cinder

2017-06-02 Thread Clay Gerrard
On Fri, Jun 2, 2017 at 12:47 PM, John Griffith wrote: > > > What I'm wondering is, even though I certainly think this is a FAR > SUPERIOR design to what we had, I don't like having both code-paths and > designs in the code base. > Might be useful to enumerate those? Perhaps drawing attention to

Re: [openstack-dev] [tc][ptls][all] Potential Queens Goal: Migrate Off Paste

2017-06-02 Thread Clay Gerrard
On Fri, Jun 2, 2017 at 1:21 PM, Matt Riedemann wrote: > > I don't think the maintenance issue is the prime motivator, it's the fact > paste is in /etc which makes it a config file and therefore an impediment > to smooth upgrades. The more we can move into code, like default policy and > privsep,

Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology

2017-06-15 Thread Clay Gerrard
On Thu, Jun 15, 2017 at 8:07 AM, Sean Dague wrote: > > I do kind of wonder if we returned the stackforge or > friends-of-openstack or whatever to the github namespace when we > mirrored if it would clear a bunch of things up for people. It would > just need to be an extra piece of info in our pro

Re: [openstack-dev] [tc] Status update, Jun 16

2017-06-16 Thread Clay Gerrard
On Fri, Jun 16, 2017 at 7:19 AM, Doug Hellmann wrote: > Excerpts from Thierry Carrez's message of 2017-06-16 11:17:30 +0200: > > > == Need for a TC meeting next Tuesday == > > > > In order to make progress on the Pike goal selection, I think a > > dedicated IRC meeting will be necessary. We have

Re: [openstack-dev] [nova] bug triage experimentation

2017-06-23 Thread Clay Gerrard
Sean, This sounds amazing and Swift could definitely use some [automated] assistance here. It would help if you could throw out a WIP somewhere. First thought that comes to mind tho storyboard.o.o :\ -Clay On Fri, Jun 23, 2017 at 9:52 AM, Sean Dague wrote: > The Nova bug backlog is just

Re: [openstack-dev] [all][tc] How to deal with confusion around "hosted projects"

2017-06-28 Thread Clay Gerrard
On Wed, Jun 28, 2017 at 7:50 AM, Thierry Carrez wrote: > > 2- it lets us host things that are not openstack but which we work on > (like abandoned Python libraries or GPL-licensed things) in a familiar > environment > > Do we no longer think openstack hosted infra holds a competitive advantage fo

Re: [openstack-dev] [all][tc] How to deal with confusion around "hosted projects"

2017-06-28 Thread Clay Gerrard
On Wed, Jun 28, 2017 at 7:50 AM, Thierry Carrez wrote: > It's hard for newcomers to the OpenStack world to see what > is a part of OpenStack and what's not. Just an aside, this Perception problems works in our favor sometimes too. I know in the past some BigCorp contributors have been told to "

Re: [openstack-dev] 503 Errors for PUT and POST calls....

2017-08-08 Thread Clay Gerrard
Probably the "devices" option in the object server is misconfigured? On my lab and production servers I configure the object-server.conf with [DEFAULT] devices = /srv/node And then I make sure my mounted devices appear at: /srv/node/d1 /srv/node/d2 /srv/node/d3 etc The path in the error messa

Re: [openstack-dev] [glance] multi threads with swift backend

2017-09-06 Thread Clay Gerrard
I'm pretty sure that would only be possible with a code change in glance to move the consumption of the swiftclient abstraction up a layer from the client/connection objects to swiftclient's service objects [1]. I'm not sure if that'd be something that would make a lot of sense to the Image Servic

Re: [openstack-dev] leverage page cache in openstack swift

2017-10-06 Thread Clay Gerrard
There's a couple of options in the object server that are related to how object data is cached (or generally *not*) https://github.com/openstack/swift/blob/master/swift/obj/server.py#L921 At scale on dense nodes it becomes challenging to keep all the filesystem metdata in the page cache, so we've

Re: [openstack-dev] Can swift leverage os page buffer?

2017-10-11 Thread Clay Gerrard
On Wed, Oct 11, 2017 at 3:46 PM, Jialin Liu wrote: > Hi, > I'm new to openstack swift, I'm a HPC user, by several days of exploration > of swift, I have some naive questions: > 1. Can swift, e.g., PUT, leverage OS' page buffer?; > Sure, but perhaps to a limited degree depending on what you're ex

[openstack-dev] [tc][election] Question for the candidates

2017-10-12 Thread Clay Gerrard
I like a representative democracy. It mostly means I get a say in which other people I have to trust to think deeply about issues which effect me and make decisions which I agree (more or less) are of benefit to the social groups in which I participate. When I vote IRL I like to consider voting r

Re: [openstack-dev] [tc][election] Question for the candidates

2017-10-12 Thread Clay Gerrard
On Thu, Oct 12, 2017 at 2:48 PM, Emilien Macchi wrote: > > The vision exercise was, in my opinion, one of the more exciting > things we have done in 2017. > Yeah for sure, that was a big goings-on. It's not an easy thing to do because of our diverses opinions, but > together we managed to write

Re: [openstack-dev] [tc][election] Question for the candidates

2017-10-12 Thread Clay Gerrard
I ment to include reference back to (what I believe) was the original work: On Thu, Oct 12, 2017 at 3:17 PM, Clay Gerrard wrote: > > > On Thu, Oct 12, 2017 at 2:48 PM, Emilien Macchi > wrote: > >> >> The vision exercise was, in my opinion, one of the more excitin

Re: [openstack-dev] [tc][election] Question for the candidates

2017-10-12 Thread Clay Gerrard
On Thu, Oct 12, 2017 at 3:20 PM, Clay Gerrard wrote: > I ment to include reference back to (what I believe) was the original work: > https://review.openstack.org/#/c/453262/ __ OpenStack Development Mailing List (n

Re: [openstack-dev] [Swift] SPDK uses Swift as a target system to support k-v store

2017-10-16 Thread Clay Gerrard
Sounds interesting! I'd be *very* interested to see *any* work that's been done to make use of SPDK functionality for faster or more efficient HTTP -> rotational media storage. Couple of thoughts... 1) I think the memcache or redis API would be a better target for SPDK research than Swift's HTTP

Re: [openstack-dev] [tc] [all] TC Report 43

2017-10-24 Thread Clay Gerrard
On Tue, Oct 24, 2017 at 11:26 AM, Chris Dent wrote: > > > Since this is the second [week in a > row](https://anticdent.org/tc-report-42.html) that Josh showed up with > an idea, I wonder what next week will bring? > > ^ That's pretty cool. Thanks for sending this as always Chris. -Clay

Re: [openstack-dev] Swift-backed volume backups are still breaking the gate

2018-01-25 Thread Clay Gerrard
Does it help that swift also had to fix this? https://github.com/openstack/swift/blob/6d2503652b5f666275113cf9f3e185a2d9b3a121/swift/common/utils.py#L4415 The interesting/useful bit is where we replace our primary loghandlers createLock method to use one of these [Green|OS]-thread-safe PipeMutex

Re: [openstack-dev] Swift-backed volume backups are still breaking the gate

2018-01-25 Thread Clay Gerrard
On Thu, Jan 25, 2018 at 7:01 PM, Matt Riedemann wrote: > Is ThreadSafeSysLogHandler something that could live in oslo.log so we > don't have to whack this mole everywhere at random times? That might make sense, unless we can get eventlet's monkey patching of the logging module to do something s

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

2016-08-10 Thread Clay Gerrard
On Wed, Aug 10, 2016 at 7:42 AM, Ben Swartzlander wrote: > > A big source of problems IMO is that tempest doesn't have stable branches. > We use the master branch of tempest to test stable branches of other > projects, and tempest regularly adds new features. > How come not this +1000 just fix t

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

2016-08-10 Thread Clay Gerrard
On Mon, Aug 8, 2016 at 8:31 AM, Matthew Treinish wrote: > When we EOL a branch all of the infrastructure for running any ci against > it goes away. But... like... version control? I mean I'm sure it's more complicated than that or you wouldn't have said this - but I don't understand, sorry. C

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

2016-08-10 Thread Clay Gerrard
On Wed, Aug 10, 2016 at 10:21 AM, Matthew Treinish wrote: > But, to keep the gate running > involves a lot of coordination between multiple projects that are tightly > coupled. Things like an entire extra set of job definitions in zuul, a > branch on > global requirements, a devstack branch, extr

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

2016-08-10 Thread Clay Gerrard
On Wed, Aug 10, 2016 at 10:57 AM, Matthew Treinish wrote: > We also test every incoming > tempest change on all the stable branches, and nothing can land unless it > works > on all supported branches. Did not know that, pretty awesome! > -Clay _

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

2016-08-10 Thread Clay Gerrard
On Wed, Aug 10, 2016 at 10:57 AM, Matthew Treinish wrote: > > http://specs.openstack.org/openstack/qa-specs/specs/tempest/implemented/ > branchless-tempest.html > > > This was actually a *great* read, thanks for that link! -Clay ___

Re: [openstack-dev] [requirements] History lesson please

2016-08-10 Thread Clay Gerrard
On Tue, Aug 9, 2016 at 11:54 AM, Hayes, Graham wrote: > > It might not make a difference to deployers / packagers who only deploy > one project from OpenStack, but they are in the minority - having a > known good minimum for requirements helps deployers who have multiple > services to deploy. >

Re: [openstack-dev] [nova] os-capabilities library created

2016-08-11 Thread Clay Gerrard
On Thu, Aug 11, 2016 at 2:25 PM, Ed Leafe wrote: > > Overall this looks good, although it seems a bit odd to have > ALL_CAPS_STRINGS to represent all:caps:strings throughout. The example you > gave: > > >>> print os_caps.HW_CPU_X86_SSE42 > hw:cpu:x86:sse42 > > Just to be clear, this project doesn

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

2016-08-12 Thread Clay Gerrard
On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja wrote: > > Lets say I was ops evaluating different options as storage vendor for > my cloud and I get told that "Here is the list of supported drivers > for different OpenStack Cinder back ends delivered by Cinder team", I > start looking what the supp

[openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

2016-08-12 Thread Clay Gerrard
The use_untested_probably_broken_deprecated_manger_so_maybe_i_can_migrate_cross_fingers option sounds good! The experiment would be then if it's still enough of a stick to keep 3rd party drivers pony'd up on their commitment to the Cinder team to consistently ship quality releases? What about ma

Re: [openstack-dev] [all][infra] Binary Package Dependencies - not only for Python

2016-08-12 Thread Clay Gerrard
I'd noticed other-requirements.txt around, but figured it needed a bunch of custom tooling to actually make it useful. And... it's a subprocess wrapper to a handful of package management tools (surprised to see emerge and pacmac - Kudos!) and a custom format for describing package requirements...

Re: [openstack-dev] [all][infra] Binary Package Dependencies - not only for Python

2016-08-12 Thread Clay Gerrard
On Fri, Aug 12, 2016 at 11:52 AM, Andreas Jaeger wrote: > > On 08/12/2016 08:37 PM, Clay Gerrard wrote: > > > > ... but ... it doesn't have a --install option? Do you know if that is > > strictly out-of-scope or roadmap or ... ? > > > Right now we don't

Re: [openstack-dev] [all] governance proposal worth a visit: Write down OpenStack principles

2016-09-08 Thread Clay Gerrard
On Thu, Sep 8, 2016 at 6:13 AM, Chris Dent wrote: > > That is, it thinks of itself as an existing truth to be ratified. > Gah! YES!! Exactly this! Well said! And this attitude keeps getting echoed again and again from the current oligarchy TC! "We know what OpenStack is; we know the principl

Re: [openstack-dev] [all] governance proposal worth a visit: Write down OpenStack principles

2016-09-12 Thread Clay Gerrard
On Sun, Sep 11, 2016 at 11:53 PM, Thierry Carrez wrote: > > FWIW I agree with Jay that the wording "a product" is definitely > outdated and does not represent the current reality. "Product" > presupposes a level of integration that we never achieved, and which is, > in my opinion, not desirable at

Re: [openstack-dev] [swift][ironic][ceph][radosgw] radosgw "support" in python-swiftclient droped for ocata and above

2018-05-09 Thread Clay Gerrard
On Wed, May 9, 2018 at 12:08 PM, Matthew Thode wrote: > > * Proper fix would be to make ceph support the account field > Is the 'rgw_swift_account_in_url' option not correct/sufficient? > * Workaround would be to specify an old swiftclient to install (3.1.0, > pre-ocata) > Doesn't seem great

Re: [openstack-dev] [swift][ironic][ceph][radosgw] radosgw "support" in python-swiftclient droped for ocata and above

2018-05-09 Thread Clay Gerrard
On Wed, May 9, 2018 at 12:35 PM, Jim Rollenhagen wrote: > > It works with both, see the link from earlier in the thread: > https://github.com/openstack/ironic/blob/214b694f05d200ac1e2ce6db631546 > f2831c01f7/ironic/common/glance_service/v2/image_service.py#L152-L185 > > Ah! Perfect! Thanks for

Re: [openstack-dev] [osc][openstackclient][zun] Collision on the keyword 'container'

2016-12-20 Thread Clay Gerrard
On Tue, Dec 20, 2016 at 1:00 PM, Hongbin Lu wrote: > > > $ openstack objectstore container > > $ openstack container container > > $ openstack secret container > > > > Thoughts? > This is the closest thing I can see that's somewhat reasonable - with the obvious exception of "container cont

Re: [openstack-dev] [osc][openstackclient][zun] Collision on the keyword 'container'

2016-12-21 Thread Clay Gerrard
On Tue, Dec 20, 2016 at 2:11 PM, Dean Troyer wrote: > > This is exactly how it should work. I do want to make an additional > important but subtle point: while it looks like those are namespaced > commands, we used 'server' not 'compute' because it is not a > compute-namespaced, but a server-spe

Re: [openstack-dev] [infra] [gate] [all] openstack services footprint lead to oom-kill in the gate

2017-02-02 Thread Clay Gerrard
On Thu, Feb 2, 2017 at 12:50 PM, Sean Dague wrote: > > This is one of the reasons to get the wsgi stack off of eventlet and > into a real webserver, as they handle HTTP request backups much much > better. > > To some extent I think this is generally true for *many* common workloads, but the speci

Re: [openstack-dev] GUI for Swift object storage

2018-09-18 Thread Clay Gerrard
I don't know about a good open source cross-platform GUI client, but the SwiftStack one is slick and doesn't seem to be behind a paywall (yet?) https://www.swiftstack.com/downloads There's probably some proprietary integration that won't make sense - but it should work with any Swift end-point.

[openstack-dev] [Swift] Redhat Research

2018-09-25 Thread Clay Gerrard
In one of the Swift sessions at the Denver PTG Doug Hellman suggested there are some programs in RedHat that work with university graduate students on doing computer science research [1] which might be appropriate for certain kinds of work on some parts of OpenStack. Swift has solved a few interes

Re: [openstack-dev] [kolla] warning about PBR issue for kolla operators

2016-09-13 Thread Clay Gerrard
There's a note in the "for development" section [1] that notes the development instructions don't include anything that puts kolla in your sys.path or any bin scripts copied out anywhere into the PATH - i.e. it's not installed That seems less than ideal for a developer - did I miss a `pip install

Re: [openstack-dev] [ptl] code churn and questionable changes

2016-09-22 Thread Clay Gerrard
FWIW, No, this is *not* just an problem for OpenStack https://youtu.be/wf-BqAjZb8M?t=531 ^ Raymond Hettinger Ultimately the problem is mis-aligned goals between the individual and the project maintainers. They want to "do stuff" and get a change landed; we want to maximize the positive results

Re: [openstack-dev] Pecan Version 1.2

2016-09-26 Thread Clay Gerrard
I'm interested to hear how this works out. I thought upper-constraints was somehow supposed to work to prevent this? Like maybe don't install a brand new shiny upstream version on the gate infrastructure test jobs until it passes all our tests? Prevent a fire drill? That bug was active back in J

Re: [openstack-dev] [elections][tc]Thoughts on the TC election process

2016-10-03 Thread Clay Gerrard
On Mon, Oct 3, 2016 at 9:46 AM, Edward Leafe wrote: > After the nominations close, the election officials will assign each > candidate a non-identifying label, such as a random number, and those > officials will be the only ones who know which candidate is associated with > which number. I'm re

Re: [openstack-dev] TC Candidacy

2016-10-03 Thread Clay Gerrard
On Thu, Sep 29, 2016 at 8:34 PM, John Griffith wrote: > > > I think what's more important and critical is the future and where > OpenStack is going over the course of the next few years. > I think this is a really important topic right now! Do you see any dangerous road blocks coming up!? > >

  1   2   >